Test-retest repeatability of the National Animal Health monitoring system dairy heifer health report in New York and Pennsylvania, USA

We did a test-retest study on 21 farms to assess the repeatability of the Dairy Heifer Health Report of the National Dairy Heifer Evaluation Project. The median retest interval was 42 days (range 14–63 days), retest interviewers were blinded as to responses on the first visits, and all date-specific questions were anchored to the dates of the first tests. The overall discrepancy percentage was 8.5% (95% CI 8.0–9.0). Individual farms had discrepancy percentages of 3.0–15.2%; 240 out of the 539 questions had no discrepancies at all, but 81 questions had more than four (out of 21 possible) discrepancies (lowest percentages among the 81: 24%; 95% lower confidence limit on 24%: 6%). Combining ‘no’ and ‘not applicable’ responses for six similarly worded questions in the Biosecurity section would reduce the overall questionnaire discrepancy percentage to 7.8% and would bring that section's discrepancy percentage down to 8.6%. The other discrepancy percentages by section were: Maternity Hygiene, 14.1 %; Preweaning Hygiene, 4.8%; Disease Agents, 3.6%; Vaccination Practices, 12.1%; Preventive Practices, 16.8%. Discrepancy percentages did not differ importantly by type of question (independent, filter, or subordinate) or by type of response required (categorical, count, or measurement). Discrepancy percentages were independent of retest interviewer, test-retest interval and order of test within state (a proxy for time since training of the interviewers) but were correlated with herd size (r_sp = 0.59).