Inter-observer agreement between two observers for bovine digital dermatitis identification in New Zealand using digital photographs

Aims: To assess the inter-observer agreement for detecting bovine digital dermatitis (BDD) lesions in digital colour photographs of the hind feet of cows, which had been taken while the animals were standing to be milked, between two trained observers.

Methods: Thirty-six photographs were selected from a total of 184 photographs held by the first author (R1), who had classified them as negative (n=11) or positive (n=25) for BDD. They were delivered to a technician (R2) who had previously visually inspected cattle for BDD lesions, and who then recorded the photographs as being either BDD-positive or BDD-negative. The percentage agreement between R1 and R2, and two other inter-observer agreement statistics, Cohen’s κ and Gwet’s first-order chance correction agreement coefficient (AC1), were calculated. The cumulative membership probabilities of Cohen’s κ and Gwet’s AC1 were then calculated for different benchmark ranges of κ.

Results: The percentage agreement between R1 and R2 was 33/36 (92%), Cohen’s κ was 0.80 (95% CI=0.57–1.0) and Gwet’s AC1 was 0.86 (95% CI=0.69–1.0). Based on the cumulative membership probabilities for Gwet’s AC1, there was 75% probability that the two observers had almost perfect agreement (κ≥0.81). For both Cohen’s κ and Gwet’s AC1, there was >95% probability that the two observers had at least substantial agreement (κ≥0.61).

Conclusions: The two trained observers had at least substantial agreement in identifying from a digital photograph as to whether BDD lesions were present or absent. Therefore results from the two could be used interchangeably.

Clinical Relevance: Visual assessment for BDD lesions in the milking parlour can be subjective. However a high agreement between these two trained BDD inspectors means BDD prevalence reported from different regions in New Zealand by these two can be directly compared.