Table 1

ICCs between expert debriefing evaluators, debriefers and learners evaluations across OSAD elements

OSAD elementLearners and debriefersLearners and expert debriefing evaluatorsDebriefers and debriefing expert evaluators
Approach0.15NS0.28*0.12NS
Learning environment0.20*0.01NS0.01NS
Learner engagement0.23NS0.45**0.22NS
Reaction0.13NS0.03NS−0.10NS
Reflection0.14NS0.43**0.35**
Analysis0.14NS0.21*0.14NS
Diagnosis0.11NS0.26*−0.02NS
Application0.40**0.18NS0.16NS
  • *p<0.05; **p<0.01.

  • ICC interpretation: lack of agreement 0.00–0.30; weak agreement 0.31–0.50; moderate agreement 0.51–0.70; strong agreement 0.71–0.90; very strong agreement 0.91–1.00 (LeBreton and Senter).16

  • ICC, intraclass correlation; NS, not statistically significant; OSAD, Objective Structured Assessment of Debriefing.