Article Text

PDF
Validation evidence of the paediatric Objective Structured Assessment of Debriefing (OSAD) tool
  1. Jane Runnacles1,
  2. Libby Thomas2,
  3. James Korndorffer3,
  4. Sonal Arora4,
  5. Nick Sevdalis5
  1. 1Department of Paediatrics, Royal Free London NHS Foundation Trust, London, UK
  2. 2Division of Medical Education, King's College, London, UK
  3. 3Department of Surgery, Tulane University, New Orleans, Louisiana, USA
  4. 4Department of Surgery and Cancer, Imperial College, London, UK
  5. 5Health Service & Population Research Department, Centre for Implementation Science, King's College, London, UK
  1. Correspondence to Dr Jane Runnacles, Department of Paediatrics, Royal Free London NHS Foundation Trust, Pond Street, London NW3 2QG, UK; jane.runnacles{at}gmail.com

Abstract

Introduction Debriefing is essential to maximise the simulation-based learning experience, but until recently, there was little guidance on an effective paediatric debriefing. A debriefing assessment tool, Objective Structured Assessment of Debriefing (OSAD), has been developed to measure the quality of feedback in paediatric simulation debriefings. This study gathers and evaluates the validity evidence of OSAD with reference to the contemporary hypothesis-driven approach to validity.

Methods Expert input on the paediatric OSAD tool from 10 paediatric simulation facilitators provided validity evidence based on content and feasibility (phase 1). Evidence for internal structure validity was sought by examining reliability of scores from video ratings of 35 postsimulation debriefings; and evidence for validity based on relationship to other variables was sought by comparing results with trainee ratings of the same debriefings (phase 2).

Results Simulation experts’ scores were significantly positive regarding the content of OSAD and its instructions. OSAD's feasibility was demonstrated with positive comments regarding clarity and application. Inter-rater reliability was demonstrated with intraclass correlations above 0.45 for 6 of the 7 dimensions of OSAD. The internal consistency of OSAD (Cronbach α) was 0.78. Pearson correlation of trainee total score with OSAD total score was 0.82 (p<0.001) demonstrating validity evidence based on relationships to other variables.

Conclusion The paediatric OSAD tool provides a structured approach to debriefing, which is evidence-based, has multiple sources of validity evidence and is relevant to end-users. OSAD may be used to improve the quality of debriefing after paediatric simulations.

  • feedback
  • simulation
  • paediatric

Statistics from Altmetric.com

Request permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.