Introduction The cornerstone of effective training is constructive, objective feedback that facilitates improvements in performance, however traditional subjective assessments by trainers can at times be unreliable and prone to bias. Global Rating Scales are observational assessment tools, which allow the expert observer to rate trainees impartially in a structured manner. The aim of this study was to develop an orthopaedic trauma global rating scale and then validate it using a trauma simulator.
Methods A Delphi method was used to develop a detailed global rating scale for assessing performance in orthopaedic trauma surgery. Consultant orthopaedic trauma surgeons participated in a focus group and online questionnaires to generate the scale points to cover open surgery and fluorosocopic techniques. This scale was then validated using a haptic VR trauma simulator with participants performing dynamic hip screw procedures. They were filmed using a dual camera recording system to synchronously record the simulator screen and anonymised hand and body movements. These videos were then assessed using the new scale by two independent experienced trauma surgeons, blinded to the identities of the participants.
Results 18 consultant orthopaedic trauma surgeons participated in the Delphi study. The global ratings scale comprised 10 dimensions, each with further descriptors of novice, intermediate and expert performance. 30 participants completed the study: Experts (Consultants n = 10), Intermediates (Orthopaedic registrars; n=10) and Novices (medical students n = 10). The scale demonstrated high internal consistency (Cronbach alpha 0.91). The intraclass correlation coefficient demonstrated high agreement between the assessors: 0.89 (p < 0.05). Construct validity was evaluated using the Kruskal-Wallis one-way analysis of variance (p < 0.05).
Conclusion This new orthopaedic trauma global rating scale has high internal consistency, excellent inter-rater reliability and construct validity. This study demonstrates that this global ratings scale is a robust and reliable method of assessing orthopaedic trauma skills and offers impartial formative assessment and feedback.
Statistics from Altmetric.com
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.