Introduction Understanding performance differences between learners may provide useful context for optimising medical education. This pilot study aimed to explore a technique to contextualise performance differences through retrospective secondary analyses of two randomised controlled simulation studies. One study focused on speaking up (non-technical skill); the other focused on oxygen desaturation management (technical skill).
Methods We retrospectively analysed data from two independent simulation studies conducted in 2017 and 2018. We used multivariate hierarchical cluster analysis to explore whether participants in each study formed homogenous performance clusters. We then used mixed-design analyses of variance and χ2 analyses to examine whether reported task load differences or demographic variables were associated with cluster membership.
Results In both instances, a two-cluster solution emerged; one cluster represented trainees exhibiting higher performance relative to peers in the second cluster. Cluster membership was independent of experimental allocation in each of the original studies. There were no discernible demographic differences between cluster members. Performance differences between clusters persisted for at least 8 months for the non-technical skill but quickly disappeared following simulation training for the technical skill. High performers in speaking up initially reported lower task load than standard performers, a difference that disappeared over time. There was no association between performance and task load during desaturation management.
Conclusion This pilot study suggests that cluster analysis can be used to objectively identify high-performing trainees for both a technical and a non-technical skill as observed in a simulated clinical setting. Non-technical skills may be more difficult to teach and retain than purely technical ones, and there may be an association between task load and initial non-technical performance. Further study is needed to understand what factors may confer inherent performance advantages, whether these advantages translate to clinical performance and how curricula can best be designed to drive targeted improvement for individual trainees.
- communication skills
- cognitive load
- simulation education
- non technical skills
Data availability statement
Data are available upon reasonable request. Raw data available from the authors and upon reasonable request made in writing.
Statistics from Altmetric.com
Presented at Accepted for abstract presentation at the Society for Simulation in Europe Annual Meeting 2021.
Contributors RJDG helped with the conception and design, data interpretation, drafting and revision of the manuscript, and final approval of the version to be published, and agreed to be accountable for all aspects of this work. CRM and AS helped with the conception and design, data interpretation, revision of the manuscript and final approval of the version to be published, and agreed to be accountable for all aspects of this work. ST helped with the conception and design, data analysis and interpretation, drafting and revision of the manuscript and final approval of the version to be published, and agreed to be accountable for all aspects of this work.
Funding The original DESAT study was supported by an intramural StAAR Grant from the Johns Hopkins Department of Anesthesiology and Critical Care Medicine.
Competing interests None declared.
Provenance and peer review Not commissioned; externally peer reviewed.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.