Article Text

PDF

Self-motivated learning with gamification improves infant CPR performance, a randomised controlled trial
  1. R J MacKinnon1,2,
  2. R Stoeter3,
  3. C Doherty4,
  4. C Fullwood5,
  5. A Cheng6,
  6. V Nadkarni7,
  7. T Stenfors-Hayes8,
  8. T P Chang9
  1. 1Department of Paediatric Anaesthesia, Royal Manchester Children's Hospital, Central Manchester University Hospitals NHS Foundation Trust, Manchester, UK
  2. 2Department of Learning, Informatics, Management and Ethics, Karolinska Institutet, Stockholm, Sweden
  3. 3Department of Paediatric Anaesthesia, Royal Manchester Children's Hospital, Central Manchester University Hospitals NHS Foundation Trust, Manchester, UK
  4. 4Department of Paediatric Anaesthesia, Royal Manchester Children's Hospital, Central Manchester University Hospitals NHS Foundation Trust, Manchester, UK
  5. 5Centre for Biostatistics, Institute of Population Health, University of Manchester, The Manchester Academic Health Sciences Centre and Manchester Biomedical Research Centre, Central Manchester University Hospitals NHS Foundation Trust, The Manchester Academic Health Science Centre, Manchester, UK
  6. 6Department of Pediatrics, Alberta Children's Hospital, University of Calgary, Calgary, Alberta, Canada
  7. 7Department of Anesthesia and Critical Care Medicine and Pediatrics, University of Pennsylvania Perelman School of Medicine, Philadelphia, Pennsylvania, USA
  8. 8Department of Learning, Informatics, Management and Ethics, Karolinska Institutet, Stockholm, Sweden
  9. 9Department of Pediatric Emergency Medicine, Childrens Hospital, Los Angeles, California, USA
  1. Correspondence to Professor RJ MacKinnon, Department of Paediatric Anaesthesia, Royal Manchester Children's Hospital, Hathersage Road, Manchester M13 9WL, UK; ralph.mackinnon{at}cmft.nhs.uk

Abstract

Background Effective paediatric basic life support improves survival and outcomes. Current cardiopulmonary resuscitation (CPR) training involves 4-yearly courses plus annual updates. Skills degrade by 3–6 months. No method has been described to motivate frequent and persistent CPR practice. To achieve this, we explored the use of competition and a leaderboard, as a gamification technique, on a CPR training feedback device, to increase CPR usage and performance.

Objective To assess whether self-motivated CPR training with integrated CPR feedback improves quality of infant CPR over time, in comparison to no refresher CPR training.

Design Randomised controlled trial (RCT) to assess the effect of self-motivated manikin-based learning on infant CPR skills over time.

Setting A UK tertiary children's hospital.

Participants 171 healthcare professionals randomly assigned to self-motivated CPR training (n=90) or no refresher CPR training (n=81) and followed for 26 weeks.

Intervention The intervention comprised 24 h a day access to a CPR training feedback device and anonymous leaderboard. The CPR training feedback device calculated a compression score based on rate, depth, hand position and release and a ventilation score derived from rate and volume.

Main outcome measure The outcome measure was Infant CPR technical skill performance score as defined by the mean of the cardiac compressions and ventilations scores, provided by the CPR training feedback device software. The primary analysis considered change in score from baseline to 6 months.

Results Overall, the control group showed little change in their scores (median 0, IQR −7.00–5.00) from baseline to 6 months, while the intervention group had a slight median increase of 0.50, IQR 0.00–33.50. The two groups were highly significantly different in their changes (p<0.001).

Conclusions A significant effect on CPR performance was demonstrated by access to self-motivated refresher CPR training, a competitive leaderboard and a CPR training feedback device.

  • cardiopulmonary resuscitation
  • training
  • self-motivation
  • game-ification

Statistics from Altmetric.com

Introduction

Paediatric arrests occur more frequently than is often appreciated, with low survival rates and significant neurological sequelae.1 ,2 Provision of high-quality chest compressions for paediatric arrest significantly improves survival outcomes.3–5 A recent meta-analysis of resuscitation studies indicated how significant a determinant high-quality cardiopulmonary resuscitation (CPR) was to survival, showing a strong correlation with appropriate rate and depth of compression.6 However, even among experienced healthcare professionals, high-quality chest compressions do not occur in 36–87% of CPR.7–10

One explanation for poor CPR skill is the current CPR training paradigm of recertification every 2–4 years. Combined with the rarity of paediatric cardiac arrests and rapid decay in resuscitation skills over time,11–14 the provision of more frequent, accessible and user-friendly approaches to CPR training have been explored as an antidote to CPR skill decay. ‘Bedside booster’ CPR sessions with frequent, short bursts of CPR training are more feasible than large-scale, intense courses.15 ,16 Life support instructor assessment of CPR performance has resource implications and is influenced by inter-instructor variability.17 Use of CPR feedback devices has shown that training with real-time feedback improves learning and skill retention,18 ,19 as well as performance of CPR delivery on actual patients.20 Such evidence relates to in-hospital and prehospital CPR survival.10 ,21 ,22

If increasing use of CPR feedback devices is key, the question arises on whether healthcare professionals would be motivated to practise more frequently given the current paradigm of mandated supervised training. To date, there has been limited exploration of harnessing learner-motivated and learner activated CPR practice. Learner activation is a phenomenon by which learners feel engagement, ownership and even heightened responsibility for the skills to be learnt.23 To date, no specific method has been described to motivate frequent CPR practice among healthcare professionals and no method has been described to motivate persistent self-directed practice to prevent skills decay.

To achieve learner activation in CPR practice, we have explored the use of competition and peer pressure as a gamification technique among healthcare professionals. Gamification is defined as the application of typical elements of game playing (eg, point scoring, competition with others, rules of play) to other areas, typically as an online marketing technique to encourage engagement with a product or service. In this study, we have utilised gamification techniques as positive motivators to increase CPR practice and performance on a self-directed CPR feedback device. Gamification to acquire knowledge and skills has been shown in healthcare and in other fields to improve learner activation and engagement.24 ,25 Since current CPR simulators provide quantitative feedback, it provided a method to measure skill improvement or decay over time using a scoring system. We postulated that public display of CPR training scores—such as on a leaderboard—would provide learner activation. Leaderboards are one method of motivating competitive behaviour in gamification and in games in general.26 ,27 We hypothesised that healthcare providers with 24 h access to a self-directed CPR training feedback device with monthly anonymised scores displayed on a leaderboard would both naturally increase frequency of CPR practice and improve their CPR performance.

Objective

The objective of our study was to determine the effect of longitudinal, repeated self-motivated CPR refresher training on CPR performance among paediatric practitioners. We compared CPR performance between those who were provided free access to a training simulator and leaderboard scores, compared to those without such access.

Methods

Trial design

This was a randomised controlled trial over 6 months in which a convenience sample of paediatric providers from two distinct practice areas, ranging from senior consultant physicians to junior nurses and support staff, was enrolled. On enrolment, baseline CPR performance data were collected on all participants using the same infant CPR feedback device (Laerdal infant QCPR manikin; Stavanger, Norway). Intervention group subjects with free access to the device were asked to submit their final scores at the end of the study period; control group participants without free access to the device were invited back after 6 months to submit final CPR performance data. The purpose of the study was to determine the effect of open access to a CPR training device and leaderboard on CPR practice and subsequent performance.

Ethical considerations

The local institutional review board approved this study.

Study setting and participants

The study took place at two healthcare delivery settings in a single UK tertiary children's hospital for 6 months starting January 2014. Participants consisted of staff from the postanaesthetic clinical unit in theatres (PACU) and the paediatric intensive care unit (PICU) (table 1). The PACU/theatres and PICU employ approximately 250 full-time staff. Inclusion criteria for enrolment included full-time employment and current certification in basic life support (BLS). Trainees were not excluded from this study.

Table 1

Demographics of the study participants

Randomisation

All subjects were randomised to either the intervention or control group for the entire duration of the study on enrolment. Stratified randomisation was accomplished using a random number generator. Stratification took place by healthcare professional role (junior nurse, senior nurse, trainee or consultant physician).

The self-motivation-based intervention

The intervention group was given 24 h/day, 7-days-a-week access to an infant CPR feedback training device consisting of a CPR manikin (Laerdal infant QCPR manikin; Stavanger, Norway) and an attached feedback device (Laerdal SkillGuide). Each manikin was located within the PACU or PICU within view of patients and families, accessible to all subjects without leaving their area of clinical practice. Instructions were provided for the intervention group. Each device recorded performance data that were displayed as ventilation and compression scores on the SkillGuide following 2 min of ventilation compression infant CPR on the manikin.28 Subjects were instructed to document their scores using a study identification number and submit the score to a collection device. They were allowed multiple attempts and unlimited submissions throughout the duration of the study. Participant scores were collated and ranked anonymously on leaderboards in the two clinical areas. The leaderboards were updated on a monthly basis. No reminders were sent to the intervention group, which was not proctored, supervised or forced to continue practising on the device. There was no tangible reward for increasing practice, nor a penalty for total avoidance.

Control

The control group participants could see the CPR devices in the workplace at all times but were informed that they could not use the devices. No preventative measures were in place to physically stop the control group from using the devices located within their clinical areas.

Study outcomes and definitions

We defined the outcome variable of CPR performance as the total composite score (TotS) of two measurements: ventilation score (VentS) and compression score (CompS). Both VentS and CompS were expressed as a percentage, ranging from 0 to 100 and were calculated by software within the SkillGuide with real-time data provided by the CPR device.28 VentS is calculated on the basis of the appropriate rate and volume of ventilation required for Paediatric Advanced Life Support for infants (2 breaths for every 15 compressions).29 CompS is amalgamated from the appropriate rate (100–120 compressions/minute), appropriate depth and appropriate full recoil. TotS is a mean score (0–100) of VentS and CompS. In addition to the outcome variables, we collected demographic variables including: Healthcare professional role (junior nurse, senior nurse, trainee, consultant physician), rank as defined by the band of the provider or seniority of the nurse, department, proximity of last BLS, advanced paediatric life support (APLS) training and last actual CPR on a patient.

Sample size

We aimed to detect a 10% difference in CPR performance (TotS) between the intervention and control groups. A sample size of 140 participants (70 in each arm) would provide a detectable effect size of 0.48 with an α of 0.05 and 80% power.

Statistical analyses

Descriptive statistics were used to summarise the subject characteristics of each group, and χ2 tests to check for any significant demographic differences between groups. A Mann–Whitney U test was used to verify that there was no significant difference between baseline scores.

For the primary outcome, we compared change from baseline between the two groups using descriptive statistics and a Mann-Whitney U test. Secondary analysis looked at the 6-month scores via descriptive statistics and paired Wilcoxon signed-rank tests to examine for a significant change.

Analyses were performed in R V.3.1.1.30

Results

Study participants

A total of 90 subjects were enrolled in the intervention group and 81 in the control group, shown in table 1. All 171 subjects provided baseline data. Thereafter, 24 (27%) subjects from the intervention group and 25 (31%) subjects from the control group dropped from the study. All dropouts were voluntary (eg, maternity leave, change of job). The apparent imbalance in the dropouts relates to those in the intervention group (n=18) who were not self-motivated to continue on the feedback device after their first baseline score at time zero. In addition, in the control arm, we were unable to track participants at 6 months (n=17) for their 6-month score, as they were on holiday, maternity or sick leave. Figure 1 illustrates the CONSORT diagram for our study.31

Figure 1

Flow diagram of study design and participant enrolment modified from Consort Transparent. Reporting of Trials.31

Study outcomes

Within the intervention group, 66 (73%) used the CPR feedback device more than once during the study period.

Primary outcome

Overall, the control group tended to show little change in their scores (median 0, IQR −7.00–5.00) from baseline to 6 months, while the intervention group had a slight median increase of 0.50, IQR 0.00–33.50. The two groups were highly significantly different in their changes (p<0.001) figure 2.

Figure 2

Box plot to show changes between control and intervention groups at baseline and 6 months.

Secondary analysis

There was no difference in baseline CPR performance between the intervention group and control group (intervention median 55.00, IQR 38.25–67.50, control median 47.00, IQR 33.00–63.00, p=0.186)). The change from baseline to 6 months was significant in the intervention group (p<0.001), while the control group did not show a significant change (p=0.475).

Discussion

Our study is one of the first to describe the effects of self-motivation on infant CPR practice using a CPR feedback device, in conjunction with a leaderboard. We demonstrate that overall infant CPR performance on the simulator improves with increased practice despite a lack of any observer, instructor or supervisor, and without an external reward or punishment. This is a novel method of motivation that is capable only because of the technologies allowing for an objective, measurable feedback of CPR quality. Cheng et al10 ,32 noted that even with trained supervision and observation, the reliability of human observation on CPR quality is inferior to CPR simulators that provide this level of feedback, and that the presence of neutral feedback does improve CPR quality. The leaderboard then provides a different level of feedback that provides a more longitudinal performance metric in the form of learner activation.

Gamification techniques to improve social competition—termed points, badges, leaderboards33—have been present in digital education, particularly with schoolchildren, but have also been described in the medical education literature among adult healthcare providers. Leaderboards in particular were reported as the most important motivators for cognitive learning among U.S. internal medicine residents.26 These are examples of learner activation, and gamification can increase the perceived value of the knowledge or skill to be gained.

Despite the life-saving potential of CPR, the inherent task-value of the CPR training is still undervalued among healthcare providers who perceive CPR as a certification to complete.34 CPR training improves confidence and often knowledge, but not always proportionately to performance,35–37 and this incongruence creates a barrier to motivating frequent practice. Providing a leaderboard and concrete, objective feedback also generates further incentive, as providers consistently overestimate the quality of CPR being provided during cardiac arrest.32 Our data support the assertion that gamification techniques can incentivise CPR training36 and also facilitate self-guided or self-directed practice.38 The literature demonstrates that these features allow for successful learning and skills retention.34 ,38–40 We demonstrate the association between increased practice and improved CPR performance, consistent with other studies that increase CPR practice frequency15 ,16; however, we showed this improvement in simulated CPR performance without the active supervision or proctoring with instructors or research assistants.

The strengths of this study include the feasibility of our leaderboard technique, as well as its generalisability to almost any unit or in-hospital or in-clinic setting, be it neonatal, paediatric or adult medicine.

Our study does have several limitations. We relied on self-reported scores; this meant scores could have been fabricated or not reported. However, we believe that this was relatively minimised because no actual reward beyond a higher leaderboard position was provided for a higher score. Ultimately, we measured only performance on the CPR simulator and did not report patient-based outcomes in this study. We would expect those who practised on the simulator to perform better than peers less familiar with it—essentially a null comparator study.41 However, our innovation was spurring spontaneous practice on the CPR feedback device using a leaderboard, the latter of which was the primary intervention in this study, not the manikin. Furthermore, we observed that the motivation gradually waned over the 6-month period, and the study did not assess skill improvement or decay past the 6-month period. It is also possible that the relatively high dropout rate, though similar in both arms, affected the results. Finally, the control group was not actively prevented from practising on the CPR feedback device throughout this study. Such a control group practice would have increased the performance of the control group arm at 6 months. Only one control group member reported scores.

In conclusion, evidence generated by our randomised controlled trial indicates that overall infant CPR performance on the simulator improves with increased practice spurred by a simple leaderboard in the intervention group over the 6-month period (p<0.001), compared with the control group. Our data suggest that longitudinal, self-motivated, rolling refresher CPR training with integrated CPR feedback can improve the quality of CPR over time. This study adds further evidence that interactive CPR manikins providing quantifiable scores can promote self-directed learning in motivated individuals.

References

View Abstract

Footnotes

  • Twitter Follow Ralph MacKinnon at @MacKinnonSIM and Todd Chang at @drtoddchang

  • Contributors RJMK conceptualised the research question and study design, co-performed the study, contributed intellectual content, drafted the initial manuscript, reviewed and revised the manuscript. RS and CD co-performed the study, contributed intellectual content, reviewed and revised the manuscript. CF contributed to the study design, statistical analysis and reviewed the manuscript. TPC, TS-H, AC and VN contributed intellectual content and reviewed and revised the manuscript. All authors approved the final manuscript as submitted.

  • Competing interests RJMK, TPC AC and VN are members of the INSPIRE network, which receives infrastructure support from the Laerdal. Foundation for Acute Medicine, project funding from Rbaby. Foundation, and funding for biannual meetings from the Society for Simulation in Healthcare and the International Pediatric Simulation Society.

  • Ethics approval Ethics opinion was sought from the research host organisation. In the UK, in accordance with current governance guidelines for health research, studies on NHS staff and service provision, this research project does not require NHS Research Ethics Approval. All participating individuals consent to do so and are provided with a participant information document.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Data sharing statement The authors will respond to reasonable requests.

Request permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.