Article Text

PDF

Impact of providing case-specific knowledge in simulation: a theory based study of learning
  1. Jocelyn Cox1,
  2. Marion McGregor2,
  3. Dominic Giuliano3,
  4. Loretta Howard4
  1. 1Department of Chiropractic, D'Youville College, Buffalo, New York, USA
  2. 2Canadian Memorial Chiropractic College, Toronto, Ontario, Canada
  3. 3Undergraduate Education, Canadian Memorial Chiropractic College, Toronto, Ontario, Canada
  4. 4Leadership, Higher and Adult Education, University of Toronto Ontario Institute for Studies in Education, Toronto, Ontario, Canada
  1. Correspondence to Dr Dominic Giuliano, Canadian Memorial Chiropractic College, 6100 Leslie Street, North York ON M2H 3J1, Canada; dgiuliano{at}cmcc.ca

Abstract

Background Simulation-based education (SBE) has been lauded for its ability to help students recognise and react appropriately to common and rare circumstances. While healthcare professions have started to implement SBE into their curriculum, there is no evidence to suggest which educational theory is best for implementation. This study explores the usage of cognitive load theory (CLT) and the unified theory of emotional learning (UTEL).

Study design A mixed methods ordered-allocation cohort study.

Methods 23 patient management teams were allocated into 2 groups. The first group received prior information about the simulation scenario; the second group did not. Each team had 1 student assigned to the role of doctor. The scenarios were filmed at time 1 (T1), time 2 (T2) and follow-up (F/U). The ‘doctor’ role was then graded with a validated checklist by a three-judge panel. The scores were evaluated to determine if prior information enabled better performance. Secondary analysis evaluated the role of gender on performance and also evaluated anxiety at the onset of the simulation.

Results 23 doctors were evaluated. There was no difference between groups in performance (t=1.54, p=0.13). Secondary analysis indicated that gender did not play a role. There was no difference in anxiety between groups at baseline (t=0.67, p=0.51).

Conclusions Trends were observed, suggesting that when students enter a simulation environment with prior knowledge of the event they will encounter, their performance may be higher. No differences were observed in performance at T2 or F/U. Withholding information appeared to be an inappropriate proxy for emotional learning as no difference in anxiety was observed between groups at baseline. All trends require confirmation with a larger sample size.

  • Simulation
  • Health Care
  • Education
  • Learning Theory

Statistics from Altmetric.com

Introduction

Heart attacks continue to be the leading killer of Canadians.1 There is a cardiac event every 7 min with more than 23 000 Canadians dying yearly due to their mismanagement.1 ,2 For every minute that CPR is delayed, the rate of survival decreases by 7–10%.3 ,4 Use of an automated external defibrillator within 4 min of a cardiac arrest improves survival rates in a variety of settings and populations.5 Emergency room incidents of cardiac events have increased by 66% in the past 20 years, at a cost of $20.9 billion yearly.2 The majority of deaths related to myocardial infarcts occur prior to arrival at the hospital.6 According to Fritz,7 50–60% of cases of mismanagement are due to human error. Human error in healthcare has been attributed to uncertainty in diagnosis, prognosis and all phases of intervention.8

The ability of healthcare professionals to effectively manage crises increases over time with experience.9–11 Management of these skills tends to degrade over time without practice.12 A shortfall in healthcare education is the expectation that students will transfer their recall knowledge from the classroom into practical application.13–15 However, by creating situation-specific environments for students to develop their experiential understanding, students gain experience in accurately managing crises.7 ,16 ‘High-fidelity’ manikin simulation (HFMS) has been developed to present common as well as complex or uncommon conditions that allow students the chance to experience failure without risking an actual patient's life.10 ,11 ,17

There are two theories that speak directly to what should be modified and how in a learning environment to achieve this objective.

The first is the cognitive load theory (CLT) and the second is the unifying theory of emotional learning (UTEL). Both are competing theories regarding the best practice in educating students. CLT relates to the executive control of working memory. It supports that as the cognitive load on the student increases, owing to scenario or instructional complexity, the ability to understand and retain the available information decreases.18 The brain may perceive a novel situation as too complex or stressful, thus overloading the finite amount of working memory available to students for learning and hindering their ability to retain information.13 Stress affects how knowledge is encoded as the student learns,19 and having too much stress can hinder the storage and retrieval of information.19 ,20 For this reason, instructors provide students with foresight into the upcoming lesson, to help the student prepare and become more familiar with the material to prevent increased cognitive demand. Inability of students to effectively manage a crisis in the environment may be traced back to a lack of experience in education.11 This further promotes the need to integrate ‘high-fidelity’ role-playing experiences to afford students the ability to practise effectively managing crises when they arise in professional practice. On the basis of CLT, medical, nursing and paramedic programmes currently provide students with information or ask them to prepare for the event to be simulated beforehand.21 ,22 This is intended to prevent overloading working memory and to enhance a student's performance in their experiential encounter.

The UTEL approach recognises the impact of emotion on the environment and includes those social interactions that would be expected to occur if the clinical experience is happening in real life.15 ,20 This theory suggests that optimally heightened emotions stimulate increased attention thus allowing for increased learning. This is supported in the work by DeMaria et al12 that indicated that students managing a cardiac event who had increased anxiety compared with controls tended to have increased performance 6 months later.

It also recognises that there is a fine balance between being underaroused and overaroused.23 ,24 If overaroused (or too emotional), a person may mentally shut down and be unable to apply themselves to the learning opportunity.20 However, if harnessed, emotion can be viewed as a catalyst for learning, recognising that an actual human encounter can generate a memory that shapes future practice.19 As such, withholding information regarding the simulated event may provide a valuable opportunity to address the practitioner's limitations. It has been suggested by some authors that these two theories are correlated.25

Fraser et al25 identified that in a simulation modelled after the CLT, managing emotion and stress throughout multiple phases of the simulation is important to a successful execution of the scenario. However, the CLT does not speak to emotion in the framework of the theory. The benefit of combining two theories is to ideally address where one theory may have limitations; however, the risk of combining theories may result in inadequately parsing out how different variables can impact the educational system.

Finally, gender may play a role in the utility of UTEL. It is understood that women prefer concrete experiences while men prefer abstract concepts.26 As such, it may be that women learn better when prior knowledge is provided. Although HFMS is successful in training students to recognise mistakes, address mismanagement and contribute collaboratively to a working team in real-time scenarios, the theoretical basis of their implementation remains undetermined.27 Currently, there is no evidence suggesting that a CLT focus of providing prior knowledge provides superior learning and retention in comparison to the UTEL approach that would be consistent with withholding knowledge. Given the risk associated with emergency situations, it is imperative to test which approach best enables students to draw on their history to quickly and appropriately execute the response in a timely manner.

This study aims to identify whether CLT (prior information) or UTEL (no prior information) has a greater influence on learning retention in HFMS.

Methods

A mixed methods approach was used in this ordered allocation cohort study approved by the institution's Research Ethics Board (REB1307X11). Qualitative data were based on self-observations by participants, and questionnaires and checklists were used to assess outcomes. Students in this investigation were fourth year interns at a chiropractic college. The college recognised the need to provide students with exposure to rare experiences in order to reduce novelty and decrease time to action when management is necessary.

Students were required to participate in each laboratory as a mandatory component of their curriculum. There were 175 participants split between 23 patient management teams (PMTs). These PMTs were predetermined clinical placements groups. Each PMT averaged eight students. All participants had previous experiences in the manikin simulation laboratory. No student had experience with the scenario evaluated in this investigation.

Each PMT participated in a 2-hour simulation experience. For CLT, prior information was provided 5 days in advance of the emergent scenario to 11 of the PMTs. Although UTEL does not provide a method for implementation, the first 12 PMTs were not provided prior information as a proxy for heightened emotion intended to capitalise on a ‘real-life’ scenario where the students would be unaware of the crisis they would have to manage in advance.

When at the laboratory, each student was provided a role. The available roles were: doctor, receptionist, family member, massage therapist or patient. Roles were used to ensure participation for each member of the PMT. Previous research indicates that role does not have an impact on learning in simulation experiences.28 Student roles were assigned in a blinded fashion by asking participants to select from overturned clipboards containing scenario assignments. All participants with the exception of the ‘doctor’ were advised of their role and character in the scenario and provided an opportunity for questions. The student playing the role of ‘doctor’ participated as though the laboratory was their private clinical setting. The high-fidelity manikin used was a Gaumard Scientific Susie S300. It was programmed to lose vital signs during the ‘doctor's’ history with Susie.

The scenario was then undertaken and video recorded. This recording was identified for study purposes as time 1 (T1). At the end of the scenario, the recording was stopped and a debrief was provided. Each intended learning outcome of the experience was discussed in a collaborative and interactive manner with examples. At the end of the debrief, the students engaged in the same scenario with the same roles, and this was again recorded. The procedure of including a repeat simulation is a standard at this educational institution and has been used to ensure that students playing the role of the doctor are able to demonstrate the learning gains made with their peers. This second recording was identified as time 2 (T2). After another short debrief, the laboratory was concluded.

Students were not notified at that time that they would be asked to complete a 6-month follow-up (F/U). Six months later, all student PMTs were brought back for an additional simulation experience. Once they entered the laboratory, students were asked to reflect on their previous laboratory experiences. For study purposes, this was defined as F/U. For the F/U, all students playing the role of ‘doctor’ at T1 and T2 were retained in the ‘doctor’ role. The F/U laboratory took ∼30 min and the scenario was again video recorded.

The primary outcome was a previously validated checklist,29 using a three-judge panel. The judges were asked to evaluate the video recordings of the scenarios that were randomised by a random number generator. Judges were blinded as to video type and to each other. The ‘doctor’ was evaluated on their ability to appropriately manage their patient in crisis and their clinical environment, based on a 16-question checklist, worth up to 18 points.

Anxiety was measured using a visual analogue scale (VAS) to assess all students’ perceived anxiety at the time when their role as doctor was revealed.30 ,31 After T1, students were asked to complete a ‘1-min paper’ commenting on their preparation for the laboratory.32 This was used to begin discussion regarding the pre-established learning objectives.

Secondary analysis included evaluation of the F/U videos to assess potential learning retention by group. In addition, groups were evaluated according to gender using descriptive statistics and Student's t-tests to search for issues related to gender and learning to inform hypothesis testing for future studies.

Analysis

All data were entered into a spreadsheet and imported into Stata 8 (StataCorp. 2004. Stata Statistical Software: Release 8. College Station, Texas, USA: StataCorp LP). For the primary outcome, a two-tailed, unpaired Student's t-test was used to evaluate for the difference in video score between groups at T1. VAS for anxiety was also assessed using a Student's t-test score. An α level of 0.025 was used to account for multiple uses of the Student's t-test.

A difference of proportions test was used to determine if the number of students who prepared who were provided prior information was comparable to those who prepared who were not provided information. Gender differences were secondarily assessed through descriptive measures and Student's t-tests. Results for T2 and F/U are provided using descriptive measures for consideration in future work.

Results

There were a total of 175 participants (86 females, 49%) in this study, divided between 23 PMTs. Within each PMT, 1 doctor role was assigned for a total of 23 doctors. Of the 23 doctors, 11 (48%) were female. There were 12 PMTs who were not provided prior information, and 11 PMTs who were. Each PMT had three video recordings (T1, T2 and F/U) for a total of 69 video recordings available for analysis. Table 1 provides the checklist scores (means and SDs) at all of the time points. Student's t-tests to evaluate for difference in scores between doctor groups at T1 yielded a t-value of 1.54 (CI −5.48 to 0.82, p=0.14). The Student's t-test for anxiety scores at the time of role revelation indicated no statistically significant difference by information groups with t=0.67 (CI −1.16 to 2.25, p=0.51).

Table 1

Performance value per PMT doctor by time

The ‘preparation’ results at T1 by information group indicated that 8/11 PMT doctors who were provided prior information prepared in some way for the simulation experience. Only 4/12 PMT doctors who were not provided prior information prepared for the simulation experience. Difference of proportions test resulted in z=1.89, p=0.06 for these data.

Table 2 provides the descriptive analysis of video scores for T1 accounting for information group and gender. Both genders appeared to have slightly higher scores when information was provided. A trend towards a difference between groups (prior information, no period information) by gender was observed for women at T1 (t=1.68, CI −7.61 to 1.12, p=0.13). No difference between groups was observed for men (t=0.59, CI −7.01 to 4.08, p=0.57).

Table 2

Gender differences by information group at time 1

Discussion

HFMS provides students with a safe learning environment in which to challenge their clinical competencies.9 ,10 The goal of education remains to provide learners with the strongest and most efficient means of developing their skills. A strong theoretical framework is critical to support this goal. The two competing theories evaluated in this work contrast perspectives that can be related to student preparation for the simulation environment. CLT suggests that student preparation will liberate working memory and therefore provide greater efficiency during a learning exercise.33 UTEL, on the other hand, would suggest that stronger emotions, perhaps associated with a lack of preparation, may result in more powerful memories and therefore greater long-term retention.15

Data from this study indicated that students who were provided information trended towards more frequently preparing for the simulation event compared with students who were not provided information (z=1.89, p=0.06). Along with this, there was a potential trend in stronger performance at T1 when prior information was provided (t=1.54, p=0.14). Power analysis resulted in a β-level of ∼0.35, suggesting a sample size for future studies of ∼38 per arm. Mean scores provided in table 1 were out of a potential total of 18 points associated with the checklist used to evaluate videotaped performances.29 Translating mean scores to percentages, the average percentage at T1 for the group with prior information was 63%, while the average percentage for the group with no prior information was 50%. Interestingly, again converting the scores in table 1 to percentages, after the simulation and debrief intervention, learning scores at T2 were identical at 89%. At the 6-month F/U, there was no statistically significant difference (t=0.93, p=0.36), and learning scores for the group with prior information was 80% while scores for the group with no prior information was 74%.

At T1, with 63% and 50% of learning outcomes met, the results suggest that only providing a single simulation per session may provide limited benefits overall, especially if it is not followed by a debrief.34 ,35 Much greater learning (average of 89% learning outcomes achieved) appeared to have occurred (and been sustained) when the debrief occurred and the simulation exercise was repeated.

Kulturel-Konak et al36 reviewed work identifying gender differences that should be considered when constructing science, technology, engineering and mathematical educational environments. In addition, there may be substantial sex-related influences on neural mechanisms for an emotional event based on the hormonal effect stress has on memory consolidation.37 Our study suggested that higher scores at T1 appeared to occur with women where prior information was provided (using the more traditional CLT approach) (t=1.68, p=0.13). Although these results were not statistically significant, power was substantial at 0.73, suggesting that another study with 15 participants per gender group would be beneficial for future investigation of this issue.

This study was limited by using ‘no prior information’ as the chosen proxy for UTEL. It was believed at the outset that when students are not given information regarding the expected scenario, anxiety levels would be higher. Heightened (but not excessive) anxiety was intended to increase awareness and therefore retention.15 However, results showed no relationship between information groups and anxiety scores (t=0.67, p=0.51). Thus, there is a question as to whether the UTEL proxy was sufficient to test the theory.

Conclusion

Trends were observed suggesting that when students enter a simulation environment with prior knowledge of the event they will encounter, their performance may be higher. No differences were observed in performance after the simulation experience or at the 6-month F/U. However, strong and sustained learning gains after reinforcement of the simulation exercise suggests that simulation laboratories tests should run with debriefing.

Additional trends associated with gender highlight possible learning differences between men and women. All trends, however, require confirmation with a larger sample size.

References

View Abstract

Footnotes

  • Contributors All authors were involved in the planning, conduct and/or reporting of this study. JC prepared the study proposal, collected data, conducted simulation laboratory tests and created the manuscript. MM served as a scientific advisor, critically reviewed and appraised the study proposal, performed the statistics and revised the final manuscript. DG conducted simulation laboratory tests and revised the final manuscript. LH served as a scientific advisor, critically reviewed and appraised the study proposal, and revised the final manuscript.

  • Competing interests None declared.

  • Ethics approval Canadian Memorial Chiropractic Research Ethics Board.

  • Provenance and peer review Not commissioned; externally peer reviewed.

Request permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.