Article Text

Download PDFPDF

Relationship between acute stress and clinical performance in medical students: a pilot simulation study
  1. Stephanie J Russ1,
  2. Ian Morrison1,
  3. Cheryl Bell2,
  4. Jeremy Charles Morse1,
  5. Rhoda Katharine Mackenzie1,
  6. Marie K Johnston2
  1. 1 Institute for Education in Medical and Dental Sciences, University of Aberdeen, Aberdeen, UK
  2. 2 Institute of Applied Health Sciences, University of Aberdeen, Aberdeen, UK
  1. Correspondence to Dr Stephanie J Russ, Institute for Education in Medical and Dental Sciences, University of Aberdeen, Aberdeen AB25 2ZD, UK; stephanie.russ{at}abdn.ac.uk

Abstract

Background Acute stress has been linked to impaired clinical performance in healthcare settings. However, few studies have measured experienced stress and performance simultaneously using robust measures in controlled experimental conditions, which limits the strength of their findings.

Aim In the current study we examined the relationship between acute stress and clinical performance in second-year medical students undertaking a simulated ECG scenario. To explore this relationship in greater depth we manipulated two variables (clinical urgency and cognitive load), and also examined the impact of trait anxiety and task self-efficacy.

Methods Second-year medical students were asked to conduct a 12-lead ECG on a simulated patient. Students were randomly assigned to one of four experimental conditions according to clinical urgency (high/low) and cognitive load (high/low), which were manipulated during a handover prior to the ECG. As part of the scenario they were asked to describe the ECG trace to a senior doctor over the phone and to conduct a drug calculation. They then received a performance debrief. Psychological stress and physiological stress were captured (via self-report and heart rate, respectively) and various aspects of performance were observed, including technical competence, quality of communication, work rate and compliance with patient safety checks. Trait anxiety and task self-efficacy were also captured via self-report.

Results Fifty students participated. While there was little impact of experimental condition on stress or performance, there was a significant relationship between stress and performance for the group as a whole. Technical competence was poorer for those reporting higher levels of psychological stress prior to and following the procedure. Neither trait anxiety nor task self-efficacy mediated this relationship.

Conclusions This study has provided evidence for a link between acute stress and impaired technical performance in medical students completing a simulated clinical scenario using real-time measures. The implications for patient safety and medical education are discussed.

  • stress
  • simulation-based education
  • patient safety

Statistics from Altmetric.com

Introduction

Hospital doctors typically experience a wide range of stressors in their work setting and report high levels of occupational stress and burnout.1–4 Clinical work presents a range of stressors which might affect performance -some specific to the clinical situation, for example, dealing with ill patients, performing clinical tasks, being observed by senior clinicians, alarms, bleeps, telephone, distractions - others are more generic, (i.e. would constitute stressors in any environment), for example, cognitive stress of retaining and using large quantities of information, heavy workload and inadequate support/supervision.5–8 The experience of occupational stress seems to be particularly marked in junior doctors making the transition from the comparatively protected learning environment of the medical student to the work environment.6 9 10

While low-intensity stressors can facilitate performance, retention of information and motivation to complete challenging tasks,11 prolonged or excessive levels of stressors, that exceed an individual’s perceived coping resources, can negatively affect cognitive functioning and performance.12–14 This presents risks for patient safety. In the presence of stressors and associated burnout there is evidence of less than optimal clinical performance, including impaired psychomotor performance, prescribing errors, increased probability of involvement in an adverse event, increased prevalence of malpractice suits and impaired non-technical skills such as communication and decision-making.15–21

One theory for these adverse impacts is that cognitive load is increased under stressful conditions and puts pressure on limited working memory capacity.22–24 Further detrimental effects of work stress could be attentional narrowing and distraction. These altered cognitive processes may adversely affect professional judgements, decision-making, efficiency, motivation andsatisfaction, and consequently render the individual more error prone.19 It has also been postulated that more stable personality traits might influencethese relationships, namely trait anxiety and self-efficacy, which have been linked to increased stress and exacerbation of the adverse impact of stress on performance.25 26

In healthcare, few studies have simultaneously measured stressors and performance due to methodological challenges, making it difficult to draw links between the two. And we are yet to understand fully which conditions are the most stress inducing and which tasks are most susceptible to disruption by stress. The simulation environment allows reproducible manipulation of variables and observation of real-time impact on performance making it possible to explore these relationships in more depth.

The aim of this pilot study was to measure the effect of stressors on various aspects of clinical performance, including techincal skills, non-technical skills and patient safety behaviours, in second-year medical students participating in a simulated clinical scenario. (Second-year medical students were selected as this was part of an effort to incorporate simulation-based education earlier on in the undergraduate medical curriculum.) We manipulated two variables via experimental conditions—first clinical urgency, by varying the severity of the patients’ illness (and thus varying the degree of demand experienced), and second, cognitive load, by varying the required level of communication with the patient (and thus varying the burden on working memory).

The specific research questions were as follows:

  • What is the relationship between stress and clinical performance?

  • How does clinical urgency and cognitive load influence these relationships?

  • Does trait anxiety and/or self-efficacy mediate these relationships?

We present data from the first cohort to undertake this simulated learning opportunity. We treated this as a pilot study to test the feasibility of the design before rolling it out with a larger cohort of medical students.

Method

Participants

All 180 second-year medical students in academic year 2014–2015 at the University of Aberdeen were invited via email to take part in a simulated clinical scenario to practise their newly acquired ECG and patient management skills. They were advised that they would also be invited to take part in a research study looking at how students react in these settings. At this stage attendance was not mandatory.

Design

This was a cross-sectional, experimental factorial design, with two factors: clinical urgency and cognitive load, each with two levels (high/low). Both factors were manipulated during a handover given to the students immediately before they entered the simulation room to conduct the ECG. Clinical urgency was manipulated by framing the scenario as either clinically urgent (high urgency) or routine (low urgency), while cognitive load was manipulated by asking the students to either explain to the patient precisely what they were doing as they went along (high cognitive load) or not (low cognitive load). Students were randomly allocated to one of four groups according to these factors:

  • Group 1: low clinical urgency, low cognitive load

  • Group 2: high clinical urgency, low cognitive load

  • Group 3: low clinical urgency, high cognitive load

  • Group 4: high clinical urgency, high cognitive load

The outcome variables were performance and stress. Additionally we assessed task self-efficacy and trait anxiety.

Measures

Performance

Various clinically relevant performance measures were recorded (figure 1). These included important patient safety checks (×4), technical competence assessments (×14) and quality of communication variables (×7), which were all scored by a clinician observer during the scenario (for the purposes of the study and to aid the debrief) and checked by a blinded observer retrospectively from video recordings. These were all binary variables (yes=1, no=0) but were additionally summed to provide a total patient safety, technical competence and quality of communication score, and an overall total performance score out of 25. We also included two continuous measures of ‘work rate’ (time taken to complete the ECG and a drug calculation)—which were scored retrospectively from the video recordings. Finally, with regard to communication, we looked at the percentage of time spent talking during the ECG and the nature of any social communication driven by the students, that is, whether the student initiated conversation about non-clinical topics (eg, asks about patient’s job). The latter variable was scored on a binary (yes/no) scale but was not included in the ‘total performance score’ as it was decided that social communication could either be positive (eg, putting the patient at ease) or negative (eg, non-timely, inappropriate). These measures were informed by Objective Structured Clinical Examination (OSCE) items for an ECG station, but were more comprehensive than the performance variables that would be scored during an OSCE.

Figure 1

Performance variables.

Stress

Physiological stress

Heart rate was captured during the simulation using a small, lightweight Actiheart (CamNTech) monitor, which measures activity from a chest-mounted accelerometer, heart rate from ECG, and computes a measure of energy expenditure by combining the two. Non-metabolically determined changes in heart rate provide a physiological indicator of stress. Students taking medication known to alter heart rate (eg, beta blockers) were excluded from this element of the study.

Psychological stress

Self-reported anxiety was captured using the shortened version of the State section of the validated State Trait Anxiety Inventory (STAI).27 This consists of six items which capture feelings pertaining to anxiety at a certain point in time, for example, ‘I feel worried’, ‘I feel tense’. Items are scored using a Likert response scale ranging from 1 (not at all) to 4 (very much so). Total scores range from 6 to 24. State anxiety scores vary reliably with situational stress and as such the State scale is commonly used as a proxy measure of acute stress in research studies.28

Trait anxiety and task self-efficacy

Trait anxiety

Trait anxiety was measured using the Trait section of the STAI. This consists of 20 items designed to capture a stable propensity to experience anxiety, depression and general negative affect, and a tendency to interpret stressful situations as threatening.29 Items are scored on a 4-point Likert scale ranging from 1 (not at all) to 4 (very much), for example, ‘I am a steady person’, ‘I feel inadequate’. Total scores range from 20 to 80.

Self-efficacy

A 9-item self-efficacy scale was constructed for the purposes of this study using published guidance.30 Items captured self-reported confidence in completing tasks relevant to the scenario, for example- How confident are you that you can … ‘perform an ECG without the support of a senior’, ‘interact with patients’, ‘perform calculations without a calculator’. Items were scored on a scale ranging from 0% to 100% confident. Participants indicated where they fell on the scale by reporting a % (eg, 60%) for each item individually. Total scores on this scale could therefore range from 0% to 900%, with higher scores indicating greater confidence.

Session evaluation

A short 6-item self-report scale was constructed to evaluate the utility of the session as perceived by the students, for example, whether they considered it a valuable learning opportunity and also whether the scenario was appropriate and felt realistic. Students were asked the degree to which they agreed with each statement using a 4-point rating scale ranging from 1 (not at all) to 4 (very much).

Procedure

Students attended the simulation individually at their allocated time slot. They were first provided with an overview of the session content and given a tour of the simulation room to familiarise them with the layout and the SimMan 3G manikin (the patient).31 Next, informed written consent was sought for inclusion in the research study and consenting students were allocated a unique ID number and randomly allocated to an experimental condition.

Those participating then filled in a ‘pre-simulation questionnaire’ which included the State and Trait portions of the STAI, the self-efficacy scale and basic demographic information (age and sex). They were then fitted with the heart rate monitor which was set up for each participant using their ID number, age, weight and height (controlled for at the data analysis stage).

The simulation then commenced. Students were introduced to a senior doctor who provided them with a handover using a structured SBAR approach (Situation, Background, Assessment, Recommendation).32 The experimental condition was manipulated as part of the handover (figure 2). This same clinician observed and scored the participant’s performance throughout the simulation through a two-way mirror.

Figure 2

Senior doctor handover according to experimental condition.

Students entered the simulation room alone and began the ECG as they saw fit. Once all ECG leads had been attached the students were handed an ECG reading and told to use the phone to speak to a senior. The students’ phone calls were answered by the senior doctor who asked them what the ECG showed. Students were expected at their stage of training to identify two features of the trace: irregularity and tachycardia. If they did not mention these features or gave an incorrect interpretation they were coached towards the correct interpretation. Once irregularity and tachycardia had been confirmed the senior doctor asked the students to complete a drug calculation for preparation of a heparin drip (figure 3) based on a potential diagnosis of atrial fibrillation. Students were asked to record their working on a sheet of paper and to return to the adjoining room once complete.

Immediately following the simulation participants removed their heart rate monitor and completed a ‘post-simulation questionnaire’ including the State anxiety and self-efficacy items. They were then provided with a full debrief on their performance which included jointly establishing learning goals moving forward. Students were given the opportunity to ask questions and explore concerns. Following the debrief, a final ‘post-debrief questionnaire’ was completed including the State anxiety scale, self-efficacy scale and session evaluation questions.

Data analysis

All analyses were conducted using IBM SPSS Statistics V.23. Descriptive statistics were computed for all variables. Histograms appeared normally distributed allowing parametric analysis. Mean heart rate during the scenario was used as a summary measure for physiological stress. To assess the pattern of subjective stress over time, total State anxiety scores were computed for each time point (pre-simulation, post-simulation and post-debrief) and a repeated measures analysis of variance (ANOVA) was computed for the sample as a whole.

In line with the research questions, we then examined the relationship between physiological and psychological stress and performance for the sample as a whole. Correlations were conducted between stress measures (mean heart rate, and total state anxiety scores at each time point) and the continuous performance measures (patient safety score, technical competence score, quality of communication score, total performance score, time spent talking and work rate). An independent samples t-test was computed to establish the difference in stress measures for those who did and did not engage in non-clinical conversation and perform an accurate drug calculation.

We then assessed the main effects and simple main effects of the experimental factors (urgency and cognitive load) for physiological and psychological stress, and all performance measures (as above), using one-way ANOVA (for continuous variables) and Χ2 for binary variables.

Finally, we examined the relationship between trait anxiety and stress/performance, and self-efficacy and stress/performance using the same approach.

Session evaluation data were summarised descriptively.

Results

Fifty students volunteered to participate in the session (28% response rate). They had an average age of 20.6 years (SD=0.4, range 18–33), and 39 (78%) were female.

Stress

Psychological stress

State anxiety scores are summarised in table 1. For all participants combined, there was a significant difference in State anxiety over time, where self-reported anxiety was significantly lower post-debrief than it was pre-simulation (mean difference 3.3, P<0.001) or immediately post-simulation (mean difference 3.8, P<0.001). The same pattern of stress over time was observed when considering each experimental condition in isolation (Fs range: 4.1–17.1, P<0.05).

Table 1

Summary of state anxiety, trait anxiety and self-efficacy scores

Physiological stress

For all participants combined the average mean heart rate during the scenario was 104.95 beats per minute (SD=18.63).

Performance

Descriptive statistics for performance measures are summarised for all participants combined in figure 4 and table 2. Compliance with standard patient safety behaviours was variable, for example, with hand wash only being used by approximately half of the sample and a patient wristband check being carried out by around three quarters. With regard to technical competence, 70% of the students placed all 10 ECG leads correctly, but correct interpretation of the ECG appeared more challenging, particularly regarding identifying tachycardia (completed successfully by only 32% of the sample). Twenty-eight per cent of students made an error in the drug calculation. Students were generally effective at communicating with the patient regarding the planned procedure (over 80% of the sample provided a clear explanation of how and why they were completing the ECG), but struggled more in communicating with a senior over the phone (44% of students did not introduce themselves and 26% did not use a structured approach to relay the ECG findings). Thirty-six per cent of the sample engaged in non-clinical conversation during the procedure. On average, the participants took just over 4 min to complete the ECG and just over 2 min to complete the drug calculation.

Figure 4

Summary of binary performance measures.

Table 2

Summary of scores for continuous performance measures

Relationship between stress and performance

Total technical competence was negatively correlated with State anxiety at all time points: pre-simulation (r=−0.35, P=0.01), post-simulation (r=−0.41, P=0.003) and post-debrief (r=−0.54, P<0.001), showing that those with higher psychological stress levels performed worse technically. Additionally, total performance score was negatively correlated with post-simulation and post-debrief State anxiety (r=−0.31, P=0.03 and r=−0.45, P<0.001, respectively). There was no relationship between physiological stress (mean heart rate) and any of the performance measures.

Impact of clinical urgency and cognitive load on stress and performance

Stress: There was no effect of experimental condition on either psychological or physiological stress (Fs range: 0.01–2.7, P>0.05).

Performance: There was a main effect of cognitive load for percentage of time spent talking during the ECG—with those in the high cognitive load group speaking significantly more (F=12.1, P<0.001). Additionally, whether or not students spoke to the patient about non-clinical topics during the ECG was significantly associated with clinical urgency—those in the high urgency groups were more likely to speak about non-clinical topics than those in the low urgency groups (X2=5.56, P=0.018). There were no additional main effects or simple main effects for clinical urgency or cognitive load for any of the performance measures.

Trait anxiety and self-efficacy

Mean trait anxiety scores and total self-efficacy scores are summarised in table 1.

Trait anxiety: Mean trait anxiety for the sample as a whole was 39.6 which is comparable to normative samples,33 34 and trait anxiety did not differ according to experimental condition. Trait anxiety was positively related to State anxiety at all time points (rs range: 0.32–0.62, Ps<0.05). With regard to performance, there was a positive relationship between Trait anxiety and time spent completing the ECG (r=0.4, P<0.01). There were no other associations between trait anxiety and performance.

Self-efficacy: For the sample as a whole, total self-efficacy scores varied significantly over time (F(2,78)=21.9, P<0.001)—they were lowest immediately post-simulation and highest post-debrief. The same effect over time was observed in each experimental condition considered in isolation.

Total self-efficacy was related to psychological stress. Pre-simulation, post-simulation and post-debrief self-efficacy scores were negatively correlated with State anxiety scores at all three time points (rs range: −0.29 to 0.59, Ps<0.05). There was no relationship between self-efficacy and physiological stress.

Pre-simulation self-efficacy was not related to any of the performance scores. However, those who got the drug calculation correct and those who completed it the quickest time had higher self-efficacy scores following the simulation (post-simulation: F(1,49)=5.7, P<0.05 and r=−0.41, P<0.01, respectively; post-debrief: F(1,49)=11.4, P<0.01 and r=−0.35, P<0.05, respectively).

Session evaluation

The session was received extremely favourably by the students in all respects (table 3).

Table 3

Session evaluation

Discussion

In this pilot study we examined the relationship between stress and clinical performance in second-year medical students undertaking a simulated ECG scenario. In an attempt to ‘unpick’ the relationship we manipulated two variables via experimental conditions: clinical urgency (a potential stressor) and cognitive load (which may mediate negative impacts of stress on performance). There was evidence of a behavioural response to the conditions. Those assigned to a high cognitive load group spoke significantly more during the scenario (in line with explaining what they were doing as they went along) and those in a high urgency group were significantly more likely to engage in social (non-clinical) conversation with their patient—which we suggest was an attempt to put the patient at ease. We failed however to find any association between experimental condition and stress or other performance measures. Alongside a potential lack of power (due to a conservative sample size), we feel that this can be explained by the fact that all students experienced a moderate degree of stress and likely experienced high cognitive load due to the novelty of the task, which masked any potential effects of the experimental manipulations.24 It was the first time they had experienced the simulated environment and had been observed and assessed individually on newly acquired clinical skills. Moreover, self-reported State anxiety levels were elevated for all students pre-simulation before any manipulation had taken place.

While there was no effect of experimental condition, it was still possible to look at variation in stress levels in relation to performance for the group as a whole. In doing so we found that those students who reported greater stress levels (both before and after the simulation) performed worse technically—they made more technical errors and/or omissions than those who reported feeling less stressed. These findings support the notion that stress might impair performance, and are in line with previous research.5 9 We cannot however attribute this to greater cognitive load in this study. Poorer communication skills and patient safety behaviours were related to higher self-reported stress levels following but not prior to the scenario, which makes it more difficult to attribute these performance deficits to the experience of stress itself. These findings have clear implications for patient safety and support investment in training programmes and workplace design interventions that are informed by human factor principles around stress management and the systems approach to understanding error.

We also measured trait anxiety and task self-efficacy to assess whether these factors were related to stress and performance and potentially mediated any impact of stress. We found no evidence of the latter. Both trait anxiety and task self-efficacy were related to State anxiety at each time point which was not unexpected given the strong relationship between these constructs reported in previous work.33 35–37 Trait anxiety was also positively related to the amount of time taken to complete the ECG (perhaps indicating that those with higher trait anxiety were more calculated in their approach) but it was not related to any other outcome variable. Task self-efficacy was lowest immediately post-simulation and highest immediately post-debrief for all students which highlights the importance of an effective debrief in addressing students’ concerns (State anxiety was lowest post-debrief which also supports this). Finally, task self-efficacy did not predict performance—scores prior to the simulation were not related to performance measures—however those who got the drug calculation correct and spent less time completing it reported higher self-efficacy post-simulation which likely reflects their awareness of their success.38

While we revealed some important correlations between stress and performance here, the results of this study are also important from an educational perspective. First, we revealed gaps in learning that require educational reinforcement. For instance, student compliance with some basic patient safety behaviours, which had recently been taught and assessed, was poor; interpretation of the ECG reading, even at a very basic level, proved challenging for many; and the use of a structured approach for communicating information to a senior was demonstrated by few. Second, until now, students at our medical school did not partake in simulation until the penultimate year (year 4) of their degree course. The current session was introduced as part of a curriculum redesign aimed at integrating stage-appropriate simulation opportunities earlier on. The positive feedback received and the ability of the students to engage appropriately suggests that the strategy to introduce simulation earlier in the curriculum was both feasible and beneficial.

Limitations, strengths and future directions

This study had certain limitations, which in part reflect the challenges of delivering robust research designs in a teaching environment. First, physiological stress was measured using average heart rate during the simulation. While this gives an impression of arousal during the task, ideally we would have measured baseline heart rate and calculated the increase, or physiological stress response for each individual in relation to their performance—however restrictions in the time available prevented us from taking baseline measures. Our measure of physiological stress was therefore fairly crude and likely lacked sensitivity in picking up variation between individuals. In future cohorts we plan to time-stamp our measures to allow analysis of how heart rate differs according to the specific task being performed (ECG vs drug calculation, and so on). Second, since attendance at the simulation was voluntary it is possible that the students who attended were not representative of the year and were skewed in terms of performance and/or anxiety, particularly given the modest response rate (28%). This will be addressed in future cohorts when attendance becomes mandatory. On a similar vein, one might question whether these findings were a virtue of involving medical students early on in their training and whether similar findings would be observed with more senior students or junior doctors. Having designed the project to be stage appropriate, that is, assessing only those clinical skills that had already been trained, we see no reason why the results should be particular to the group assessed. However, it would be interesting to apply the design to scenarios with more senior students going forward. Next, we undertook multiple statistical comparisons without imposing Bonferroni or similar corrections which increases the chance of type 1 error. While it is important to take this into account, we felt this was the appropriate approach to take at this exploratory/pilot stage to avoid missing potentially important results which will inform hypothesis building moving forward. Finally, we feel that our manipulation of cognitive load may have been flawed. While previous experimental studies have typically increased cognitive load (burden on working memory) by requiring participants to engage in dual-processing tasks such as counting backwards, we wanted to ensure our manipulation held clinical relevance to preserve the fidelity of the scenario. We therefore asked students to articulate their clinical actions as they went along (an additional communication task). While this likely increased cognitive load, ‘thinking out loud’ in this way also has the potential to enhance performance by allowing transition to a slower, more analytical decision-making style.39 In future those in high cognitive load groups will be distracted by a serious of questions by the patient which should increase demand on working memory but avoid thinking out loud.

Despite these limitations this study was one of few to endeavour to assess the relationship between stress and performance simultaneously using robust, real-time measures. Performance was observed rather than self-reported retrospectively; the outcomes were clinically relevant and included seldom assessed behaviours such as compliance with patient safety checks; validated measures were used where possible; and the simulated environment offered high clinical fidelity while also allowing a good degree of experimental control and standardisation.

Conclusion

In a population of medical students we have supported the emerging evidence base suggesting that acute elevated stress can impair performance of clinical skills. This reinforces the importance of taking stress seriously in medical settings from a patient safety perspective, and putting systems in place to support effective stress management and awareness. Future work should continue to unpick which tasks are most vulnerable to interference from stress using high fidelity and well-controlled methodologies, such as those afforded by simulation. From a medical educational perspective we would encourage those with a role in curriculum design to incorporate stage-appropriate introduction to simulation-based education right from the start of medical school.

References

Footnotes

  • Contributors All authors were involved in formulation of the research concept, research questions and study design. SJR, IM, CB, JCM and RKM were involved in data collection. SJR and RKM undertook data analysis. SJR and MKJ drafted the manuscript and all authors were involved in critically reviewing it. All authors have approved the manuscript for submission.

  • Funding This study ran as part of the MBChB curriculum at the University of Aberdeen and no additional funding was required.

  • Competing interests SJR has recently become a member of the editorial board for BMJ Simulation & Technology Enhanced Learning. All other authors declare no competing interests.

  • Patient consent Obtained.

  • Ethics approval University of Aberdeen College Ethics Review Board.

  • Provenance and peer review Not commissioned; externally peer reviewed.

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.