Table 1

Roles for simulating in different study types aimed at evaluating improvement interventions

Class of studiesPotential role(s) for simulationPotential data collection method(s)Example(s)
Quality improvement projects
  • Understand the problem situation

  • Assess intervention feasibility

  • Evaluate the effect of the intervention

  • Optimise design and implementation of the intervention

  • Observational checklists

  • Focus groups and interviews

  • Simulator-collected measures

  • Physiological measures

  • Questionnaires

  • Combination of simulation and Failure Modes and Effects Analysis in a prospective risk analysis90

  • Identification of latent threats in a new hospital facility91

Effectiveness studies:
Randomised controlled trials (RCTs)
Quasi-experimental studies
Observational studies
  • Evaluate the effectiveness of the intervention

  • Pre-test interventions

  • Observational checklists

  • Simulator-collected measures

  • Physiological measures

  • Questionnaires

  • RCT: simulation-based trials of surgical checklists,6 46 trial on the impact of sleep deprivation on non-technical skills (box 2),79 trial on the impact of training on speaking-up behaviour (box 1)88

  • Quasi-experimental studies: uncontrolled study of a new drug packaging system (box 4),92 uncontrolled evaluation of a paediatric resuscitation training package,93 non-randomised controlled study of an intervention to improve the management of distractions and interruptions during ward rounds94

  • Observational studies: simulation-based longitudinal study of crisis resource management ability95

Process evaluations
  • Evaluate how the intervention is received by participants

  • Observational checklists

  • Focus groups and interviews

  • Simulator-collected measures

  • Physiological measures

  • Questionnaires

  • Use of simulated patients as part of a multimethod process evaluation of an intervention to improve youth‐friendly services for sexually transmitted infections96

Qualitative studies
  • Explore perceptions of the intervention

  • Produce descriptions and theoretically informed analysis of scenarios

  • Focus groups and interviews

  • (Video-)ethnography

  • Qualitative study of the simulation of an audio-visual telehealth service97

Economic evaluations
  • Feed data to economic models and projections

  • Observational checklists

  • Simulator-collected measures

  • Physiological measures

  • Questionnaires

  • Use of clinical simulation in the cost-effectiveness evaluation of an electronic health record system98