Article Text

Download PDFPDF

Simnovate supplement
Evaluating the role of simulation in healthcare innovation: recommendations of the Simnovate Medical Technologies Domain Group
  1. Amin Madani1,2,
  2. Benoit Gallix1,
  3. Carla M Pugh3,
  4. Dan Azagury4,
  5. Paul Bradley5,
  6. Dennis Fowler6,
  7. Blake Hannaford7,
  8. Sumaira Macdonald8,
  9. Kiyoyuki Miyasaka9,
  10. Natalia Nuño10,
  11. Amir Szold11,
  12. Vedat Verter12,
  13. Rajesh Aggarwal1,2
  1. 1Steinberg Center for Simulation and Interactive Learning, McGill University, Montreal, Quebec, Canada
  2. 2Faculty of Medicine, Steinberg Centre for Simulation and Interactive Learning, McGill University, Montreal, Quebec, Canada
  3. 3Department of Surgery, University of Wisconsin Hospital & Clinics, Madison, Wisconsin, USA
  4. 4Department of Surgery, Stanford University School of Medicine, Stanford, California, USA
  5. 5Johnson & Johnson Medical Companies, Cincinnati, Ohio, USA
  6. 6Titan Medical Inc., Toronto, Ontario, Canada
  7. 7Department of Electrical Engineering, University of Washington, Seattle, Washington, USA
  8. 8Silk Road Medical Inc., Sunnyvale, California, USA
  9. 9Department of Anaesthesia, St. Luke's International University, Tokyo, Japan
  10. 10Department of Automated Manufacturing Engineering, École de Technologie Supérieure, Montreal, Quebec, Canada
  11. 11Assia Medical Group, Tel Aviv, Israel
  12. 12Faculty of Management, McGill University, Montreal, Quebec, Canada
  1. Correspondence to Dr Amin Madani, Department of Surgery, Faculty of Medicine, McGill University Health Centre, 1650 Cedar Avenue, Rm D6-257, Montreal, Quebec, Canada, H3G 1A4; amin.madani{at}


Background Innovation in healthcare is the practical application of new concepts, ideas, processes or technologies into clinical practice. Despite its necessity and potential to improve care in measurable ways, there are several issues related to patient safety, high costs, high failure rates and limited adoption by end-users. This mixed-method study aims to explore the role of simulation as a potential testbed for diminishing the risks, pitfalls and resources associated with development and implementation of medical innovations.

Methods Subject-matter experts consisting of physicians, engineers, scientists and industry leaders participated in four semistructured teleconferences each lasting up to 2 hours each. Verbal data were transcribed verbatim, coded and categorised according to themes using grounded theory, and subsequently synthesised into a conceptual framework. Panelists were then invited to complete an online survey, ranking the (1) current use and (2) potential effectiveness of simulation-based technologies and techniques for evaluating and facilitating the product life cycle pathway. This was performed for each theme of the previously generated conceptual framework using a Likert scale of 1 (no effectiveness) to 9 (highest possible effectiveness) and then segregated according to various forms of simulation.

Results Over 100 hours of data were collected and analysed. After 7 rounds of inductive data analysis, a conceptual framework of the product life cycle was developed. This framework helped to define and characterise the product development pathway. Agreement between reviewers for inclusion of items after the final round of analysis was 100%. A total of 7 themes were synthesised and categorised into 3 phases of the pathway: ‘design and development’, ‘implementation and value creation’ and ‘product launch’. Strong discrepancies were identified between the current and potential roles of simulation in each phase. Simulation was felt to have the strongest potential role for early prototyping, testing for safety and product quality and testing for product effectiveness and ergonomics.

Conclusions Simulation has great potential to fulfil several unmet needs in healthcare innovation. This framework can be used to help guide innovators and channel resources appropriately. The ultimate goal is a structured, well-defined process that will result in a product development outcome that has the greatest potential to succeed.

  • innovation
  • prototype
  • device
  • medical technology
  • simulation

Statistics from


The clinical atmosphere has evolved considerably into one that faces an explosion of scientific and technological advancements, a greater focus on patient safety, transparency and accountability and the increasing expectations to deliver excellence in patient care. Healthcare innovation continues to play a strong role in this process, often driven by the very organic symbiotic relationship between healthcare, technology, academia, industry, economic and market variables and changing societal needs. Healthcare innovation can be described as the practical application of a new concept, idea, process or technology into clinical practice to improve care in a measurable way.1 ,2 Frequently, this involves combining previously distinct concepts in some tangible way to produce a more useful product. It is important to emphasise that innovation, while often idealised as a medical device or hardware that provides diagnostic or therapeutic benefits, can also take the form of a new methodology, practice or software to improve processes of care.3–5

The process of translating and implementing an idea into practice is the end-result of a highly complex journey requiring a needs assessment, ideation, significant experimentation, capital, resources, collaboration with a multidisciplinary team of experts and perseverance.3 As a result, the development and process of allocating value to a great idea in order to translate it from conception to implementation typically lasts more than a decade and is associated with a high failure rate of >90% after 10 years.6 Several variables, such as commercialisation, corporate and product development, intellectual property and regulatory approval, play important roles throughout this process and can act as major pitfalls for innovators. Furthermore, there are a number of issues related to the unintended consequences of innovations, including protecting patients from harm secondary to new technologies,7 ,8 the increasing costs to institutions, patients and healthcare systems and the lack of penetration and adoption by end-users, especially in low-income geographic regions. This raises the question whether the innovation pathway can be optimised and ameliorated in order to facilitate the process of translating an idea or concept into clinical care.

A potential solution that can help address some of these deficiencies is the use of simulation—broadly defined as “an environment to replace or amplify real-life experiences with guided experiences that are artificially devised in order to evoke and replicate substantial elements of the real environment in an interactive, immersive and experiential manner.”9 Strong evidence supports the use of simulation as a cost-effective adjunct for health professions education,9 ,10 as well as other domains such as aviation safety,11 and industry and manufacturing to enable the validation and testing of new products into the market.12 The healthcare innovation pathway involves understanding current behaviours and formulating projections on future states, be it assessing product effectiveness, product usability and quality, stakeholder needs, user opinions or manufacturing quality. It is therefore conceivable that simulation can be potentially harnessed to achieve many of these objectives and help accelerate and diminish some of the risks associated with the introduction of novel ideas into clinical practice, while minimising costs and harm to patients.

We sought to develop a framework to better define the role of simulation for medical innovation. The purpose of this mixed-method study is to define the current role and potential utility of simulation for various stages of the product development pathway for medical innovation.

Materials and methods

Study design

In order to map out the role of simulation within the product development pathway, we performed a two-phase mixed-method study that consisted of qualitative methodologies to develop a hybrid medical innovation pathway, and a subsequent quantitative component, where the current and potential roles of various forms of simulation are defined for each step of this hybrid model. We specifically focused on the development of medical technologies and excluded innovation in the pharmaceutical industry. The study did not receive ethics review, but was found by the Chair of the Institutional Review Board within the Faculty of Medicine at McGill University, to be conducted in an ethically acceptable manner.

Qualitative component

A combination of qualitative methods was used to explore and define the various components and elements of the innovation pathway. Subject-matter experts (SMEs) were initially selected to form a panel as part of the Simnovate Medical Technologies domain group in order to bring together a multidisciplinary group of individuals and experts in the areas of simulation, innovation and education. This working group was spearheaded by two co-chairs, the simnovate lead and a research fellow, with the aim to explore and define the role of simulation throughout the process of innovation in healthcare. Given the highly collaborative and multidisciplinary nature of innovation, SMEs were selected from a wide variety of backgrounds in order to enrich the breadth of qualitative data and to capture as many ideas and experiences as possible. These included individuals with a track record in innovation in healthcare and with expertise in various fields of medicine, simulation, education and academia, applied science and engineering, business, economics and industry and patient safety. SMEs were also chosen to represent various demographics and geographic locations (10 institutions from 5 countries).

Four teleconferences were conducted between September 2015 and December 2015. A detailed agenda for each focus group was initially developed by the four organisers, including a template of questions to use as discussion points throughout the meeting. Prior to the start of teleconferences, all panelists were required to identify any potential conflicts of interests. All teleconferences were led by a moderator with prior experience in running focus groups using best practices. Close-ended dichotomous questions and interruption of dialogue by the moderator were minimised in order to avoid bias or leading questions and to encourage discussion.

All teleconferences were transcribed verbatim and analysed to extract various components of the product development pathway, as well as the potential role simulation can play to improve this process. For triangulation and to identify data that may not have surfaced throughout the conversations, verbal data were supplemented with content from the literature. This also helped to ensure saturation of data. Published literature was selected to represent textbook chapters, review articles and innovation frameworks—within and outside of healthcare.

One investigator with experience in qualitative research coded transcriptions according to three major phases of the innovation pathway as defined by the Stanford Biodesign model: ‘identify’, ‘invent’ and ‘implement’.13 Following data extraction, grounded theory methodology was used to compare, match, merge and refine items and synthesise themes that define the innovation pathway. In addition, the interdependence and temporal relationship of these various elements were identified as part of this analysis and amalgamated to produce a comprehensive framework. Grounded theory involves several rounds of iterative inductive analysis whereby data are coded and combined into a list of items in order to extract emerging, anticipated and overarching themes of a particular construct.14 Data analysis was performed throughout the data collection process for ongoing assessment and theoretical sampling of emerging themes to guide additional data collection. Once data reached saturation and no additional items could be synthesised, data collection was terminated.

Quantitative component

In order to identify the areas where simulation can have a high impact for improving the product development pathway for healthcare innovation, an anonymous online survey was administered to all panelists (, Ottawa, Canada) for an 8-week period. SMEs were asked to rank (1) the current use, and (2) the potential effectiveness of simulation-based technologies and techniques for facilitating the product life cycle pathway for each theme of the innovation pathway using a Likert scale of 1 (no effectiveness) to 9 (highest possible effectiveness). Given the highly heterogeneous methods of simulation, this ranking was performed for six various forms of simulation using an established taxonomy by the Canadian Network for Simulation in Healthcare's 2013 Guidelines Working Group.15 These include ‘simulated patients’ (eg, actor or patient-based), ‘mannequin-based simulation’, ‘procedural simulation’ (eg, synthetic, animal, cadaver, bench-top and virtual reality simulation), ‘immersive simulation’ (eg, simulated ward and simulated operating room), ‘in-situ simulation’ (simulation performed in the actual clinical environment) and ‘computer-based simulation’ (eg, computer modelling, virtual worlds and serious games). Panelists had the opportunity to comment on any item or to list additional items that, in their opinion, were not included using free text comment box throughout the survey.

Mean effectiveness was calculated for each theme of the pathway, for each simulation type. These were converted into two topographical maps to characterise the current and potential use of various forms of simulation for each element of the pathway.


Qualitative component

Data were obtained from a panel of 13 SMEs using audio transcriptions from four teleconferences, each lasting up to 2 hours long. Additional qualitative data were acquired from nine published models of the medical innovation pathway. SMEs included six surgeons, two engineers, two scientists, one medical device industry leader, one interventional radiologist and one anaesthesiologist. All members have a track record with the development of novel medical technologies and intellectual property.

A conceptual framework was developed to define and characterise the product development pathway for healthcare innovation after seven rounds of inductive data analysis using grounded theory. Agreement between reviewers for inclusion of items after the final round of analysis was 100%. A total of seven themes were synthesised and categorised into three phases of the pathway: ‘design and development’, ‘implementation and value creation’ and ‘product launch’ (figure 1). The seven themes included: ‘needs-assessment’, ‘concept generation’, ‘prototyping’, ‘pre-clinical evaluation’, ‘analysis and strategy planning’, ‘training and education’ and ‘research & development’. The framework was further refined to highlight the elements within each theme and to demonstrate the interdependence and temporal relationship of the various items of the framework (figure 2).

Figure 1

The seven themes of the product development pathway.

Figure 2

A conceptual framework to define and characterise the product development pathway for healthcare innovation. The framework is divided into three phases.

Quantitative component

A topographical map was developed to describe the current use (figure 3) and potential effectiveness (figure 4) of simulation-based technologies and techniques for facilitating the product life cycle pathway. Procedural and computer-based simulation were found to have the greatest role currently for accelerating and improving the innovation process, whereas immersive and in-situ simulation were felt to have the greatest potential utility. Strong discrepancies were identified between the current and potential role of simulation for all types of simulation (especially immersive and in-situ simulations), with the exception of computer-based and mannequin-based simulation, which had marginal differences.

Figure 3

A topographical map representing the current utility of various simulation-based technologies and techniques (columns) for facilitating the elements of the product life cycle pathway (row). Ratings are based on the mean effectiveness according to a Likert scale of 1 (no effectiveness) to 9 (highest possible effectiveness).

Figure 4

A topographical map representing the potential utility of various simulation-based technologies and techniques (columns) for facilitating the elements of the product life cycle pathway (row). Ratings are based on the mean effectiveness according to a Likert scale of 1 (no effectiveness) to 9 (highest possible effectiveness).

With regard to the individual elements of the product development pathway, simulation was felt to have the strongest potential role throughout the implementation and value creation phase, including prototyping at all levels (especially earlier in the innovation process), testing for safety and product quality and testing for product effectiveness, usability and ergonomics. There was also a high impact identified for early stages (such as needs assessment and concept screening and selection) and late stages (including training and education of end-users, subsequent research and development, and for user feedback and quality assurance and monitoring). Similarly, there was a strong discrepancy between current and potential roles of simulation for all phases of the process, with the exception of analysis and strategy planning stage.


Innovation is a fundamental component of the healthcare industry, whose impact reaches patients, professionals, institutions and medical societies, and will continue to play a prominent role towards shaping our future. Nevertheless, the process of translating ideas into tangible products and protocols and applying them into practice is long, expensive and prone to high risk and is characterised by a high long-term failure rate and a poor adoption rate by end-users.6 ,16 Given the success of simulation in industry, aviation, athletics, military and a variety of other high-stakes professional environments within and outside healthcare,17 we examined its role for improving the product development pathway for medical innovation. Specifically, we performed a mixed-method study with experts to determine the role and utility of various forms of simulation for the many elements of the innovation process and showed that there are strong discrepancies between their current usage and potential effectiveness in many areas.

At its core, simulation provides the means to recreate a scenario before the real-life event, with ample opportunities to replay simulated scenarios, while changing selected parameters, for evaluation and assessment. Given the unpredictability of the innovation process and its heavy dependence on making future projections of current states after the introduction of a particular product, simulation provides a vast array of processes and technologies to predetermine the outcome without subjecting the innovators to spending the necessary resources in order to test their product in a real clinical environment. Specifically, a variety of analyses can be performed to obtain objective/quantitative and subjective/qualitative data that will help inform the investigators on whether to proceed, or revise and re-strategise. This can substantially diminish the cost and risk associated with product development and testing. For instance, simulated environments can provide the means to perform a thorough evaluation to determine where the true needs are located and avoid channelling resources towards developing a product that does not address the true need of the user, thereby accelerating the process and improving the likelihood of successful implementation. Also throughout development, products can be tested in a variety of simulated environments to help determine whether or not they result in the intended outcomes or, more importantly, to confirm that their use does not result in unintended and potentially dangerous outcomes.

In this study, simulation was noted to be potentially high-yield for several areas of the product development pathway. First, throughout various phases of prototype development and assessment, validation studies can be performed to evaluate clinically relevant metrics that will inform investigators whether or not a proof-of-concept or proof-of principle can be achieved. This includes both the early stages of the innovation process, including during early prototyping, preclinical evaluations or postlaunch phases, whereby detailed data can be obtained on a product's effectiveness (eg, successful relevant outcomes), quality (eg, integrity of equipment), safety (eg, negligible collateral injury), ergonomics and usability (eg, optimal design) and user opinions. These studies and experiments are particularly relevant for medical technologies and apparatuses that can be physically tested either in-situ or in an immersive simulated environment, such as the use of a novel device that provides non-invasive monitoring of critically ill patients on a ward, or a new surgical tool for use in the operating room. The criticality of early testing of a prototype may not become apparent until very late in the development phase, where a concept can be completely derailed or lack any penetration into practice and market due to a small, albeit significant, detail in the design of the prototype that renders it either suboptimal in performance or ergonomically inferior to other alternative options. Early and thorough testing and analysis can help identify issues, concepts and design flaws that are more likely to fail at an early stage and allow innovators to address them with less impact on cost, time and resources. Importantly, immersive and in-situ simulations can play a crucial role, in the preclinical and postlaunch phases, to analyse the integration of an innovation in the healthcare environment. This includes its interaction with various members of the team, as well as its impact on healthcare processes, workflow and various material and human resources required to successfully introduce a product. This may further lead to modifications either to the innovation itself (eg, newer prototypes and models to overcome any potential obstacles) or the environment (eg, creating the necessary physical space to store and use the product appropriately) in order to ameliorate this integration and avoid potential patient safety concerns.

In addition to testing the product itself, another major area for which simulation can play a significant role is that of training and education of end-users who will eventually be impacted by the implementation of the innovation. Much has been published in the literature regarding the ethics of introducing a novel medical technology into clinical practice and various working groups and professional societies have attempted to develop ethical standards and appropriate safeguards to help guide the safe implementation of innovation and avoid the unmonitored capability of practitioners to test new devices and processes of care on patients, regardless of their positive intentions.18–20 One of the greatest advantages of simulation is its ability to provide educators the means to capture the simulated environment (eg, using audiovisual equipment) and to analyse performance and outcomes in manners that provide meaningful qualitative and quantitative data. These data can subsequently be used to as part of educational curricula to provide focused feedback, with immediate and ample opportunities for repetition in order to deliberately practice specific skillsets required for the usage of the innovation.21 In addition, such environments can allow for errors and experimentation to occur without putting patient safety at risk.

This can be illustrated by the advent of minimally invasive technologies for thoracic and abdominal surgeries, such as laparoscopy and video-assisted thoracoscopic surgery, which have revolutionised surgical care and patient recovery. Despite their contribution towards improving clinical and patient-reported outcomes for the management of benign and malignant diseases, concerns have been raised regarding the unusually high rates of inadvertent injuries, such as major bile duct injuries during cholecystectomy and intra-abdominal injuries with the use of laparoscopic energy devices.7 ,8 ,22–25 Furthermore, despite strong evidence to support the superiority of minimally invasive approaches for certain procedures, universal adoption remains a significant challenge, such as for the operative management of cancers of the colon and rectum,26 ,27 and early lung cancer.28 ,29 In light of these patient safety concerns and the limited acceptability of established surgeons to re-train themselves to incorporate a disruptive technology into their practice, the Society of American Gastrointestinal and Endoscopic Surgeons has introduced and disseminated over the last decade the Fundamentals of Laparoscopic Surgery curriculum in order to teach the necessary knowledge and principles, as well as the basic psychomotor skills required to safely perform laparoscopic surgery.30

This study does suffer from limitations, which need to be considered in terms of the generalisability and applicability of the research. First, the selection of experts to engage in this group was based on a non-systematic approach, grounded on availability, engagement and past working relationships with those included in the domain group. Nonetheless, we did succeed to engage experts from various backgrounds and geographical areas. Second, the methodology used, which involved the development of a product development pathway through engagement of experts on teleconference calls, and then a survey of the pre-defined SMEs, is open to bias and also suffers from the small number of SMEs queried. Third, there may have been varying degrees of expertise with regard to health simulation in our group of SMEs, which again could have been a factor in the results achieved. Nonetheless, this study does offer a snapshot view of the current and potential role of simulation in healthcare innovation and would undoubtedly benefit from further large-scale replication to identify more clearly the forward path in this arena.

For the successful incorporation of simulation within the innovation process, the form, quality and fidelity of the simulation and its method of application are essential components that need to be well thought out prior to being used for data acquisition and analysis. Often the tools used to recreate scenarios meant to replicate real life are used inappropriately and in the wrong context and do not possess the necessary cognitive fidelity and depiction of physical environments necessary to engage participants in a meaningful way. Furthermore, the use of simulation itself can also be associated with high costs, such as with the purchase of expensive apparatuses or tissues, recruitment of parts and patients and human resources required to develop and run simulated scenarios on a given platform.10 Ultimately, early investment of capital for routinely incorporating simulation consistently when it is indispensable, and avoiding the unnecessary costs and time delays associated with research, development and regulatory approval, can eventually help innovators see a return on their investment that offsets any initial costs.


In summary, this mixed-method study of SMEs describes the many potential roles simulation can play throughout the various elements of the medical technologies innovation pathway. Those wishing to accelerate the innovation process, shorten and limit the risk associated with creatively attempting to solve difficult clinical problems by transforming good ideas into practice through innovation can use this framework in order to help guide their focus and channel resources appropriately where they are most needed, and where they will result in the development of an outcome that has the least potential to fail.



  • Twitter Follow Amin Madani @aminmadaniMD and Rajesh Aggarwal @docaggarwal

  • Contributors AM contributed to study conception, study design, data collection, data analysis and drafting manuscript. BG, CMP and RA involved in study conception, study design, data analysis and critical revision of the manuscript. DA, PB, DF, BH, SM, KM, NN, AS and VV contributed to study conception, study design and critical revision of the manuscript.

  • Funding AM is supported by a Quebec Health Science Research (FRQ-S) scholarship and the McGill Surgeon-Scientist Program. RA is a consultant for Applied Medical and is supported by the Chercheurs-Boursiers Cliniciens of the FRQ-S and the Montreal General Hospital Foundation. NN is supported by the Natural Sciences and Engineering Research Council of Canada (NSERC). This work was supported by an unrestricted donation from the Blema and Arnold Steinberg Foundation.

  • Competing interests None declared.

  • Ethics approval Institutional Review Board, Faculty of Medicine, McGill University.

  • Provenance and peer review Commissioned; externally peer reviewed.

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.