Article Text

Download PDFPDF

Virtual patient technology to educate pharmacists and pharmacy students on patient communication: a systematic review
  1. Charlotte Lucy Richardson,
  2. Simon White,
  3. Stephen Chapman
  1. School of Pharmacy and Bioengineering, Keele University, Keele, Staffordshire, UK
  1. Correspondence to Charlotte Lucy Richardson, School of Pharmacy, Keele University, Newcastle under Lyme ST5 5BG, UK; c.l.richardson{at}keele.ac.uk

Abstract

Background Virtual patients (VPs) are a sub-type of healthcare simulation that have been underutilised in health education. Their use is increasing, but applications are varied, as are designs, definitions and evaluations. Previous reviews have been broad, spanning multiple professions not accounting for design differences.

Objectives The objective was to undertake a systematic narrative review to establish and evaluate VP use in pharmacy. This included VPs that were used to develop or contribute to communication or counselling skills in pharmacy undergraduates, pre-registration pharmacists and qualified pharmacists.

Study selection Eight studies were identified using EBSCO and were quality assessed. The eligibility criteria did not discriminate between study design or outcomes but focused on the design and purpose of the VP. All the included studies used different VP applications and outcomes.

Findings Four themes were identified from the studies: knowledge and skills, confidence, engagement with learning, and satisfaction. Results favoured the VPs but not all studies demonstrated this statistically due to the methods. VP potential and usability are advantageous, but technological problems can limit use. VPs can help transition knowledge to practice.

Conclusions VPs are an additional valuable resource to develop communication and counselling skills for pharmacy students; use in other pharmacy populations could not be established. Individual applications require evaluation to demonstrate value due to different designs and technologies; quality standards may help to contribute to standardised development and implementation in varied professions. Many studies are small scale without robust findings; consequently, further quality research is required. This should focus on implementation and user perspectives.

  • virtual patient
  • simulation
  • counselling
  • education
  • pharmacy

This is an open access article distributed in accordance with the Creative Commons Attribution Non Commercial (CC BY-NC 4.0) license, which permits others to distribute, remix, adapt, build upon this work non-commercially, and license their derivative works on different terms, provided the original work is properly cited, appropriate credit is given, any changes made indicated, and the use is non-commercial. See: http://creativecommons.org/licenses/by-nc/4.0/.

View Full Text

Statistics from Altmetric.com

Introduction

Virtual patients (VPs) are a sub-type of healthcare simulation that have been underutilised.1 There are many VP applications, designs and definitions, which can incorporate modalities such as voice recognition, animations and videos. VP variations are recognised to be problematic when comparing studies2–4 and so the following definition was adopted.

A virtual patient is an interactive computer simulation of a computer programmable patient (or avatar) in a real-life clinical scenario for the purpose of medical training, education, or assessment that will respond to learner decisions. [Adapted Ellaway and Bracegirdle]5 6

General advantages of VPs are that they can deliver education to large numbers,7 they can easily incorporate assessments and are learner-centred.8 VPs have a role in mobile and remote learning which is increasingly being utilised due to advantages in accessibility and usability.9 Common uses of VPs include applications around communication, clinical reasoning and history taking skills.1 Previous barriers to VP adoption are being overcome, including recognition of the ability of VPs to pose scenarios that are missed during practice; better understanding of VP technology; and an increasing appreciation for alternative educational modalities.1

There is one meta-analysis on VPs, which considered VP uses across healthcare against either no intervention or another form of education. A mixture of quantitative intervention-controlled and non-controlled, and comparative studies were included. Direct comparisons between this range of methods were difficult and only limited conclusions could be drawn.3 The review reported that VPs have positive effects compared with no intervention (pooled effect sizes>0.80 for outcomes of knowledge, clinical reasoning and other skills), but when compared with non-computer instruction this effect was small (pooled effect sizes −0.17 to 0.10 and non-significant). A high-quality integrative review in nursing demonstrated the value of VPs for developing non-technical skills, which in part included communication, specifically development of new skills and practice opportunities.10

A narrative review by Jabbur-Lopes et al highlighted that there were only seven studies on pharmacy VPs.2 Although the review covered a broad range of technology it only included undergraduate students and did not consider differences between the technologies or their purposes but focused on the users’ experiences.2 The review concluded that VPs have ‘the potential to be an innovative and effective educational tool in pharmacy education’.2 Kane-Gill and Smithburger’s review of wider simulation in pharmacy students included two VP studies (n=17); most studies included high-fidelity manikin or technologies which did not deploy an animated interactive patient.11 Simulation improved clinical skills, but this was not discussed relative to VPs specifically.11

The focus of this review is VPs as a simulation sub-type. The objective was to establish and evaluate the literature on the use of VPs in pharmacy, where ‘VPs’ are clearly defined. This included VPs that were used to develop or contribute to, communication or counselling skills, in pharmacy undergraduate students, pre-registration pharmacists and registered pharmacists. Communication and counselling skills were chosen as there has not been a previous review which investigated this specifically, despite this being recognised as a purpose of VPs.11

Methods

As the studies used a variety of VPs, evaluation designs, methods and outcomes, a narrative review method was used since a meta-analysis was not possible.4 12 13

Eligibility

The patient, intervention, comparisons, outcomes and study design (PICOS) approach to determine eligibility and the Preferred Reporting Items for Systematic Reviews and Meta Analyses (PRISMA) guidelines were used.14 15 The eligibility criteria did not discriminate between study designs or outcomes but focused on the VP designs and purposes (table 1).15 This maximised technological eligibility; studies had to include VPs incorporating communication or counselling skills. EBSCO was used to access electronic databases (MEDLINE, Allied and Complementry Medicine Database, Cumulative Index of Nursing and Allied Health Literature and Eduction Resources Information Center) using a comprehensive series of search terms selected from Medical Subject Headings and combined using Boolean operators (figure 1). Similar to Peddle et al, only studies from post-2000 were included; no other limits were imposed.10 Electronic databases searches were accompanied by manual searches of the reference lists of eligible studies.

Table 1

Eligibility criteria for study inclusion in the systematic review

Study selection was based on the work by Moher et al (figure 1).15 The search was undertaken and duplicates removed before the titles were screened. Abstracts were then screened before an in-depth review which included an analysis of methodological quality and findings. If the nature of the VP was unclear, but a named technology was cited, a Google search was used to establish its nature. If this was still unclear the study was excluded. The study findings were thematically analysed; this approach was used due to difficulties in making direct comparisons between studies.16

Quality assessment

To assess the quality of studies an appraisal tool to evaluate educational intervention studies was used.17 The critical appraisal skills programme tools for qualitative research, cohort studies and randomised controlled trials were also used to supplement the tool by Morrison et al.18–20 The quality of the studies is summarised in online supplementary appendix 1.

Results

The 490 studies initially identified were reduced to 57 after exclusions and after full reviews 8 studies remained (figure 1). The most common reason for exclusion was that the technology did not meet the VP definition. The majority of studies which were excluded, despite describing a ‘VP’, did not include direct patient contact.

Four themes were identified in the findings of the studies: knowledge and skills; confidence; engagement with learning; and satisfaction.

Study characteristics

The studies all used different technologies and applications, further highlighting the difficulty in comparing them (table 2.). VP purposes included improving subject-specific knowledge,21–23 clinical care24 25 and communication skills.26 In relation to participants, four studies used pharmacy students,22 26–28 one study used pre-registration pharmacists22 and three used various healthcare professional (HCP) students.21 24 25 The latter was included as it incorporated pharmacy perspectives and results were available for specific types of HCP. The studies used different evaluation methods, with the the most common being questionnaires before and after using the VP; some studies used randomisation.21 24 26 27

Table 2

Characteristics of studies included in the review

All of the review studies evaluated VPs within formal educational programmes, the majority being undergraduate courses. Each VP was specific to its own application and evaluation; thus, direct comparisons of the educational values or of the evaluation outcomes are limited. The studies were selected based on their technology, but studies could have been excluded due to an unclear description. To minimise this, whole papers were closely scrutinised where clarity was lacking.

Study quality

Half of the studies (n=4) did not report whether they had received ethical approval22 25 26 28 and none of the studies discussed sample sizes considerations, although one did acknowledge that their sample was too small to statistically assess changes (online supplementary appendix 1).21

Shoemaker et al24 and Fleming et al21 each randomised two groups and had medium/high-quality studies, respectively. The randomisation process was not detailed in either study, but both took measures to standardise outcomes and address the quality and reliability of their findings. In both studies, the particular VP’s educational contexts were not discussed. In contrast, two other medium quality studies used randomised controlled designs.26 27 Bindoff et al detailed their instrument design but used a small sample (n=33), limiting findings, especially as some results were not significant.27 In the other study, by Taglieri et al, some VP cases were compulsory and some were not, possibly skewing the sample.26 Participation also declined throughout the study and this could have affected results.26

Two studies used before and after tests as a comparison without a control group.22 23 Douglass et al presented the VP’s educational context clearly, but there was limited discussion of potential bias or of how the questionnaire was developed. The study was of medium quality.22 Similarly, Zlotos et al provided a limited discussion of the VP’s educational theory and its place within a curriculum. The evaluation was of high quality but the study lost 43.4% of participants in the 6 months follow-up, although this is not uncommon in longitudinal studies.23

Loke et al’s qualitative exploratory study was of low quality. Sampling was not reported and there was limited information on the methods, but the study did report reliability, validity and trustworthiness. The VP’s educational principles were detailed.28 In Zary et al’s study, which was also of low quality the evaluation methods did not confer much depth and there were limitations due to the small-scale evaluation, although the description of the VP design was detailed.25

Knowledge and skills

All eight studies incorporated some sort of knowledge and/or skills assessment. Zlotos et al measured knowledge changes after VP use by testing students knowledge. Findings established a significant change post-VP (median scores across two topics were 16 pre-test and 19 post-test; p<0.001). This decreased after 6 months, but was higher than pre-VP (median at 6 months 16; p<0.05).23 In contrast, the study by Bindoff et al found an insignificant knowledge improvement between intervention groups and pre/post-VP use.27 Despite this, there was a significant increase in self-measured counselling competency between groups (mean difference in pre/post-self-rated competencies computer vs paper group 0.9; p=0.005) and the VP may have increased perception of counselling ability.27 These studies reported conflicting knowledge outcomes although Bindoff et al’s participants still perceived a value in VP use. This is similar to Zary et al, who reported that the VP helped students identify knowledge gaps and motivated acquisition of knowledge, although this was not measured.25

In the work by Taglieri et al, participant performance in conducting mock patient interactions was an outcome and participant performance improved after VP use (control 45.18%, intervention 53.19%; p<0.001).26 The authors discussed that a purpose of their VP was to enable reflection on performance and to apply skills and knowledge.26 This juxtaposes the participants who themselves did not perceive the VP as helpful for improving performance. Participants found the VP useful for understanding how to ask patients questions.26 Similarly, Douglass et al assessed clinical-competency skills in drug-therapy problems post-VP use using standardised patient interactions. There was rigorous development of a competency checklist which included a quality assurance process. Student performance improved by 12% across the study.22 This outcome was a useful measure, possibly more so than overtly testing knowledge. A clear value was demonstrated although like in the study by Taglieri et al, participant perspectives may contribute to establishing VP’s worth.

Further considering participant performance, Fleming et al showed significantly greater performance in excessive alcohol consumption screening and referral after VP use over 6 months (improvement in screening, control 3.7% vs experimental 14.4%). While the VP may have statistically improved participants’ screening skills there is no evidence of an improvement in problem-solving, communication and professional skills, as eluded to by the authors.21

The work by Shoemaker et al used a VP to facilitate an interprofessional educational activity; it measured competency using elements of two recognised scales (Interprofessional Education Collaborative and Readiness for Interprofessional Learning Scale) to consider interprofessional competency and communication. Results favoured the VP compared with a control (improvement on four out of five questions measuring communication: OR=20.18, p=0.000; OR=7.22, p=0.002; OR=4.64, p=0.012; OR=3.60, p=0.027),24 although the value of the VP to the user is unknown.

Confidence

Two studies assessed confidence among particular elements of participant knowledge and/or competence.23 26 Zlotos et al reported increased participant confidence immediately after VP use (p<0.001), which appeared to be maintained after 6 months although self-assessed.23 In the study by Taglieri et al, results were varied as both increased and decreased confidence were identified across different VP cases. Confidence decreased after initial modules (p=0.001), but increased to baseline after later modules (p=5.209). Participant numbers also declined by over 40% across the study (296 to 122); changes in confidence were ultimately not significant (p=0.209).26 In this case, a quantitative measure of confidence resulted in confusion and participant perspectives may have helped to explain statistical findings.

It is not clear if VPs are better than alternatives, but knowledge and confidence changes have been shown to occur and VPs may lend themselves to skill-based applications. It is worth highlighting that measuring confidence can be conflicting and a new approach to this may be required.

Engagement with learning

Elements of experiential learning occurred throughout the studies; Zlotos et al demonstrated that VPs can test clinical and ethical decision-making, and allow users to see the consequences of decisions.23 Similarly, Loke et al stated that students can ‘live through the consequences of their actions’. Students identified with the VP as a real patient, felt responsible for the outcomes of the case and were able to see the consequences of actions.28 This was similar to Zary et al, who discussed VP use for the repetitive and deliberate practice of skills in a safe and less stressful environment than with a real patient.25 Likewise, Zlotos et al identified that VPs allow for an opportunity to appreciate challenges associated with interacting with patients which students would not necessarily interact with in practice. Their simulation involved substance misusers.23 This is in line with a review by Hege et al, which recommended that authenticity and learner engagement should be considered when designing VPs.29

Educational uses of the VPs varied by the design of the VP and by the participants of the studies. Bindoff et al highlighted VP use for contextualisation of learning further supported VP use to learn from mistakes; this included experiential learning around how to frame questions.27 The study by Loke et al supports this as participants were driven to complete the case by the tasks presented rather than by generic, pre-defined steps.28 A novel discussion within the study by Taglieri et al concerned time limits for VP use. This appeared to be counterproductive to learning and demonstrated the closely linked nature of VP design and educational outcomes; it is not clear if this was rectified when the time limit for VP use was removed.26 The study also suggested that increased familiarity with the technology can improve learning.26 This is reasonable given the interlinked features of the technology with outcomes and usability.

Satisfaction

A third of the studies identified that VPs need to be available without specialist computers or software,23 25 27 and that VPs should be accessible ‘anytime anywhere’25 but, across the studies, a number of technical difficulties which impacted outcomes were reported.

In the studies by Zlotos et al, Bindoff et al, Taglieri et al and Douglass et al, significant technical difficulties were reported by the users. Technical difficulties and issues with technical design appear to limit use; these studies demonstrated the need for usable technology from the user’s perspective. Technological issues included ‘glitches’ which potentially impacted performance and competency,22 and problems with game navigation.27

Two studies discussed VP accessibility. Shoemaker et al stated that they had actively chosen technology so that the VP could ‘run on a wider variety of computer specifications without the need for the latest graphics card technology’.23 Bindoff et al identified an advantage of their system was its accessibility and usability on the most common web browsers.27 These designs were not evaluated in either study and there are no data to support these decisions.

Three studies explicitly measured or addressed technology satisfaction, and all reported positive findings.22 26 27 Douglass et al had clear results concerning pharmacy student enjoyment when using the VP (85% support).22 Two studies commented on VP use compared with paper-based alternatives,25 27 although the findings by Bindoff et al varied across different pharmacy student cohorts which made it difficult to interpret if the VP was truly more enjoyable.27 Zary et al found the VP more engaging than paper-based alternatives (I found the cases in Web-SP engaging 1=strongly disagree, 6=strongly agree, mean=4; n=10).25

Finally, Taglieri et al discussed that their technology, despite being easy to use and realistic, had a decrease in its perceived helpfulness by students across the study (84.8 and 78.4 pre-study and 73.8 and 58.2 post-study for two helpfulness questions; p<0.001).26 This suggests that even when the designers believe that the technology is usable the users may have different perceptions.

Discussion

The common findings of this review were that, despite variance in applications, VPs can improve user’s knowledge, confidence, skills and competency. The majority of the studies were not high-quality evaluations and thus should be interpreted with care especially as a number of studies reported conflicting findings within their own discussions. Some studies did not demonstrate that VP use resulted in statistically significant changes in performance but the VPs still appeared to be useful, with user benefits.

Multiple studies commented on the VP allowing an opportunity for practice, an advantage well discussed in the literature.4 30–32 Whatever the purpose of VPs, particular benefits are that they can provide richly contextualised learning applied to practice, and in such a way so that the user can safely learn from mistakes. This is in line with experiential learning where the focus is on learner-driven investigations, often in pursuit of a real or artificial task.33 Similarly, when using VPs for experiential learning users are in an active learning environment where they are able to formulate their own learning, through inquiry, problem-solving and discovery.34 This was explicitly referred to in some of the review studies,25 27 28 and links to ideas concerning reflective learning and the importance of experiences within learning.35 VPs are able to provide new and safe experiences for users and putting learners in control helps users to refine mental models of tasks.35 VPs can also allow the learner to self-regulate their own learning and focus on personal learning objectives. In order for new VPs to be useful a level of knowledge regarding experiential learning is required by facilitators and designers.35 By using VPs there is an opportunity to simulate a clinical scenario and for the user to practice, build confidence and increase accuracy.31 Accessible, standardised, safe and reliable practice opportunities are the benefits of VP reported in some manner consistently across all of the studies.

The majority of studies reported positive VP satisfaction, despite some limited delivery and usability. This was important to the study outcomes and multiple studies identified technological difficulties, which need to be considered in future development and implementation. The technology must be accessible and easy to use and maintain, as it is only once these barriers are overcome that a VP becomes beneficial.8 Similarly, there may be increased VP utilisation and implementation when more applications are developed due to a better understanding of the technology.1 The place of the VP within a curriculum needs to also be carefully considered in the future. It is important that throughout the VP design process educational principles are applied to ensure that the VP is of high educational value. This has been suggested in other VP studies for different professions.7 32 36 37 Similarly, a greater level of comparison and evaluation of VPs would be possible if there was a standard definition of a VP and quality standards for their development. This would not only overcome some technical limitations but also help to develop a better literature base for their use across health professions.

Technological delivery of VPs should be well thought out with justifications for decisions that have consequences for user satisfaction and learner outcomes, as delivery issues can distract from learning.38 It is important that functional technology is incorporated into further evaluations alongside robust outcomes. There should also be greater detail provided in VP evaluations concerning how evaluation instruments have been development and used. The review suggests that VPs are a useful resource for experiential learning, particularly for undergraduates but their role in clinical practice or for training pharmacists cannot be inferred.

All of the studies focused on establishing the use of the VPs by measuring set outcomes, similar to the review by Cook et al, who established VPs were more useful than no intervention.3 There are difficulties with this approach and the subjectivity of measuring outcomes such as confidence. What appears to be absent is research addressing the user value of such tools. Some qualitative research has been conducted on VPs but these studies have not focused on exploring the user’s perspectives and experiences.25 28 39 In this way, users may provide a perspective different from that collected when measuring ability and provide valuable insights into the apparently contradictory findings from quantitative studies.

None of the studies included registered pharmacists demonstrating a gap in the literature for the profession. Despite this, in the eight studies, the VP users, took on the role of a qualified pharmacist when using the VP. This suggests that VPs are well-suited to simulate a pharmacist’s role, but VP use has not been directly evaluated as a training method for this population. This should be addressed. Wider implications of this review are that health professionals who conduct patient counselling should consider VPs as a useful additional resource to help teach and develop communication skills, particularly in students as there are clear demonstrated advantages such as the ability to practice. The concept of a standard definition and of quality standards would better contribute to the overall literature base for VP use spanning any nature of HCPs who use VPs.

Conclusions

VPs testing communication and counselling skills have demonstrated uses for pharmacy students, but despite this, applications still require evaluation due to the demonstrated individuality of the technologies. VPs are an additional valuable resource to develop communication and counselling for pharmacy students; use for qualified pharmacists could not be established. Quality standards may contribute to standardised development and implementation of VPs in varied professions. The studies were not robust enough to fully establish the educational merit of VPs compared with other resources. Further research should focus on implementation and user perspectives.

References

View Abstract

Footnotes

  • Contributors CLR conducted the narrative review and wrote the manuscript. SC and SW planned and designed the study, and contributed to the writing of the manuscript.

  • Funding The authors have not declared a specific grant for this research from any funding agency in the public, commercial or not-for-profit sectors.

  • Competing interests None declared.

  • Provenance and peer review Not commissioned; externally peer reviewed.

  • Data availability statement There are no data in this work.

Request Permissions

If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.