Table 3

Features and methodology of included studies

PaperOmodei and McLennan8 Elliot9 Unsworth4 Unsworth6
Publication year1994200020012001
Participants (n)6413 (+2 novice)
Participant typeOrienteersComputer war games
players
Occupational therapistsOccupational therapists
Gender3 male:3 femaleAll maleAll femaleAll female
Mean age16Not stated.Not stated.Not stated.
DesignCrossover.
Mixed methods.
Between-subjects comparison.
Mixed methods.
Case report.Mixed methods.
AimCompare two methods for studying naturalistic decision making
(NDM).
Compare two methods of studying
NDM.
Basic description of
technique.
Compare cognitions of experts and novices.
ContextCompetitive orienteering circuits.Mission commander in simulated air defence tasks.An encounter between occupational therapist and client during a physical rehabilitation episode.An encounter between occupational therapist and client during a physical rehabilitation
episode.
Cognitions relate toNavigation-related thoughts and feelings.Situation, intentions and constraints in relation to self and adversary.Clinical reasoning.Clinical reasoning.
ComparatorFree recall.Adversarial crew cognitive walkthrough.Concurrent reporting—free recall.
Retrospective reporting with assisted recall (audio or external video).
No.
Primary outcomeLevel of experiential immersion.
Extent of insight into decision-making processes.
Extent of insight into decision-making processes.Feasibility.
Level experiential immersion (surrogate markers of Point of view camera and visibility of cues).
NA.
Methods of data collection and/or analysis1. Orienteering review analysis checklist (quantitative).
2. Structured interview protocol comparing each technique (qualitative).
1. MacShapa program: quantity of information taken as a function of time (quantitative).
2. Transcription and coding of commentaries with quantitative count of recollections.
3. Informal researcher observations (qualitative).
Author’s own experiences.NA.
Secondary
outcome
NA.NA.NA.Compare the cognitions of experts vs
novices.
Method of data collection and/or analysisNA.NA.NA.Transcription and coding of commentaries.
Quantitative count of recollections and qualitative thematic analysis.
Coding approachNA.A priori—framework not stated.NA.A priori (Mattingly and Fleming14, Schell and Cervero15 clinical reasoning frameworks).
Statistical method
(quantitative only)
Wilcoxon signed-rank test comparing differences between
techniques.
Basic descriptive, no formal statistical analysis applied.NA.χ2 analysis for differences between groups.
Simple descriptive statistics.
Potential utilityTraining students/beginners to think like experts.Training students/beginners to think like experts.Training.Training. Trainers can target the cognitive gap that separates novices and experts.