Table 2

Quality analysis

Unsworth4 Unsworth5 Unsworth6 Pelaccia et al 7 Omodei10
Clear statement of aimsYes. Description of techniques to study naturalistic decision making (NDM) and case report cued-recall-debrief (CRD).Yes. To conceptualise clinical reasoning of occupational therapists (OTs).Yes. To determine differences in clinical reasoning between novice and expert OTs.Yes. To identify how emergency medicine doctors make diagnoses.Yes. To conceptualise decision making of firefighters.
Qualitative methodology appropriate?Yes.Yes.Yes. Mixed methods.Yes.Yes. Mixed methods.
Research design appropriate?Yes. Basic description.Yes.Yes.Yes.Yes. Basic description.
Recruitment strategy appropriate?Unclear.Yes: participants nominated by OT manager.Yes. Clear inclusion criteria.Yes. Multicentre. 6-point inclusion criteria checklist.No information provided.
Data collection appropriate?Unclear.Yes, detailed description.Yes, detailed description. Identical debrief prompts.Yes, detailed description of process.Yes, detailed description.
Relationship between researcher and participants considered?No critical examination of researcher’s own role.Yes. Lead researcher did not conduct debriefs.Yes. Researchers blinded to participant experience level.Yes. Researcher did not work in same hospital as participants.No critical examination of researchers’ roles.
Ethical issues considered?No description of ethics, consent or confidentiality.Yes. Ethical approval granted. Consent obtained.Yes. Ethical approval granted. Consent obtained.Yes. Ethical approval granted. Consent obtained.Unclear. No details relating to ethics or consent.
Rigorous data analysis?Anecdotal evidence. No formal analysis.Yes. Transcription of debrief commentary, a priori coding using established clinical reasoning models. 100% agreement between researchers in data coding. Thematic analysis.Yes. Transcription and a priori coding of debrief commentary. Coding discrepancies reviewed, subsequent near perfect agreement. Thematic analysis.Yes. Transcription debrief commentary, deductive coding approach with development of codebook. Intercoder reliability 96%. Thematic analysis.Yes, a priori transcription and coding of debrief commentary with thematic analysis.
Quantitative methodology commentsNA.NA.Counts of code types. χ2 analyses to determine differences between groups.NA.Counts of types of codes. Basic descriptive data.
Clear statement of findings?No. Findings jumbled with existing literature. Difficult to identify novel data.Yes. Identified types of reasoning employed and a new subset of clinical reasoning.Yes. Clear differences in way in which experts and novices employ clinical reasoning.Yes. Detailed description of when, how and number of diagnoses made.Yes. Novel finding relating to importance of emotional self-regulation.
Value of researchNovel study. Outlined two future planned studies.Yes. Discusses contribution to what is known, identifies new areas for research (broadening to other clinical settings).Yes. Implications for promoting development of clinical reasoning skills among clinicians.Yes. Discusses implications for training (improving diagnostic efficiency.) Suggests avenues for future research.Yes. Suggests future direction for research (sets out aims of study 2). Implications for training firefighters.