case study in quantitative research pdf

Quantitative study designs: Case Studies/ Case Report/ Case Series

Quantitative study designs.

• Introduction
• Cohort Studies
• Randomised Controlled Trial
• Case Control
• Cross-Sectional Studies
• Study Designs Home

Case Study / Case Report / Case Series

Some famous examples of case studies are John Martin Marlow’s case study on Phineas Gage (the man who had a railway spike through his head) and Sigmund Freud’s case studies, Little Hans and The Rat Man. Case studies are widely used in psychology to provide insight into unusual conditions.

A case study, also known as a case report, is an in depth or intensive study of a single individual or specific group, while a case series is a grouping of similar case studies / case reports together.

A case study / case report can be used in the following instances:

• where there is atypical or abnormal behaviour or development
• an unexplained outcome to treatment
• an emerging disease or condition

The stages of a Case Study / Case Report / Case Series

Which clinical questions does Case Study / Case Report / Case Series best answer?

Emerging conditions, adverse reactions to treatments, atypical / abnormal behaviour, new programs or methods of treatment – all of these can be answered with case studies /case reports / case series. They are generally descriptive studies based on qualitative data e.g. observations, interviews, questionnaires, diaries, personal notes or clinical notes.

What are the advantages and disadvantages to consider when using Case Studies/ Case Reports and Case Series ?

What are the pitfalls to look for?

One pitfall that has occurred in some case studies is where two common conditions/treatments have been linked together with no comprehensive data backing up the conclusion. A hypothetical example could be where high rates of the common cold were associated with suicide when the cohort also suffered from depression.

Critical appraisal tools 

To assist with critically appraising Case studies / Case reports / Case series there are some tools / checklists you can use.

JBI Critical Appraisal Checklist for Case Series

JBI Critical Appraisal Checklist for Case Reports

Real World Examples

Some Psychology case study / case report / case series examples

Capp, G. (2015). Our community, our schools : A case study of program design for school-based mental health services. Children & Schools, 37(4), 241–248. A pilot program to improve school based mental health services was instigated in one elementary school and one middle / high school. The case study followed the program from development through to implementation, documenting each step of the process.

Cowdrey, F. A. & Walz, L. (2015). Exposure therapy for fear of spiders in an adult with learning disabilities: A case report. British Journal of Learning Disabilities, 43(1), 75–82. One person was studied who had completed a pre- intervention and post- intervention questionnaire. From the results of this data the exposure therapy intervention was found to be effective in reducing the phobia. This case report highlighted a therapy that could be used to assist people with learning disabilities who also suffered from phobias.

Li, H. X., He, L., Zhang, C. C., Eisinger, R., Pan, Y. X., Wang, T., . . . Li, D. Y. (2019). Deep brain stimulation in post‐traumatic dystonia: A case series study. CNS Neuroscience & Therapeutics. 1-8. Five patients were included in the case series, all with the same condition. They all received deep brain stimulation but not in the same area of the brain. Baseline and last follow up visit were assessed with the same rating scale.

References and Further Reading  

Greenhalgh, T. (2014). How to read a paper: the basics of evidence-based medicine. (5th ed.). New York: Wiley.

Heale, R. & Twycross, A. (2018). What is a case study? Evidence Based Nursing, 21(1), 7-8.

Himmelfarb Health Sciences Library. (2019). Study design 101: case report. Retrieved from https://himmelfarb.gwu.edu/tutorials/studydesign101/casereports.cfm

Hoffmann T., Bennett S., Mar C. D. (2017). Evidence-based practice across the health professions. Chatswood, NSW: Elsevier.

Robinson, O. C., & McAdams, D. P. (2015). Four functional roles for case studies in emerging adulthood research. Emerging Adulthood, 3(6), 413-420.

• << Previous: Cross-Sectional Studies
• Next: Study Designs Home >>
• Last Updated: Jun 13, 2024 10:34 AM
• URL: https://deakin.libguides.com/quantitative-study-designs

Quantitative Research Design and Methodology

• First Online: 04 July 2017

Cite this chapter

• Wilhelm Griga 3  

Part of the book series: Internationale Wirtschaftspartner ((INWI))

444 Accesses

To recap, following the introduction of the field of study and a review of the theoretical foundation of inpatriation, specific research questions were derived and corresponding hypotheses developed. This led to the proposed conceptual model of assignment effectiveness. A qualitative pre-study in the form of a case study was used to explore the practical relevance of the considered dimensions of assignment effectiveness and to identify competence areas of interest for the analysis of competence build-up through inpatriation.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

• Available as PDF
• Read on any device
• Instant download
• Own it forever
• Compact, lightweight edition
• Dispatched in 3 to 5 business days
• Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Unable to display preview.  Download preview PDF.

Author information

Authors and affiliations.

University of Bayreuth, Bayreuth, Germany

Wilhelm Griga

You can also search for this author in PubMed   Google Scholar

Corresponding author

Correspondence to Wilhelm Griga .

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer Fachmedien Wiesbaden GmbH

About this chapter

Griga, W. (2017). Quantitative Research Design and Methodology. In: Managing Inpatriation . Internationale Wirtschaftspartner. Springer Gabler, Wiesbaden. https://doi.org/10.1007/978-3-658-18829-0_5

Download citation

DOI : https://doi.org/10.1007/978-3-658-18829-0_5

Published : 04 July 2017

Publisher Name : Springer Gabler, Wiesbaden

Print ISBN : 978-3-658-18828-3

Online ISBN : 978-3-658-18829-0

eBook Packages : Business and Management Business and Management (R0)

Share this chapter

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Publish with us

Policies and ethics

• Find a journal
• Track your research

Research Writing and Analysis

• NVivo Group and Study Sessions
• SPSS This link opens in a new window
• Statistical Analysis Group sessions
• Using Qualtrics
• Dissertation and Data Analysis Group Sessions
• Defense Schedule - Commons Calendar This link opens in a new window
• Research Process Flow Chart
• Research Alignment Chapter 1 This link opens in a new window
• Step 1: Seek Out Evidence
• Step 2: Explain
• Step 3: The Big Picture
• Step 4: Own It
• Step 5: Illustrate
• Annotated Bibliography
• Literature Review This link opens in a new window
• Systematic Reviews & Meta-Analyses
• How to Synthesize and Analyze
• Synthesis and Analysis Practice
• Synthesis and Analysis Group Sessions
• Problem Statement
• Purpose Statement
• Conceptual Framework
• Theoretical Framework
• Locating Theoretical and Conceptual Frameworks This link opens in a new window
• Quantitative Research Questions
• Qualitative Research Questions
• Trustworthiness of Qualitative Data
• Analysis and Coding Example- Qualitative Data
• Thematic Data Analysis in Qualitative Design
• Dissertation to Journal Article This link opens in a new window
• International Journal of Online Graduate Education (IJOGE) This link opens in a new window
• Journal of Research in Innovative Teaching & Learning (JRIT&L) This link opens in a new window

Writing a Case Study

What is a case study?

A Case study is: 

• An in-depth research design that primarily uses a qualitative methodology but sometimes​​ includes quantitative methodology.
• Used to examine an identifiable problem confirmed through research.
• Used to investigate an individual, group of people, organization, or event.
• Used to mostly answer "how" and "why" questions.

What are the different types of case studies?

Note: These are the primary case studies. As you continue to research and learn

about case studies you will begin to find a robust list of different types. 

Who are your case study participants?

What is triangulation ? 

Validity and credibility are an essential part of the case study. Therefore, the researcher should include triangulation to ensure trustworthiness while accurately reflecting what the researcher seeks to investigate.

How to write a Case Study?

When developing a case study, there are different ways you could present the information, but remember to include the five parts for your case study.

Was this resource helpful?

• << Previous: Thematic Data Analysis in Qualitative Design
• Next: Journal Article Reporting Standards (JARS) >>
• Last Updated: May 29, 2024 8:05 AM
• URL: https://resources.nu.edu/researchtools

Warning: The NCBI web site requires JavaScript to function. more...

An official website of the United States government

The .gov means it's official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you're on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• Browse Titles

NCBI Bookshelf. A service of the National Library of Medicine, National Institutes of Health.

Maben J, Griffiths P, Penfold C, et al. Evaluating a major innovation in hospital design: workforce implications and impact on patient and staff experiences of all single room hospital accommodation. Southampton (UK): NIHR Journals Library; 2015 Feb. (Health Services and Delivery Research, No. 3.3.)

Evaluating a major innovation in hospital design: workforce implications and impact on patient and staff experiences of all single room hospital accommodation.

Chapter 5 case study quantitative data findings.

• Introduction

This chapter provides the results of the analysis of quantitative data from three different sources:

• Staff activity: task time distribution. Observations of staff activities were undertaken in each study ward to understand the types of tasks undertaken by staff and the proportion of time spent on each. Staff were shadowed by a researcher who logged their activities.
• Staff travel distances. These were collected by staff wearing pedometers. These data were collected before and after the shadowing sessions.
• Staff experience surveys. Staff surveys on each ward were conducted before and after the move to the new hospital and these data provide a comparison of perceptions of the ward environment in the old and new wards.

The survey probed perceptions of many aspects of the ward environment before and after the move. As discussed in Chapter 3 , the trust, the designers and stakeholders held various expectations about the benefits of the 100% single room design. We examined whether or not these expectations (or hypotheses about the effect of the move) were fulfilled. Specifically, the new hospital was designed to increase patient comfort, prevent infections, reduce numbers of patient falls, reduce patient stress, increase patient-centred care and increase the time spent by nurses on direct care (see Appendix 16 ). Concerns were raised about the possible reduction in staff observing and monitoring patients, increased travel distances and patient isolation.

This chapter primarily addresses the following two research questions:

• What are the advantages and disadvantages of a move to all single rooms for staff?
• Does the move to all single rooms affect staff experience and well-being and their ability to deliver effective and high-quality care?
• Staff activity: task time distribution results

Preliminary analysis showed that five activity categories accounted for 78% of observation data before the move and 83% of observation data after the move. This meant that numbers in the remaining categories were too low for analysis, so all subsequent analyses were confined to these five categories: direct care, indirect care, professional communication, medication tasks and ward-related activities. Proportion of time was derived by calculating the duration of each event from its start and end time, and then aggregating duration by activity for each observation session. The number of events for each activity was also counted ( Table 23 ).

Observations (events) per session before and after new build

Proportion of time spent in each type of activity was analysed using a general linear model with proportion of time as the dependent variable. The first model consisted of a single independent variable for before and after the new build and was used to ascertain the effect of the move to a new build, prior to adjusting for other variables. To this model were added ward (maternity, surgical, older people, AAU), staff group (midwife, RN, HCA) and day of the week. This second model was used to ascertain the effect of the move to the new build having adjusted for these variables.

Events were defined as a switch of activity (either to a new activity or to continue a previously interrupted activity) and were captured by a new entry in the PDA. The number of events (new or continuation of a previous activity) per hour was modelled in the same way except that a generalised linear model with a Poisson distribution and shift length in hours specified as offset (equivalent to modelling the hourly rate) was fitted to the data. An unadjusted analysis (before and after the new build only) and adjusted analysis (before and after the new build, ward, staff group and day of week) were performed.

Analysis of medication tasks was confined to RNs only. The fact that RMs work only on the postnatal ward means that it would not be possible to interpret whether any obtained results were due to the effect of the professional group or the ward. Therefore, staff group (i.e. midwives) was dropped from this model. On average the number of events (either new or continuations of previous activities) observed per session was higher before the move than after (177 vs. 153).

However, the move to the new build did not result in a significant change to the proportion of time spent on different activities ( Table 24 ). Although there was an increase in the proportion of direct care, indirect care, professional communication and medication tasks and a decrease in ward-related activities such as cleaning, bed making and stocking the utility room in adjusted analyses, none of these changes was statistically significant (see Table 24 ).

Mean proportion of time spent in each type of activity before and after move

Table 25 shows results for the analysis of the number of events per hour. The adjusted number of recorded events per hour decreased significantly for direct care ( p  = 0.039) and professional communication ( p  = 0.002), and increased significantly for medication tasks. A decrease in the number of events per hour for an activity, and no change in the proportion of time spent on that activity, suggests that there were fewer interruptions during these tasks and work was, therefore, less fragmented. This interpretation is supported by qualitative data showing that nurses could focus on direct care and communication tasks more easily in the single room environment. Staff had difficulty locating each other and also felt reluctant to interrupt a colleague providing direct care in a single room, and there were more frequent structured opportunities for professional communication within the small nursing teams.

Number of events per hour by type of activity before and after move

The number of events per hour increased significantly for medication tasks ( p  = 0.001), showing increased fragmentation for this task. Again, this interpretation is supported by the qualitative data showing that when staff entered a patient room to administer medication they were likely to engage in other direct care activities; thus medication administration was not carried out in a single medication round, but integrated into patient care activities generally.

We also assessed the changes in patients’ contact time per patient-day to check if nurses spent more time with the patient instead of doing other activities. The analysis draws on day shift observation data (based on 118.5 hours of staff shadowing before the move and 254.5 hours after the move). The proportion of contact time was applied to the total NHPPD to provide an estimate of the patients’ contact time per patient-day (see Table 26 ).

Patients’ contact time per patient-day before and after move in the case study wards

After the move, the contact time per patient-days increased in all units, apart from surgery, where there was a decrease in direct care and an increase in indirect care activities, for example medication activities and professional communication, and essential ward/patient care activities.

These changes are the result of a combination of two factors: a change in the proportion of care (i.e. an increase/decrease in the time spent with the patient) and a change in NHPPD (i.e. an increase/decrease in the number of nurses working full-time during a day).

• Staff travel distances results

Statistical analysis

The data were analysed using a repeated measures general linear mixed model (GLMM) with steps per hour as the dependent variable and pre/post new build, ward (maternity, surgical, older people, AAU), observation session (repeated measure), staff group (midwife, RN, HCA) and day of the week as independent variables. The first GLMM analysis investigated the main effects of ward, pre/post move, staff group and day of the week. The second GLMM analysis investigated the interactions between pre/post move and ward, and between pre/post move and staff group. Because midwives were employed only on the maternity ward, there was potential confounding between the effects of ward and staff type. Initial analyses confirmed that removing maternity from the analyses improved the fit of the models. The first sensitivity analysis added a variable to the model that indicated whether or not a member of staff contributed to both the pre- and post-build samples. Only five staff contributed to both. The effect on the overall results was minor. A second sensitivity analysis fitted a model to first observation session data only, but allowed data to repeat across individual staff before and after the build. We report the results below, including where sensitivity analyses identified differences.

The data set contains information on 140 sessions collected on 53 staff (49%) prior to and 56 staff (51%) after the new build. A number of staff contributed more than one observation session: 85 provided one session, 18 provided two sessions, five provided three sessions and one provided four sessions. There were 73 sessions (52%) collected prior to the new build and 67 sessions (48%) after the new build. The average numbers of sessions per member of staff were 1.38 and 1.20, respectively. A small number of staff ( n  = 5, 4%) were observed at both times (one RN and four HCAs). Table 27 shows descriptive data for ward and staff group.

Steps per hour before and after new build

The unadjusted means (see Table 27 ) show an increase in the number of steps per hour for all wards and staff groups. Staff working on the older people’s ward (from 664 to 845) and RNs (from 639 to 827) have seen the biggest increases.

Table 28 shows results for the main effects of ward, pre/post move, staff group and day of the week. The number of steps per hour increased significantly from a mean of 715 before the move to a mean of 839 [ F (1,83) = 10.36; p  = 0.002] after the move. HCAs took significantly more steps per hour than nurses [ F (1,83) = 8.01; p  = 0.006]. There were also significant differences between days of the week [ F (4,21) = 3.40; p  = 0.027]. There was no significant difference between wards in the distances travelled ( Table 29 ).

F -tests on main effects

Mean steps per hour by wards, pre-/post move, staff group and day of the week

Table 30 shows results for the interactions between pre/post move and ward, and between pre/post move and staff group. Neither of the two interactions was statistically significant.

F -tests on interaction effects

The estimated marginal means ( Table 31 ) showed that there was an increase from pre to post build across all wards. Although the size of this increase did not differ significantly between wards, the increases in the surgical and older people’s wards were larger than for the AAU. RNs experienced a larger increase (from 624 to 811) in the number of steps per hour (from 3.74 to 4.86 miles) than HCAs (from 828 to 862 steps; from 4.96 to 5.17 miles).

Mean steps per hour for the interactions

The estimated marginal means from the second sensitivity analysis suggested a decrease in the number of steps per hour for the AAU from 901 to 836 and for HCAs from 876 to 855, rather than an increase as shown in Table 31 . The change in means for the remaining two wards and for RNs, from pre to post build, were in the same direction, and of the same order of magnitude (see Table 31 ).

• Staff experience survey

Because of staff leave, shift patterns and staff turnover during the course of the study, it was not possible to use a completely within-subjects design, in which the pre- and post-move surveys were completed by the same people. Despite this, 19 participants did complete surveys at both times, which meant a mixed within- and between-subjects design. One potential problem with this is that the subgroup who completed both surveys could have been sensitised to the research questions and, therefore, could have been more likely to report differences after the move than those who completed only one survey; that would bias our results. We addressed this by treating the design as a between-subjects design and checking for bias by comparing the results of our analyses for the whole group with separate within-subjects analyses on the subgroup who completed both surveys. The results were identical except for a small difference: perceptions of the effect of the accommodation on the delivery of care approached significance (0.099) in the within-subjects analysis whereas for the whole group this effect was significant (0.011). This can be attributed to lack of power in the subsample of 19. On this basis we proceeded with the analysis by treating the ‘before’ and ‘after’ samples as independent groups.

There were 152 items in the staff survey. Our approach to analysis was multifaceted. First, we explored the potential for grouping questions into subscales that would summarise a topic area. We thematically analysed the questions to determine those that were likely to be measuring attitudes to related aspects of the ward design, and then tested these subscales using statistical reliability analysis. Where reliability was not adequate we revised the items in the subscales until we had identified coherent subscales. These were then analysed using independent sample t -tests to determine if post-move responses were significantly different from the pre-move scores for each subscale. Similar analyses were undertaken for the teamwork and safety climate scales. Qualitative open-ended questions were analysed thematically using a content analytic approach. The well-being and stress items were compared before and after the move using the Pearson chi-squared test and Fisher’s exact test when expected frequencies were less than 5.

One of the aims of the study was to investigate if there were differences between the case study wards in their perceptions of the positives and negatives of the new single room accommodation. However, the relatively small number of staff in each of the case study wards meant that it was not possible to explore this question statistically. We therefore used correspondence analysis and perceptual mapping to examine the interaction between ward attributes and case study wards. Correspondence analysis is an exploratory mapping tool that allows visualisation of relationships in the data that would be difficult to identify if presented in a table. 114 It is related to other techniques such as factor analysis and multidimensional scaling. It does not rely on significance testing and is best viewed as an exploratory technique that provides insights into the similarities and differences between two variables. 115 Correspondence analysis does not address questions of whether or not there were differences in ratings between the attributes (e.g. whether or not privacy for patients was rated more highly than staff teamwork). Instead, it focuses on the differences between case study wards and the interaction between ratings and wards. It allows an examination of to what extent which wards are associated with particular ratings. In this way it allows us to qualitatively explore the quantitative data.

Ward environment survey subscales

Ten reliable subscales were formed. Table 32 shows the subscales and example items from each.

Description of subscales

Appendix 19 contains a complete list of all items used for each subscale.

Table 33 summarises the statistical analysis of the subscales showing means, Cronbach’s alpha and the number of items for each subscale before and after the move. According to accepted criteria, 115 alpha above 0.60 is acceptable for exploratory analyses, above 0.70 is acceptable for confirmatory purposes and above 0.80 is good for confirmatory purposes. Obtained coefficients were generally good, ranging mostly between 0.67 and 0.92. The lowest alpha, of 0.53, was obtained for the family/visitors subscale after the move, suggesting that this subscale is not internally consistent. However, the pre-move alpha was good (0.70), so it was decided to retain this subscale for exploratory purposes.

Mean subscale scores and reliability analysis before and after the move

Table 34 shows the results of independent sample t -tests comparing subscale scores before and after the move. Staff perceived significant improvements in the efficiency of the physical environment, the patient amenity, the effect of the environment on infection control, patient privacy, and family and visitors. The largest increases were found for perceptions of infection control and patient privacy. Perceptions of the effect of the ward environment on teamwork and care delivery were significantly more negative after the move. There were no significant differences in staff perceptions of staff facilities, patient safety and staff safety.

Results of t -tests comparing perceptions of the ward environment before and after the move

Although all subscales showed moderate to very good reliability, changes were not uniform for all items in every subscale; there were some exceptions to the overall trend. Overall ratings for the subscale ‘efficiency of physical environment’ increased, but ratings for the item ‘ward design/layout minimises walking distances for staff’ decreased. These perceptions were confirmed by our findings from the analysis of travel distances showing that staff took significantly more steps after than before the move. Some aspects of the design increased the amenity of the ward for staff but others did not. For example, staff toilet facilities, locker facilities and space at staff bases were rated more highly but ratings for social interaction and natural light decreased. These positive and negative aspects meant there was no significant difference in staff amenity before and after the move. The new ward was rated as much more positive for patients but there were reduced scores for three items after the move: social contact between patients, ability of patients to see staff and way finding. All aspects of teamwork and training were rated less positively, except for the item ‘discussing patient care with colleagues’, which increased. This finding is supported by our analysis of observation data showing that professional communication activities were less fragmented.

Although there were no significant differences in the effect of the ward layout on perceptions of patient safety, examination of the items showed that ratings for two items increased (‘minimising risk to patients of physical/verbal abuse from other patients/visitors’ and ‘minimising the risk of medication errors’) while ratings for two items decreased (‘responding to patient calls for assistance’ and ‘minimising the risk of falls/injury to patients’). This suggests that, although staff thought some risks to safety were reduced, they perceived an increased risk of falls and delays in responding to calls for assistance. Staff perceptions of a rise in risk of falls are detailed in Chapter 6 . Staff also reported being unable to hear calls for assistance when in a single room with a patient.

There were five items that did not fit into any of the subscales. These items were analysed singly using Fisher’s exact test and the results are shown in Table 35 . There was a significant relationship between the move and ratings for the number and location of hand basins, ease of keeping patient areas clean and quiet, and the overall comfort of patients, which all increased after the move. There was no relationship between the move and judgements of whether or not the location of the dirty utility room (where bedpans are stored and disposed of) reduces cross-contamination.

Results of single-item analyses

The distribution of responses for the four significant items showed that significantly more staff rated these aspects of single room accommodation as more positive after the move than before ( Tables 36 – 39 ).

Distribution of responses for the item ‘Number and location of CHWBs supports good hand hygiene’

Distribution of responses for the item ‘Easy to keep patient care areas clean’

Distribution of responses for the item ‘Overall comfort of patients’

Distribution of responses for the item ‘Easy to keep patient care areas quiet’

Expectations before the move and reality after the move

Before the move, staff were asked to rate on a five-point scale whether they thought single rooms would be better or worse for different aspects of clinical work (e.g. minimising the risk of patient falls, maintaining patient confidentiality, knowing when other staff might need help). After the move they again rated whether single rooms were better or worse for clinical work, thus providing a measure of whether or not their expectations about single rooms were met in reality. The questions were a subset of 23 questions from the first part of the survey and were analysed using Fisher’s exact test.

Results ( Table 40 ) showed that staff perceptions of whether or not single rooms were better than multibedded wards changed after the move for five items. Staff perceptions of whether or not single rooms were better for responding to calls for assistance, knowing when other staff might need help and minimising walking distances were rated as worse or much worse by significantly more staff after than before the move. Staff rated single rooms as positive for patient sleep and rest and for interactions between patients and visitors after the move.

Relationship between expectations before the move and reality after the move

Tables 41 – 45 show the distribution of significant responses.

Distribution of responses for the item ‘Responding to patient calls for assistance’

Distribution of responses for the item ‘Minimising staff walking distances’

Distribution of responses for the item ‘Patient sleep and rest’

Distribution of responses for the item ‘Knowing when other staff might need a helping hand’

Distribution of responses for the item ‘Patient interaction with visitors’

Teamwork and safety climate survey

To take into account our changes to the survey, we combined the four items about the quality of communication with doctors, nurses, nursing assistants and AHPs with the items in the information handover subscale to form a new subscale of seven items. Although this is different from the scales reported by Hutchinson et al. , 98 reliability analysis confirmed the original factor structure of the survey. There were two teamwork subscales and three safety climate subscales with good to high reliability ( Table 46 ). See Appendix 20 for a list of the items contained in each subscale.

Mean scores for all subscales decreased following the move. Independent sample t -tests showed that ratings for information handover and communication decreased significantly following the move [ t  = 3.23, degrees of freedom (df) = 108, p  = 0.002], indicating that information exchange and sharing within teams was perceived to be worse after the move. There were no other significant differences.

Correspondence analysis

Correspondence analysis transforms cross-tabulated data into a biplot showing distances between variables. In this study, case study ward was a column variable and mean questionnaire subscale score was a row variable (see Table 33 ). As appropriate when analysing mean scores, Euclidean distance was used and standardisation by removing row means was used. 114 , 116 This means that differences between the subscale means were not represented in the perceptual map, as we were not interested in whether or not, for example, infection control was rated more highly than privacy. Differences between wards, contained in the columns, were of interest and are represented in the perceptual map. Separate analyses were conducted for before and after the move and for the ward attributes and teamwork/safety climate survey.

Figure 11 shows perceptual maps of the association between ward attributes and wards before and after the move. The pre-move map shows that the points on the map were dispersed, indicating that the ratings were not strongly associated with particular wards. There was one exception in that ratings for the efficiency of the physical environment, privacy and infection control were higher for the older people’s ward than for the other wards. The post-move map shows that the highest ratings for the efficiency of the physical environment, the delivery of care, the staff facilities and teamwork were obtained in the older people’s and surgical wards, indicated by proximity on the map. Ratings for patient amenity, infection control, privacy and family/visitors were highest for the surgical ward. High ratings for patient safety were obtained in maternity and the surgical ward. Ratings for staff safety were similar in the older people’s, surgical and maternity wards. The acute assessment ward was not associated with any particular ward attributes, as was the case before the move.

Perceptual maps of (a) pre- and (b) post-move ward attributes by ward.

Figure 12 shows perceptual maps before and after the move of the association between teamwork/safety climate ratings and wards. The teamwork/safety climate survey consisted of two teamwork subscales – team input into decisions, and information handover and communication – and three safety climate subscales – attitudes to safety within own team, overall confidence in safety of organisation and perceptions of management attitudes to safety. The pre-move map shows that ratings of input into decisions, information and handover, and overall confidence in safety of the organisation were highest for the acute assessment ward. Ratings of safety attitudes within the team and management attitudes to safety were highest for the surgical ward. After the move, the surgical ward had the highest ratings for safety attitudes within the team, overall attitudes to safety and management; ratings for team input into decisions and information handover and communication were highest for the older people’s ward. Ratings for all safety climate subscales decreased in the acute assessment ward, which is indicated on the perceptual map by its location in a quadrant by itself. Maternity scores did not show a consistent pattern.

Perceptual map of (a) pre- and (b) post-move ratings of teamwork/safety climate by ward. Att., attitude; mgt., management.

These maps reveal some differences between wards in perceptions of the ward environment and show that perceptions were different before and after the move.

Staff ward preferences

Nursing staff were asked to indicate whether they would prefer single rooms, multibedded accommodation or a combination. There was a range of views ( Figure 13 ). In each phase, fewer than 18% of staff indicated a preference for 100% single rooms. The most common preference in each phase was a combination of 50% of beds in single rooms and 50% in bays (see Figure 13 ). In the pre-move survey, more staff reported a preference for more beds in bays ( n  = 20) than in the post-move phase ( n  = 12).

Nurse preferences for single room or multibedded accommodation.

Staff stress and well-being

There were five categorical questions about staff well-being that investigated whether or not they had experienced injuries and harassment in the previous 12 months ( Table 47 ). There were three items about job stress that asked participants to rate their stress on a five-point Likert scale ( Table 48 ) . Results showed no differences in staff well-being and stress before and after the move.

Relationship between move and staff well-being

Relationship between move and staff stress

Staff were asked 10 questions about their satisfaction with their own performance of various tasks during their last shift, and one question about their overall job satisfaction. Results ( Table 49 ) showed no significant effect for any of the job satisfaction items.

Relationship between job satisfaction and move

Qualitative survey data

Four open-ended questions were used to gain qualitative data about staff attitudes. The questions were:

• What two things do you think would most improve the current ward environment for staff?
• What two things do you think would most improve the current ward environment for patients?
• What two things are you most looking forward to in relation to the move to 100% single rooms in the new hospital?
• What two things are you most concerned about in relation to the move to 100% single rooms in the new hospital?
• What two things do you like the most about single room wards in the new hospital?
• What two things do you dislike most about single room wards in the new hospital?

In the following sections we present the results of the thematic analysis with frequency data (almost equal numbers of staff responded before and after the move, n  = 55 and n  = 54 respectively) and examples from participants’ written responses where appropriate. Table 50 shows that staff identified a number of things that would improve the ward accommodation for patients. The need for more space, improved patient facilities, privacy, and rest and sleep were largely met, since there were fewer people identifying these as needs after the move. However, the need for improved patient–staff ratios and a day room to provide patient social interaction were still reported after the move.

What would improve the current ward environment for patients ? Response frequencies

The need that staff perceived before the move for space around patient beds and staffing levels had decreased after the move ( Table 51 ). However, ventilation/heating/lighting, access to equipment and supplies and facilities for staff, including staff bases, were identified as needing improvement after move. In addition there was a need for improvements in monitoring patients, keeping track of colleagues, reducing isolation and reducing walking distances. These have all been identified by other parts of our results (see Chapter 6 ).

What would improve the current ward environment for staff ? Response frequencies

Staff were asked about the features of the ward they were most looking forward to in the pre-move phase, and most liked in the post-move phase ( Table 52 ). Results showed that staff most liked the increased patient privacy, patient sleep and rest, increased space, working in a modern environment and improved patient bathroom facilities.

What are you most looking forward to/do you most like about 100% single room accommodation? Response frequencies

Table 53 shows that staff were most concerned about being able to monitor patients, patient isolation and the risk of falls. Being unable to find staff and increased walking distances also emerged as features staff disliked about single rooms.

What are you most concerned about/do you most dislike about 100% single room accommodation? Response frequencies

• Most staff would prefer a mix of single rooms and multibedded rooms on wards.
• Staff activity events observed per session were higher after the move and direct care and professional communication events per hour decreased significantly, suggesting fewer interruptions and less fragmented care.
• A significant increase in medication tasks among recorded events suggests medication administration was integrated into patient care activities and was not undertaken as a medication ‘round’.
• Travel distances increased for all staff, with highest increases for staff in the older people’s ward and surgical wards and for RNs/RMs.
• efficiency in carrying out tasks
• patient amenity, including comfort, space, sleep, light and ventilation
• infection control
• patient privacy
• patient interaction with family/visitors and their involvement in care.
• In open comments, staff most liked the increased patient privacy, working in a modern environment, improved patient sleep and rest, and space around the bedside.
• delivery of care, including factors such as spending time with patients, communication with patients, monitoring patients and remaining close to patients, responding to calls for assistance, minimising the risks to staff, minimising walking distances and staff spending time with patients
• teamwork, including being able to locate staff, obtain assistance from colleagues, informal learning, keeping team members updated, discussing care with colleagues and knowing when other staff might need help.
• In addition, in open comments staff were most concerned about patient isolation, the risk of falls and staff isolation.
• There were no perceived differences in staff amenity and patient and staff safety.
• Ratings for information handover and communication decreased significantly following the move. This suggests that information exchange and sharing within teams – and between professions – was perceived to be worse after the move.
• Different wards valued different aspects of the ward environment.
• Ratings for staff toilet facilities, locker facilities and space at staff bases were rated more highly but ratings for social interaction and natural light decreased.
• No differences were found in staff job satisfaction, well-being or stress before and after the move.
• The need for improved patient–staff ratios and a day room to provide patient social interaction was still reported after the move.

Included under terms of UK Non-commercial Government License .

• Cite this Page Maben J, Griffiths P, Penfold C, et al. Evaluating a major innovation in hospital design: workforce implications and impact on patient and staff experiences of all single room hospital accommodation. Southampton (UK): NIHR Journals Library; 2015 Feb. (Health Services and Delivery Research, No. 3.3.) Chapter 5, Case study quantitative data findings.
• PDF version of this title (96M)

In this Page

Other titles in this collection.

• Health Services and Delivery Research

Recent Activity

• Case study quantitative data findings - Evaluating a major innovation in hospita... Case study quantitative data findings - Evaluating a major innovation in hospital design: workforce implications and impact on patient and staff experiences of all single room hospital accommodation

Your browsing activity is empty.

Activity recording is turned off.

Turn recording back on

Connect with NLM

National Library of Medicine 8600 Rockville Pike Bethesda, MD 20894

Web Policies FOIA HHS Vulnerability Disclosure

Help Accessibility Careers

• Open access
• Published: 11 June 2024

Perception of enhanced learning in medicine through integrating of virtual patients: an exploratory study on knowledge acquisition and transfer

• Zhien Li 1 ,
• Maryam Asoodar 1 ,
• Nynke de Jong 2 ,
• Tom Keulers 3 ,
• Xian Liu 1 &
• Diana Dolmans 1  

BMC Medical Education volume  24 , Article number:  647 ( 2024 ) Cite this article

227 Accesses

Metrics details

Introduction

Virtual Patients (VPs) have been shown to improve various aspects of medical learning, however, research has scarcely delved into the specific factors that facilitate the knowledge gain and transfer of knowledge from the classroom to real-world applications. This exploratory study aims to understand the impact of integrating VPs into classroom learning on students’ perceptions of knowledge acquisition and transfer.

The study was integrated into an elective course on “Personalized Medicine in Cancer Treatment and Care,” employing a qualitative and quantitative approach. Twenty-two second-year medical undergraduates engaged in a VP session, which included role modeling, practice with various authentic cases, group discussion on feedback, and a plenary session. Student perceptions of their learning were measured through surveys and focus group interviews and analyzed using descriptive statistics and thematic analysis.

Quantitative data shows that students highly valued the role modeling introduction, scoring it 4.42 out of 5, and acknowledged the practice with VPs in enhancing their subject matter understanding, with an average score of 4.0 out of 5. However, students’ reflections on peer dialogue on feedback received mixed reviews, averaging a score of 3.24 out of 5. Qualitative analysis (of focus-group interviews) unearthed the following four themes: ‘Which steps to take in clinical reasoning’, ‘Challenging their reasoning to enhance deeper understanding’, ‘Transfer of knowledge ‘, and ' Enhance Reasoning through Reflections’. Quantitative and qualitative data are cohered.

The study demonstrates evidence for the improvement of learning by incorporating VPs with learning activities. This integration enhances students’ perceptions of knowledge acquisition and transfer, thereby potentially elevating students’ preparedness for real-world clinical settings. Key facets like expert role modeling and various authentic case exposures were valued for fostering a deeper understanding and active engagement, though with some mixed responses towards peer feedback discussions. While the preliminary findings are encouraging, the necessity for further research to refine feedback mechanisms and explore a broader spectrum of medical disciplines with larger sample sizes is underscored. This exploration lays a groundwork for future endeavors aimed at optimizing VP-based learning experiences in medical education.

Peer Review reports

In Medical Education, a persistent challenge lies in the bridge between acquiring theoretical knowledge and applying it in real-world clinical scenarios. Many medical students struggle with translating their classroom learning into practical settings. The primary challenge lies in effectively translating the concepts students have learned into authentic patient interactions. This gap is particularly concerning because it affects the quality of patient care, as medical students are not just learning to acquire knowledge but must be able to apply this knowledge in complex healthcare settings.

One approach to address this challenge is the use of Virtual Patients (VPs), a computer-based simulation of real-life clinical scenarios for students to train clinical skills [ 1 ]. Research has shown that using VPs in the classroom can effectively improve various aspects of learning, from core knowledge and clinical reasoning to decision-making skills and knowledge transfer [ 2 , 3 , 4 , 5 ]. The VPs provide students with the opportunity to practice skills in a safe and controlled simulation environment.

Recent studies have focused on optimizing the design and arrangement of VPs as part of learning activities to facilitate both knowledge acquisition and retention [ 6 , 7 , 8 ]. For instance, Verkuyl, Hughes [ 8 ] demonstrated that using VPs as gamification tools can improve students’ confidence, engagement, and satisfaction.

However, studies focusing on the specific factors that contribute to these improvements when integrating VPs into the classroom are limited, particularly in understanding how to use VPs in the classroom to facilitate the transfer of knowledge students’ gain from the class to the subsequent studying stage of their education and eventual practice.

Acquisition and transfer of knowledge are critical factors in medical education, as medical students must be able to apply their knowledge and skills to real-world clinical scenarios [ 9 ]. Research suggests that for the effective transfer of knowledge, students should be immersed in authentic environments, enabling the transition of learned competencies to advanced stages [ 10 , 11 , 12 , 13 ].

Despite the consensus on the efficacy of VPs as a tool, there is a gap in understanding how to integrate VPs in the classroom to optimize students’ learning, especially in facilitating learning transfer. The effectiveness of VPs is not just in their use but also in how they are used by students to enhance their understanding on how to reason and make decisions about medical treatments when dealing with clinical cases. Without a clear and deep understanding, we risk underutilizing their potential and losing opportunities for medical students to become well prepared for real-world clinical scenarios.

Certain elements, such as role modeling instruction [ 14 , 15 , 16 ], using various authentic cases [ 17 , 18 , 19 ], and engaging in peer discussions on feedback [ 20 , 21 , 22 ], emerge as potential key components that could be integrated to maximize the knowledge acquisition via VPs. For instance, Stalmeijer, Dolmans [ 23 ] show how an expert, serving as a role model, provides guidance that facilitates student learning by demonstrating clinical skills and reasoning out loud. While there is ample evidence supporting the advantages of inclusion of VPs in education, there is not enough research focusing on the detailed aspects of effective instructional design techniques. This paper delves into these components, seeking to understand how the VP integration influences students’ learning and knowledge transfer. Figure  1 shows the theoretical framework of how integrating VPs in class affects students’ learning and might impact the transfer of learning in a simulated VP environment to practice.

Relationship of implementing, impact factor, and transfer of training

This exploratory study aims to investigate how instructional design elements such as role modeling, various authentic cases, and peer dialogues on feedback within VP sessions affect students’ learning from the learner’s perceptions. The core research question in this study focuses on how the implementation of role modeling, various authentic cases, and peer dialogue on feedback in VPs, influences learners’ perception of knowledge gain and transfer in personalized medicine.

The study was conducted at Maastricht University in the elective course, “Personalized Medicine in Cancer Treatment and Care”. This course is open to second-year undergraduate medical students of Maastricht University.

Participants

Initially, 24 students enrolled in this course for the academic year of 2022–2023, and 22 students participated in the Virtual Patient session. In total, 19 students voluntarily completed the survey designed to evaluate their experiences and perceptions of the Virtual Patients session. Thereafter, 9 of the 19 survey respondents voluntarily agreed to participate in three focus group interviews, with 2–4 students in each focus group. Students were informed that participation in this research study had no impact on student’s academic performance or their continuation in their studies.

Intervention

The instructional approach for the VP cases was structured in a specific format for the students. Figure  2 shows the instructional design for VP integration. The first stage was a role-modeling phase, where an expert demonstrated the clinical reasoning process using VP Case A. This was followed by a practice session where students worked in pairs on two different VP cases (Case B and C). After that, students formed two larger groups each including 5 or 6 students, and discussed the system feedback that was provided by VP platform. Finally, the expert summarized the session and addressed students’ questions. The whole intervention lasted 120 min. Figure  1 gives an overview of the intervention steps.

The flow of integrated virtual patient session

1. Role modeling (30 min): The intervention started with an expert, a clinician with teaching experience, demonstrating a clinical case (Case A) and showing the clinical reasoning process by thinking aloud. The expert served as a role model in showcasing the approach toward clinical problem-solving, provided supportive information, and demonstrated how to proceed through the case. The aim of the role modeling session was to empower students to apply the insights and methodology gained from experts in case A to solve subsequent cases (case B and case C), Although these cases shared similarities in underlying principles, they diverged on patient characteristics such as age, complications, and smoking history that can influence patient treatment outcomes.

2 and 3. Two VP pair tasks (20 min each): In this segment, the 22 participating students were paired, resulting in 11 pairs. These pairs were then divided into two groups. Group 1 (6 pairs) and group 2 (5 pairs) alternated in going through Case B and Case C to account for the practice effect. These cases were variations of the clinical cases introduced during the role-modeling demonstration, differing in patient characteristics such as age, complications, and smoking history to challenge the students’ reasoning. Students were encouraged to work collaboratively.

4. Feedback discussion (30 min): Upon completion of the VP cases, an automated feedback is immediately provided about the reasoning analysis. Participants were instructed to save this feedback for later discussion. After that, Students were organized into groups of six, based on the sequence in which they engaged with the cases. For instance, those who first practiced with Case B and then proceeded to Case C formed Group (1) Conversely, students who started with case C and then moved on to case B were assembled into Group (2) To foster meaningful dialogue, students engaged in discussions focused on the feedback generated by the Virtual Patient system, guided by a printed discussion guide distributed to each group (see Appendix 2 ). The discussion aimed to deepen students’ understanding and enrich their conversations about the cases they had just completed.

5. Plenary (15 min): This part lasted 15 min. Hosted by the expert to summarize the session and address questions or doubts raised by students.

During the practice and discussion sessions, the expert circulated among the groups to offer additional guidance and support.

The virtual patient cases

Three Virtual Patient (VP) cases (Case A, B, and C) were created to enhance students’ comprehension of specific concepts, knowledge, and skills in clinical reasoning. The VP practice was developed on the P-Scribe ( www.pscribe.nl ) learning platform, a web-based e-learning system based in the Netherlands. The platform facilitates the design and implementation of text-based VP sessions (Appendix 4 ).

While these cases shared a foundation on authentic head and neck cancer treatment, they were characterized by varying patient characteristics in terms of age, gender, and medical history (anamnesis).

VP case flow chart

Within each VP case, students were presented with a scenario related to neck cancer. Figure  3 shows the chart of a VP case. Each case starts with an overview of the patient and their medical history which students had to use to make an initial assessment. After this, students encountered a mix of multiple-choice and open-ended practice questions. These questions guided students in planning diagnostics, formulating a diagnosis, and devising a treatment plan tailored to the patient’s specific needs. Immediate feedback was provided after students submitted each response, and comprehensive summative feedback was given at the conclusion of each case to foster understanding and learning from any potential misjudgments or oversights (See Appendix 4 ).

Measurement instruments

Learning-perception survey : The survey (Appendix 1 ) consisted of 20 items, structured into five primary sections: general experience, intended learning outcome, role modeling, practicing with various authentic cases, and reflection on peer dialogue around feedback. The first item asked about students’ general experience through the whole session. The second item focused on their perception of intended learning outcomes. Six items then focused on the students’ perceptions of learning through role modeling followed by 5 items addressing perceptions related to their learning on practicing with authentic cases. The final seven items explored students’ perception of learning from dialogue around feedback. Participants indicated their level of agreement for each statement using a 5-point Likert scale: 1 denoting “Strongly Disagree”, 2 for “Disagree”, 3 for “Neutral”, 4 for “Agree”, and 5 for “Strongly Agree”. For interpretation, average scores below 3 were considered as “in need for improvement”, those of 4 or higher as ‘good’, and those between 3 and 4 as ‘neutral’.

Focus group interviews : Three focus group interviews (Appendix 3 ) were conducted to dive deeper into students’ perceptions of their learning experience, knowledge gain, and knowledge transfer in real-world settings. The focus group took place after the survey and the survey data did not affect the development of the focus group questions. In focus group 1, two students, in focus group 2, two students and in focus group 3, five students participated. The interviews were structured around a series of questions that explored students’ perceptions of their learning across specifically designed sections. These sections included Role Modeling, Practice with Various Authentic Cases, and Dialogue around Feedback. The structure aimed to understand students’ perspectives on each key component of the learning sections.

The analysis of the survey data was conducted by calculating the mean, standard deviation, and the Alpha Coefficient for the responses pertaining to each of the five key dimensions of the survey. The mean score provided an indicator of the average student perception, while the standard deviation offered insights into the variability of the responses. The Alpha Coefficient, a measure of internal consistency, was computed to assess the reliability of the survey dimensions. Through these statistical measures, an overall understanding of the students’ perceptions regarding the various aspects of the Virtual Patients was attained, facilitating a robust analysis aligned with the research objectives.

The focus-group interview data were analyzed following the thematic analysis procedure set out by Braun and Clarke [ 24 ]: (1) familiarize yourself with your data, (2) generate initial codes, (3) search for themes, (4) review themes, (5) define and name themes, and (6) produce the report. The interview was guided by pre-existing frameworks or theories in medical education. This ensured the capture of major aspects of the VP learning experience as underscored in the existing literature: role modeling, using various authentic cases, and peer dialogue around feedback [ 16 , 17 , 18 , 20 , 21 ]. The focus group interview was recorded, transcribed, and coded by three team members and ordered in initial themes (Z.L, M.A, and X.L). These themes were discussed with the larger team. We used a process of inductive and deductive analysis and used the three design principles of role modeling, practice with various authentic cases, and group discussion on feedback as sensitizing concepts to study the data [ 24 ]. Thereafter, quantitative and qualitative analyses were collectively appraised, compared, and checked for inconsistencies. In this triangulation, the themes identified in focus-group interviews were explanatory to the descriptive statistics of the survey.

Trustworthiness

Several measures were taken to enhance the study’s trustworthiness. First, triangulation was achieved by employing multiple data collection methods, including surveys and focus group interviews. The interview data collection continued until saturation was reached, ensuring a comprehensive understanding of the student’s experiences and perceptions. Secondly, the coding process followed an iterative approach. Team members initially coded transcripts independently, and then met to reach a consensus before moving on to code subsequent transcripts. Three researchers conducted the coding independently to minimize bias and enhance the validity of the findings. Finally, a member check among a sample of the focus group interviewees was conducted. In response to the question asking whether they agreed with summaries of preliminary results and would provide comments, confirmatory responses were received as well as some minor additional comments and clarifications. The latter were taken into account in the analysis and interpretation of the data.

Ethical approval

The Maastricht University Ethical Committee reviewed and approved this study. The approval number is FHML-REC/2023/021.

The findings from both the survey data and focus group interviews were presented to explore students’ perceptions of the effectiveness of the Virtual Patient (VP) Session in enhancing their clinical reasoning skills.

Survey data

The survey explored students’ perceptions across five key dimensions: General Experience, Intended Learning Outcome, Role Modeling, Practicing with Various Authentic Cases, and students’ reflection on Peer Dialogue around Feedback. The students scored the VP sessions on 20 items (Table  1 ). The scores varied between M = 2.95 to M = 4.58, on a scale of 1–5.

For the General Experience of Virtual Patient Session (Items Q1-Q2) the average score was M = 4.13 (SD = 0.70). Specifically, the overall experience was positively rated at M = 4.11. The component that assessed the improvement of clinical reasoning skills received an average score of M = 4.16.

Regarding the Students’ Perception of Learning from Role Modeling (Items Q3-Q8), the average score was M = 4.38 (SD = 0.61). Students agreed that the expert demonstration at the start of the session helped them understand the intended learning outcomes and was useful in guiding them through the Virtual Patient cases, with scores ranging from M = 4.26 to M = 4.58.

Students’ perception of learning from practicing with various authentic cases (Items Q9-Q13), received an average score of M = 4.00 (SD = 0.86). The scores measured the students’ perception of how well the provided Virtual Patient cases matched their current level of understanding, enhanced their comprehension of the subject matter, and helped them grasp the complexities inherent in real-world clinical scenarios.

For their perception of learning from Peer Dialogue around Feedback (Questions 14–20), the average score was M = 3.24 (SD = 1.05). These scores measure the students’ perception of the effectiveness of peer dialogue in enhancing understanding, generating strategies to address feedback, and prioritizing areas of improvement.

Focus group interview data

The interviews revealed five themes: ' Which steps to take in clinical reasoning’, ' Asking challenging questions to enhance deeper understanding of knowledge’, ‘The variety in cases helps to enhance transfer to the real world’, and ‘Deeper understanding of reasoning through reflections’.

Which steps to take in clinical reasoning

Students acknowledged the expert’s initial demonstration helped them to develop structured knowledge and gain understanding of the clinical reasoning process.

I think it (Role modeling) helps to find a pattern in clinical reasoning as well. At first, it (the expert) explained to us. For example, are there possible lymph nodes? Yes or no. Then you need to do this and this…Then you can make kind of…pattern that differs for the diagnosis and the prognosis. So you can make kind of a diagram in your head. Which you can use later on. And your knowledge becomes more structured. (Focus Group 2, Student B)

Students also perceived that the integrated practice with Virtual Patients helped them to anticipate the subsequent steps in clinical reasoning. They indicated the patterns learned through practicing with virtual Patients helped them understand the procedures they needed to follow to evaluate the patient.

I think now I know the steps which they (the procedural) followed to evaluate the patient, so first we can do this and then that. First, you determine the TNM (Tumour, Node, Metastasis) staging and do the endoscopy, then the TNM staging, and then you make the treatment plan. Now it’s more clear how they do those steps. (Focus Group 1, Student A)

Moreover, students thought the pair work and dialogue helped them think and clarify with each other what steps they needed to do in clinical reasoning when they had different opinions.

Yeah, that (pair working) was really nice because you can discuss, like I think do this and the other one says, you know, I think do that step, and then you’re already discussing the answers which is really nice to have. (The discussion) really make you think about the steps. (Focus Group 1, Student b)

Challenging their reasoning to enhance deeper understanding

Students reported how the course design differed from other blocks. According to the students, the VP practice was particularly beneficial in helping them integrate knowledge, and make the knowledge their own.

It (the VP practice) helps you to integrate knowledge because other blocks are really only lectures, they are all listening and listening. So the virtual patient was really nice to make this stuff our own. (Focus Group 2, Student A)

Students indicated the examples given by the expert helped them get a better understanding of the more detailed TNM (Tumor, Node, Metastasis) table, that are used in clinical reasoning.

Yeah, she (the expert) gave examples and guided the reading of the tables for TNM (Tumor, Node, Metastasis) staging, and those were also in the Virtual Patient cases, but because she already used them once and explained how we have to use them, it became more clear to us, what these tables are for and how they are used (Focus Group 1, Student B) .

The students noted that in VP practice sessions, compared with passive learning in traditional lectures, they were challenged to engage directly with the material by making clinical decisions, such as selecting appropriate tests to reach a diagnosis.

In lectures, we passively learn the trajectory from symptoms to diagnosis. During Virtual Patient practice, we actively process it. So you have to make decisions and select the test etc. (Focus Group 2, Student B)

Students indicated that practicing with the VP cases challenged them to look up information and reasoned by themselves. They gave an example of the imaging practice in which they were tasked with examining specific body parts in medical images on their own, they thought they were challenged to reason about what they saw instead of getting the information directly.

Yeah, also the (medical) imaging in the assignments where you need to look at a specific part of the body, normally you just see a picture and someone says, yeah, this is the stomach or this is the heart, whatever, and now you need to look it up yourself and think about it yourself, what you see, so that really helps. (Focus Group 1, Student B)

Furthermore, they emphasized the questions asked by experts challenged them to think, put the knowledge in their own words and apply the knowledge with their own reasoning.

The questions she (the expert) asked really make you think about the things she’s learning(teaching). So if she asks questions, you’re really thinking, and yeah, you’re challenged to put it in your own words. (Focus Group 1, Student B) For instance, she (the expert) asked questions that not from official guidelines, instead, it came from where widely doctor worked and her personal experiences. I applied what she said with my own reasoning behind it. (Focus Group 2, Student B)

Transfer of knowledge

Students perceived that practicing with VP cases in different situations offered them hands-on experience, where they actively engaged with various situations, which prepared them for future patient interactions.

Having cases that are closer to the real world, like the comorbidity we discussed, would make it more realistic. (For instance, ) What if he also has obesity or diabetes? Those are the patients that we are going to see in the future. So it helps out a lot to have those different conditions as well. (Focus Group 2, Student B)

Students also indicated their preference for the structured approach of the VP session, where an initial demonstration by an expert, sharing their clinical experience, followed by hands-on practice with VP cases was perceived to enhance transfer to practice. This method, as described by the student, bridged the gap between theoretical knowledge and practical application. They think this structure made the knowledge clear and further helped them to transfer their knowledge from theory to practice.

You (the Virtual Patient session that integrated with role modeling, authentic VP practice, and peer discussion around feedback) made it (the clinical reasoning) clear for me because of the first case we discussed with the teacher. Well, he discussed it and showed us how to think, and how to get things from certain perspectives with risk factors, age, et cetera. And then we do it ourselves. We had to find out what was wrong and go on. So I quite liked it. It gave me a deeper understanding. (Focus Group 3, Student A)

Students indicated the sense of practical immersion is amplified by the “side information that you don’t really need” (Focus Group 3, Student E) from the cases. They highlighted the side information represented the interaction with real patients and made them think of clinical situations in real-world settings.

(Side) information would be more realistic, also side information that you don’t really need because a patient also tells you a lot of things, and some of those things aren’t as important, but you still need to decide if they are important or not. What do you see, why do you see it, what’s different than normal. (Focus Group 3, Student E)

Moreover, several students indicated that the hypothetical “what-if” discussions during the role modeling session helped them with reasoning, prompting them to consider complications that might arise in real-life medical situations.

So for example, about age, it’s more difficult to do a treatment above 70. (What if that patient) has things like smoking history and that kind of stuff. I think it’s really valuable because you have already had an example about it (Demonstrating Case A). (Focus Group 1, Student A)

Students indicated that the diagnosis practice in VP led them to realize the difference in real-world scenarios. They said while in the simulated environment might seem easy to choose multiple diagnostic options, in the real world, medical professionals must make more selective decisions due to limitations. They think this experience taught them to think of prioritizing and decision-making in a realistic medical setting.

Yeah, maybe also there (in VP cases) were also a question about which imaging techniques you would use and then it was Echo or CT, MRI, there was also an option where you could listen to the lungs and some of the people also checked that one, but it isn’t really necessary, so you think it only takes one minute, so why not, but in the real world there isn’t always time to do everything, so it’s also good to think what is really necessary and what’s not. (Focus Group 1, Student A)

Enhance reasoning through reflections

During the VP session, students received feedback and conducted conversations around the feedback provided by the Virtual Patient system. Students thought the peer dialogues around feedback provided opportunities for collective reflection and insights, allowing them to pinpoint areas of improvement.

I thought that (the peer dialogue) was really useful, because sometimes one person, for example, when the teacher explains everything, you don’t pick up everything he says. She (your peer) might pick up a different thing, and I pick up a different thing, and we can ask each other, do you know how this works? So I thought that was really useful. (Focus Group 3, Student B)

The students emphasized the importance of expressing and discussing different opinions. They noted that such interactions could provide new insights and perspectives that they would not have considered independently, thereby enriching their understanding.

When you do have different opinions, I think they (your peers) can give you insight that you maybe didn’t have for yourself. So you can add to each other’s knowledge. If somebody has another view, then we can discuss it. It (the discussion) brightens my tunnel view. Also having to say it (the knowledge) out loud and explaining your thoughts to someone else can also help, I think. (Focus Group 2, Student A)

When talking about the peer dialogues around feedback during the VP session, Some students highlighted the benefits of immediate feedback, which provided them with clarity and instant validation. However, others saw value in delayed feedback, as it fostered discussion and multiple interpretations.

I liked that the Virtual Patient program, that it gave you immediate feedback. That was really handy. And I also liked the discussion afterward so we could speak about it a bit more (Focus Group 3, Student B) . There was immediate feedback on most questions, so you knew if you had been correct or wrong. But for the learning process it might be handy to have that after the group discussion, because now we all have the same answer. (Focus Group 2, Student B)

The study demonstrated the perception of students’ learning and knowledge transfer by integrating VP cases with role modeling introductions, and peer dialogue around feedback, specifically in the context of personalized medicine in cancer treatment and care. The survey reflected a positive learning experience and students reported they gained a better understanding of the clinical reasoning process as well as which steps to take when dealing with a clinical case through this specific course design with integration of VP cases. Qualitative data showed that the integration of VPs into the educational setting clearly shifted the students from being passive observers in a traditional lecture-based format to active participants in a simulated clinical environment. This shift is in line with previous research findings, which suggest that the use of VPs in clinical training actively engages learners and encourages the application of their knowledge [ 4 ].

The quantitative data revealed that students highly valued the role modeling session, as indicated by the high average scores. Qualitative data explained that the role modeling session enabled students to not only observe the clinical process being demonstrated but also to engage in active thinking by interacting with the expert. As discussed by Cruess, Cruess [ 15 ], role modeling not only consciously imparts knowledge but also unconsciously influences students’ attitudes and behaviors, making the learning experience more relatable to the clinical environment. In this study, by sharing clinical reasoning and personal anecdotes during the class, experts made the learning experience more relatable to the clinical environment that students would face in the future. This mirrored the role modeling research by Morgenroth, Ryan [ 25 ] which emphasizes the importance of role models in shaping the self-concept and motivation of individuals. Moreover, the qualitative data showed that the demonstration by the expert serves as a fundamental pre-knowledge for students to cover the knowledge gap and prepare them with the following practice. This finding aligns with van Merrienboer’s scaffolding concept emphasizing the importance of initial expert guidance in learning processes [ 16 ].

Followed by the role modeling demonstration, students practiced on two VP cases in pairs and perceived that the VP practice enhanced their clinical reasoning skills, and also helped them understand the real-world clinical setting. The result showed that the variety and real-life complexity of cases in the VP sessions were perceived to be essential for students’ knowledge gain and transfer. The positive perception of various authentic cases aligns with previous research highlighting the importance of exposure to diverse and authentic scenarios in medical training [ 17 , 18 ]. Moreover, the hypothetical “what-if” scenarios further enhanced students’ analytical abilities, preparing them for the multifaceted challenges they would encounter in real-world medical situations. Survey responses (Q10, mean = 4.37; Q13, mean = 4.05 in Table  1 ) indicated a consensus among students on the improvement with this practice in understanding and applying knowledge. Our findings corroborate with Jonassen and Hernandez-Serrano [ 26 ]’s study emphasis on the importance of authentic learning environments for effective knowledge transfer.

After the practice, students discussed the feedback provided by the VP system. Despite its mixed quantitative reception, the peer dialogue on feedback was qualitatively found to be a vital component for promoting critical thinking, discussion, and reflection. The Feedback from the VPs, both immediate and delayed, along with peer dialogue, emerged as crucial elements in students’ learning process. In this study, students showed different preferences for receiving feedback. Some students preferred immediate feedback, however, others preferred delayed feedback. How feedback was provided notably influenced peer interactions. Given that immediate feedback was dispensed upon submission of answers, the peer dialogues automatically started when students noticed disparities or encountered obstacles. Such dialogues not only served to resolve ambiguities but also fostered collective reflection, enhancing comprehension of the subject. By vocalizing their thoughts and engaging in active discussions, students were able to solidify their understanding and uncover nuances they might have missed otherwise. This aligns with the importance of engaging in peer discussions on feedback as outlined in the theoretical background [ 20 , 21 , 22 ].

When looking at the integration of VP cases with the particular course design, students perceived that the expert demonstration, followed by VP practice, and peer dialogue around feedback fostered a comprehensive understanding, allowing them to integrate diverse clinical knowledge, which in turn promoted understanding. The “Watch-think-do-reflect” structure not only ensured better knowledge retention but also enhanced students’ enthusiasm towards the subject. Observing model demonstrations enabled students to assimilate clinical nuances and contemplate real-world applications. Subsequent hands-on practice with VP cases fortified their cognitive structures, honing their clinical reasoning. Ultimately, students perceived that reflective peer discussions on feedback solidified their learnings, enhancing knowledge retention.

Limitations

This study employed a survey and focus group interviews that provided a comprehensive understanding of students’ perceptions of learning. However, there are several limitations. The study had a small sample size and was conducted in the context of an elective course, which may limit the generalizability of the findings. Furthermore, the study was exploratory in nature and did not measure actual learning outcomes or long-term retention, which are critical aspects of educational impact.

Implications for future research

Future research should investigate whether integrating Virtual Patients (VPs) into classroom activities enhance student learning outcomes by incorporating learning assessments and involving larger and more diverse participant groups to validate our findings. Additionally, a deeper analysis of students’ reasoning processes and interactions could provide insights into how and why knowledge gain and transfer are fostered or hindered. Furthermore, it is also important to understand the most beneficial moment for integrating VPs into educational settings to enhance transfer from a simulated to a real practice setting. This understanding could inform the development of more effective educational strategies and interventions.

The integration of Virtual Patients into classroom learning appears to offer a promising approach to enrich medical education. Key elements such as role modeling and various authentic cases contribute positively to students’ perception of learning, as well as peer dialogue on feedback. However, the approach to peer dialogue on feedback may need to be refined for more consistent benefits. Furthermore, studies with larger sample sizes and broader participant groups are essential to provide robust support for the efficacy of this educational approach and its components.

Data availability

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Cook DA, Triola MM. Virtual patients: a critical literature review and proposed next steps. Med Educ. 2009;43(4):303–11.

Article   Google Scholar  

Garrett BM, Callear D. The value of intelligent multimedia simulation for teaching clinical decision-making skills. Nurse Educ Today. 2001;21(5):382–90.

Peddle M, Bearman M, Nestel D. Virtual patients and nontechnical skills in undergraduate health professional education: an integrative review. Clin Simul Nurs. 2016;12(9):400–10.

Sanders CL, Kleinert HL, Free T, Slusher I, Clevenger K, Johnson S, et al. Caring for children with intellectual and developmental disabilities: virtual patient instruction improves students’ knowledge and comfort level. J Pediatr Nurs. 2007;22(6):457–66.

Sijstermans R, Jaspers MWM, Bloemendaal PM, Schoonderwaldt EM. Training inter-physician communication using the dynamic patient Simulator®. Int J Med Informatics. 2007;76(5):336–43.

Buttussi F, Chittaro L. Effects of different types of virtual reality display on presence and learning in a safety training scenario. IEEE Trans Vis Comput Graph. 2017;24(2):1063–76.

Makransky G, Bonde MT, Wulff JS, Wandall J, Hood M, Creed PA, et al. Simulation based virtual learning environment in medical genetics counseling: an example of bridging the gap between theory and practice in medical education. BMC Med Educ. 2016;16(1):1–9.

Verkuyl M, Hughes M, Tsui J, Betts L, St-Amant O, Lapum JL. Virtual gaming simulation in nursing education: a focus group study. J Nurs Educ. 2017;56(5):274–80.

Burke LA, Hutchins HM. Training transfer: an integrative literature review. Hum Resour Dev Rev. 2007;6(3):263–96.

Durning SJ, Artino AR Jr, Schuwirth L, Van Der Vleuten C. Clarifying assumptions to enhance our understanding and assessment of clinical reasoning. Acad Med. 2013;88(4):442–8.

Marei HF, Donkers J, Al-Eraky MM, van Merrienboer JJ. The effectiveness of sequencing virtual patients with lectures in a deductive or inductive learning approach. Med Teach. 2017;39(12):1268–74.

Marei HF, Donkers J, Al-Eraky MM, Van Merrienboer JJ. Collaborative use of virtual patients after a lecture enhances learning with minimal investment of cognitive load. Med Teach. 2019;41(3):332–9.

Tolsgaard MG, Jepsen RM, Rasmussen MB, Kayser L, Fors U, Laursen LC, et al. The effect of constructing versus solving virtual patient cases on transfer of learning: a randomized trial. Perspect Med Educ. 2016;5:33–8.

Burgess A, Oates K, Goulston K. Role modelling in medical education: the importance of teaching skills. Clin Teach. 2016;13(2):134–7.

Cruess SR, Cruess RL, Steinert Y. Role modelling—making the most of a powerful teaching strategy. BMJ. 2008;336(7646):718–21.

Van Merriënboer JJ, Kirschner PA. Ten steps to complex learning: a systematic approach to four-component instructional design: Routledge; 2017.

Google Scholar  

Berman NB, Durning SJ, Fischer MR, Huwendiek S, Triola MM. The role for virtual patients in the future of medical education. Acad Med. 2016;91(9):1217–22.

Lowell VL, Yang M. Authentic learning experiences to improve online instructor’s performance and self-efficacy: the design of an online mentoring program. TechTrends. 2023;67(1):112–23.

Sevy-Biloon J, Chroman T. Authentic use of technology to improve EFL communication and motivation through international language exchange video chat. Teach Engl Technol. 2019;19(2):44–58.

Dmoshinskaia N, Gijlers H, de Jong T. Giving feedback on peers’ concept maps as a learning experience: does quality of reviewed concept maps matter? Learn Environ Res. 2022;25(3):823–40.

Foster A, Chaudhary N, Kim T, Waller JL, Wong J, Borish M, et al. Using virtual patients to teach empathy: a randomized controlled study to enhance medical students’ empathic communication. Simul Healthc. 2016;11(3):181–9.

Schillings M, Roebertsen H, Savelberg H, Whittingham J, Dolmans D. Peer-to-peer dialogue about teachers’ written feedback enhances students’ understanding on how to improve writing skills. Educational Stud. 2020;46(6):693–707.

Stalmeijer RE, Dolmans DH, Snellen-Balendong HA, van Santen-Hoeufft M, Wolfhagen IH, Scherpbier AJ. Clinical teaching based on principles of cognitive apprenticeship: views of experienced clinical teachers. Acad Med. 2013;88(6):861–5.

Braun V, Clarke V. Using thematic analysis in psychology. Qualitative Res Psychol. 2006;3(2):77–101.

Morgenroth T, Ryan MK, Peters K. The motivational theory of role modeling: how role models influence role aspirants’ goals. Rev Gen Psychol. 2015;19(4):465–83.

Jonassen DH, Hernandez-Serrano J. Case-based reasoning and instructional design: using stories to support problem solving. Education Tech Research Dev. 2002;50(2):65–77.

Download references

Acknowledgements

Thanks to all the participants and education workers who contributed to the study. ZL was supported by a scholarship granted by the China Scholarship Council. Thanks for the support of my family, and thanks Ang Li for joining our family.

ZL was supported by a scholarship granted by the China Scholarship Council (CSC, 202208440100).

Author information

Authors and affiliations.

Department of Educational Development & Research, School of Health Professions Education, Faculty of Health, Medicine and Life Sciences, Maastricht University, Maastricht, Netherlands

Zhien Li, Maryam Asoodar, Xian Liu & Diana Dolmans

School of Health Professions Education, Department of Health Services Research, Faculty of Health, Medicine and Life Sciences, Maastricht University, Maastricht, Netherlands

Nynke de Jong

Department of Radiation Oncology (Maastro), GROW School for Oncology and Reproduction, Maastricht University Medical Centre, Maastricht, The Netherlands

Tom Keulers

You can also search for this author in PubMed   Google Scholar

Contributions

ZL, MA, DD, and NJ conceived of the presented idea. MA and DD verified the analytical methods. TK and ZL contribute to the creation of learning materials. ZL analyzed the data and drafted the manuscript under the supervision of MA and DD. All authors contributed to the article and approved the submitted version.

Corresponding author

Correspondence to Zhien Li .

Ethics declarations

Ethics approval and consent to participate.

The Maastricht University Ethical Committee reviewed and approved this study. The approval number is FHML-REC/2023/021. All participants were informed about the aims, methods, their right to withdraw, and anticipated benefits of the study. Written informed consent was obtained from all participants prior to their inclusion in the study.

Consent for publication

Not applicable.

Competing interests

The authors declare no competing interests.

Additional information

Publisher’s note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary Material 1

Supplementary material 2, supplementary material 3, supplementary material 4, rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ . The Creative Commons Public Domain Dedication waiver ( http://creativecommons.org/publicdomain/zero/1.0/ ) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Cite this article.

Li, Z., Asoodar, M., de Jong, N. et al. Perception of enhanced learning in medicine through integrating of virtual patients: an exploratory study on knowledge acquisition and transfer. BMC Med Educ 24 , 647 (2024). https://doi.org/10.1186/s12909-024-05624-7

Download citation

Received : 18 March 2024

Accepted : 03 June 2024

Published : 11 June 2024

DOI : https://doi.org/10.1186/s12909-024-05624-7

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Virtual patient
• Medical education
• Role modeling
• Authentic cases

BMC Medical Education

ISSN: 1472-6920

• Submission enquiries: [email protected]
• General enquiries: [email protected]

Information

• Author Services

Initiatives

You are accessing a machine-readable page. In order to be human-readable, please install an RSS reader.

All articles published by MDPI are made immediately available worldwide under an open access license. No special permission is required to reuse all or part of the article published by MDPI, including figures and tables. For articles published under an open access Creative Common CC BY license, any part of the article may be reused without permission provided that the original article is clearly cited. For more information, please refer to https://www.mdpi.com/openaccess .

Feature papers represent the most advanced research with significant potential for high impact in the field. A Feature Paper should be a substantial original Article that involves several techniques or approaches, provides an outlook for future research directions and describes possible research applications.

Feature papers are submitted upon individual invitation or recommendation by the scientific editors and must receive positive feedback from the reviewers.

Editor’s Choice articles are based on recommendations by the scientific editors of MDPI journals from around the world. Editors select a small number of articles recently published in the journal that they believe will be particularly interesting to readers, or important in the respective research area. The aim is to provide a snapshot of some of the most exciting work published in the various research areas of the journal.

Original Submission Date Received: .

• Active Journals
• Find a Journal
• Proceedings Series
• For Authors
• For Reviewers
• For Editors
• For Librarians
• For Publishers
• For Societies
• For Conference Organizers
• Open Access Policy
• Institutional Open Access Program
• Special Issues Guidelines
• Editorial Process
• Research and Publication Ethics
• Article Processing Charges
• Testimonials
• Preprints.org
• SciProfiles
• Encyclopedia

Article Menu

• Subscribe SciFeed
• Recommended Articles
• Google Scholar
• on Google Scholar
• Table of Contents

Find support for a specific problem in the support section of our website.

Please let us know what you think of our products and services.

Visit our dedicated information section to learn more about MDPI.

JSmol Viewer

Analysis of spatial and temporal pattern evolution and decoupling relationships of land use functions based on ecological protection and high-quality development: a case study of the yellow river basin, china, share and cite.

Du, H.; Wang, Z.; Li, H.; Zhang, C. Analysis of Spatial and Temporal Pattern Evolution and Decoupling Relationships of Land Use Functions Based on Ecological Protection and High-Quality Development: A Case Study of the Yellow River Basin, China. Land 2024 , 13 , 862. https://doi.org/10.3390/land13060862

Du H, Wang Z, Li H, Zhang C. Analysis of Spatial and Temporal Pattern Evolution and Decoupling Relationships of Land Use Functions Based on Ecological Protection and High-Quality Development: A Case Study of the Yellow River Basin, China. Land . 2024; 13(6):862. https://doi.org/10.3390/land13060862

Du, Hanwen, Zhanqi Wang, Haiyang Li, and Chen Zhang. 2024. "Analysis of Spatial and Temporal Pattern Evolution and Decoupling Relationships of Land Use Functions Based on Ecological Protection and High-Quality Development: A Case Study of the Yellow River Basin, China" Land 13, no. 6: 862. https://doi.org/10.3390/land13060862

Article Metrics

Article access statistics, further information, mdpi initiatives, follow mdpi.

Subscribe to receive issue release notifications and newsletters from MDPI journals

The state of AI in early 2024: Gen AI adoption spikes and starts to generate value

If 2023 was the year the world discovered generative AI (gen AI) , 2024 is the year organizations truly began using—and deriving business value from—this new technology. In the latest McKinsey Global Survey  on AI, 65 percent of respondents report that their organizations are regularly using gen AI, nearly double the percentage from our previous survey just ten months ago. Respondents’ expectations for gen AI’s impact remain as high as they were last year , with three-quarters predicting that gen AI will lead to significant or disruptive change in their industries in the years ahead.

About the authors

This article is a collaborative effort by Alex Singla , Alexander Sukharevsky , Lareina Yee , and Michael Chui , with Bryce Hall , representing views from QuantumBlack, AI by McKinsey, and McKinsey Digital.

Organizations are already seeing material benefits from gen AI use, reporting both cost decreases and revenue jumps in the business units deploying the technology. The survey also provides insights into the kinds of risks presented by gen AI—most notably, inaccuracy—as well as the emerging practices of top performers to mitigate those challenges and capture value.

AI adoption surges

Interest in generative AI has also brightened the spotlight on a broader set of AI capabilities. For the past six years, AI adoption by respondents’ organizations has hovered at about 50 percent. This year, the survey finds that adoption has jumped to 72 percent (Exhibit 1). And the interest is truly global in scope. Our 2023 survey found that AI adoption did not reach 66 percent in any region; however, this year more than two-thirds of respondents in nearly every region say their organizations are using AI. 1 Organizations based in Central and South America are the exception, with 58 percent of respondents working for organizations based in Central and South America reporting AI adoption. Looking by industry, the biggest increase in adoption can be found in professional services. 2 Includes respondents working for organizations focused on human resources, legal services, management consulting, market research, R&D, tax preparation, and training.

Also, responses suggest that companies are now using AI in more parts of the business. Half of respondents say their organizations have adopted AI in two or more business functions, up from less than a third of respondents in 2023 (Exhibit 2).

Gen AI adoption is most common in the functions where it can create the most value

Most respondents now report that their organizations—and they as individuals—are using gen AI. Sixty-five percent of respondents say their organizations are regularly using gen AI in at least one business function, up from one-third last year. The average organization using gen AI is doing so in two functions, most often in marketing and sales and in product and service development—two functions in which previous research  determined that gen AI adoption could generate the most value 3 “ The economic potential of generative AI: The next productivity frontier ,” McKinsey, June 14, 2023. —as well as in IT (Exhibit 3). The biggest increase from 2023 is found in marketing and sales, where reported adoption has more than doubled. Yet across functions, only two use cases, both within marketing and sales, are reported by 15 percent or more of respondents.

Gen AI also is weaving its way into respondents’ personal lives. Compared with 2023, respondents are much more likely to be using gen AI at work and even more likely to be using gen AI both at work and in their personal lives (Exhibit 4). The survey finds upticks in gen AI use across all regions, with the largest increases in Asia–Pacific and Greater China. Respondents at the highest seniority levels, meanwhile, show larger jumps in the use of gen Al tools for work and outside of work compared with their midlevel-management peers. Looking at specific industries, respondents working in energy and materials and in professional services report the largest increase in gen AI use.

Investments in gen AI and analytical AI are beginning to create value

The latest survey also shows how different industries are budgeting for gen AI. Responses suggest that, in many industries, organizations are about equally as likely to be investing more than 5 percent of their digital budgets in gen AI as they are in nongenerative, analytical-AI solutions (Exhibit 5). Yet in most industries, larger shares of respondents report that their organizations spend more than 20 percent on analytical AI than on gen AI. Looking ahead, most respondents—67 percent—expect their organizations to invest more in AI over the next three years.

Where are those investments paying off? For the first time, our latest survey explored the value created by gen AI use by business function. The function in which the largest share of respondents report seeing cost decreases is human resources. Respondents most commonly report meaningful revenue increases (of more than 5 percent) in supply chain and inventory management (Exhibit 6). For analytical AI, respondents most often report seeing cost benefits in service operations—in line with what we found last year —as well as meaningful revenue increases from AI use in marketing and sales.

Inaccuracy: The most recognized and experienced risk of gen AI use

As businesses begin to see the benefits of gen AI, they’re also recognizing the diverse risks associated with the technology. These can range from data management risks such as data privacy, bias, or intellectual property (IP) infringement to model management risks, which tend to focus on inaccurate output or lack of explainability. A third big risk category is security and incorrect use.

Respondents to the latest survey are more likely than they were last year to say their organizations consider inaccuracy and IP infringement to be relevant to their use of gen AI, and about half continue to view cybersecurity as a risk (Exhibit 7).

Conversely, respondents are less likely than they were last year to say their organizations consider workforce and labor displacement to be relevant risks and are not increasing efforts to mitigate them.

In fact, inaccuracy— which can affect use cases across the gen AI value chain , ranging from customer journeys and summarization to coding and creative content—is the only risk that respondents are significantly more likely than last year to say their organizations are actively working to mitigate.

Some organizations have already experienced negative consequences from the use of gen AI, with 44 percent of respondents saying their organizations have experienced at least one consequence (Exhibit 8). Respondents most often report inaccuracy as a risk that has affected their organizations, followed by cybersecurity and explainability.

Our previous research has found that there are several elements of governance that can help in scaling gen AI use responsibly, yet few respondents report having these risk-related practices in place. 4 “ Implementing generative AI with speed and safety ,” McKinsey Quarterly , March 13, 2024. For example, just 18 percent say their organizations have an enterprise-wide council or board with the authority to make decisions involving responsible AI governance, and only one-third say gen AI risk awareness and risk mitigation controls are required skill sets for technical talent.

Bringing gen AI capabilities to bear

The latest survey also sought to understand how, and how quickly, organizations are deploying these new gen AI tools. We have found three archetypes for implementing gen AI solutions : takers use off-the-shelf, publicly available solutions; shapers customize those tools with proprietary data and systems; and makers develop their own foundation models from scratch. 5 “ Technology’s generational moment with generative AI: A CIO and CTO guide ,” McKinsey, July 11, 2023. Across most industries, the survey results suggest that organizations are finding off-the-shelf offerings applicable to their business needs—though many are pursuing opportunities to customize models or even develop their own (Exhibit 9). About half of reported gen AI uses within respondents’ business functions are utilizing off-the-shelf, publicly available models or tools, with little or no customization. Respondents in energy and materials, technology, and media and telecommunications are more likely to report significant customization or tuning of publicly available models or developing their own proprietary models to address specific business needs.

Respondents most often report that their organizations required one to four months from the start of a project to put gen AI into production, though the time it takes varies by business function (Exhibit 10). It also depends upon the approach for acquiring those capabilities. Not surprisingly, reported uses of highly customized or proprietary models are 1.5 times more likely than off-the-shelf, publicly available models to take five months or more to implement.

Gen AI high performers are excelling despite facing challenges

Gen AI is a new technology, and organizations are still early in the journey of pursuing its opportunities and scaling it across functions. So it’s little surprise that only a small subset of respondents (46 out of 876) report that a meaningful share of their organizations’ EBIT can be attributed to their deployment of gen AI. Still, these gen AI leaders are worth examining closely. These, after all, are the early movers, who already attribute more than 10 percent of their organizations’ EBIT to their use of gen AI. Forty-two percent of these high performers say more than 20 percent of their EBIT is attributable to their use of nongenerative, analytical AI, and they span industries and regions—though most are at organizations with less than $1 billion in annual revenue. The AI-related practices at these organizations can offer guidance to those looking to create value from gen AI adoption at their own organizations.

To start, gen AI high performers are using gen AI in more business functions—an average of three functions, while others average two. They, like other organizations, are most likely to use gen AI in marketing and sales and product or service development, but they’re much more likely than others to use gen AI solutions in risk, legal, and compliance; in strategy and corporate finance; and in supply chain and inventory management. They’re more than three times as likely as others to be using gen AI in activities ranging from processing of accounting documents and risk assessment to R&D testing and pricing and promotions. While, overall, about half of reported gen AI applications within business functions are utilizing publicly available models or tools, gen AI high performers are less likely to use those off-the-shelf options than to either implement significantly customized versions of those tools or to develop their own proprietary foundation models.

What else are these high performers doing differently? For one thing, they are paying more attention to gen-AI-related risks. Perhaps because they are further along on their journeys, they are more likely than others to say their organizations have experienced every negative consequence from gen AI we asked about, from cybersecurity and personal privacy to explainability and IP infringement. Given that, they are more likely than others to report that their organizations consider those risks, as well as regulatory compliance, environmental impacts, and political stability, to be relevant to their gen AI use, and they say they take steps to mitigate more risks than others do.

Gen AI high performers are also much more likely to say their organizations follow a set of risk-related best practices (Exhibit 11). For example, they are nearly twice as likely as others to involve the legal function and embed risk reviews early on in the development of gen AI solutions—that is, to “ shift left .” They’re also much more likely than others to employ a wide range of other best practices, from strategy-related practices to those related to scaling.

In addition to experiencing the risks of gen AI adoption, high performers have encountered other challenges that can serve as warnings to others (Exhibit 12). Seventy percent say they have experienced difficulties with data, including defining processes for data governance, developing the ability to quickly integrate data into AI models, and an insufficient amount of training data, highlighting the essential role that data play in capturing value. High performers are also more likely than others to report experiencing challenges with their operating models, such as implementing agile ways of working and effective sprint performance management.

About the research

The online survey was in the field from February 22 to March 5, 2024, and garnered responses from 1,363 participants representing the full range of regions, industries, company sizes, functional specialties, and tenures. Of those respondents, 981 said their organizations had adopted AI in at least one business function, and 878 said their organizations were regularly using gen AI in at least one function. To adjust for differences in response rates, the data are weighted by the contribution of each respondent’s nation to global GDP.

Alex Singla and Alexander Sukharevsky  are global coleaders of QuantumBlack, AI by McKinsey, and senior partners in McKinsey’s Chicago and London offices, respectively; Lareina Yee  is a senior partner in the Bay Area office, where Michael Chui , a McKinsey Global Institute partner, is a partner; and Bryce Hall  is an associate partner in the Washington, DC, office.

They wish to thank Kaitlin Noe, Larry Kanter, Mallika Jhamb, and Shinjini Srivastava for their contributions to this work.

This article was edited by Heather Hanselman, a senior editor in McKinsey’s Atlanta office.

Explore a career with us

Related articles.

Moving past gen AI’s honeymoon phase: Seven hard truths for CIOs to get from pilot to scale

A generative AI reset: Rewiring to turn potential into value in 2024

Implementing generative AI with speed and safety