critical thinking skills journal articles

Information

• Author Services

Initiatives

You are accessing a machine-readable page. In order to be human-readable, please install an RSS reader.

All articles published by MDPI are made immediately available worldwide under an open access license. No special permission is required to reuse all or part of the article published by MDPI, including figures and tables. For articles published under an open access Creative Common CC BY license, any part of the article may be reused without permission provided that the original article is clearly cited. For more information, please refer to https://www.mdpi.com/openaccess .

Feature papers represent the most advanced research with significant potential for high impact in the field. A Feature Paper should be a substantial original Article that involves several techniques or approaches, provides an outlook for future research directions and describes possible research applications.

Feature papers are submitted upon individual invitation or recommendation by the scientific editors and must receive positive feedback from the reviewers.

Editor’s Choice articles are based on recommendations by the scientific editors of MDPI journals from around the world. Editors select a small number of articles recently published in the journal that they believe will be particularly interesting to readers, or important in the respective research area. The aim is to provide a snapshot of some of the most exciting work published in the various research areas of the journal.

Original Submission Date Received: .

• Active Journals
• Find a Journal
• Proceedings Series
• For Authors
• For Reviewers
• For Editors
• For Librarians
• For Publishers
• For Societies
• For Conference Organizers
• Open Access Policy
• Institutional Open Access Program
• Special Issues Guidelines
• Editorial Process
• Research and Publication Ethics
• Article Processing Charges
• Testimonials
• Preprints.org
• SciProfiles
• Encyclopedia

Article Menu

• Subscribe SciFeed
• Recommended Articles
• PubMed/Medline
• Google Scholar
• on Google Scholar
• Table of Contents

Find support for a specific problem in the support section of our website.

Please let us know what you think of our products and services.

Visit our dedicated information section to learn more about MDPI.

JSmol Viewer

Predicting everyday critical thinking: a review of critical thinking assessments.

1. Introduction

2. how critical thinking impacts everyday life, 3. critical thinking: skills and dispositions.

“the use of those cognitive skills and abilities that increase the probability of a desirable outcome. It is used to describe thinking that is purposeful, reasoned, and goal directed—the kind of thinking involved in solving problems, formulating inferences, calculating likelihoods, and making decisions” ( Halpern 2014, p. 8 ).

4. Measuring Critical Thinking

4.1. practical challenges, 4.2. critical thinking assessments, 4.2.1. california critical thinking dispositions inventory (cctdi; insight assessment, inc. n.d. ), 4.2.2. california critical thinking skills test (cctst; insight assessment, inc. n.d. ), 4.2.3. cornell critical thinking test (cctt; the critical thinking company n.d. ), 4.2.4. california measure of mental motivation (cm3; insight assessment, inc. n.d. ), 4.2.5. ennis–weir critical thinking essay test ( ennis and weir 2005 ), 4.2.6. halpern critical thinking assessment (hcta; halpern 2012 ), 4.2.7. test of everyday reasoning (ter; insight assessment, inc. n.d. ), 4.2.8. watson–glaser tm ii critical thinking appraisal (w-gii; ncs pearson, inc. 2009 ).

“Virtual employees, or employees who work from home via a computer, are an increasing trend. In the US, the number of virtual employees has increased by 39% in the last two years and 74% in the last five years. Employing virtual workers reduces costs and makes it possible to use talented workers no matter where they are located globally. Yet, running a workplace with virtual employees might entail miscommunication and less camaraderie and can be more time-consuming than face-to-face interaction”.

5. Conclusions

Institutional review board statement, informed consent statement, data availability statement, conflicts of interest.

• Ali, Marium, and AJLabs. 2023. How Many Years Does a Typical User Spend on Social Media? Doha: Al Jazeera. Available online: https://www.aljazeera.com/news/2023/6/30/how-many-years-does-a-typical-user-spend-on-social-media (accessed on 13 November 2023).
• Arendasy, Martin, Lutz Hornke, Markus Sommer, Michaela Wagner-Menghin, Georg Gittler, Joachim Häusler, Bettina Bognar, and M. Wenzl. 2012. Intelligenz-Struktur-Batterie (Intelligence Structure Battery; INSBAT) . Mödling: Schuhfried GmbH. [ Google Scholar ]
• Arum, Richard, and Josipa Roksa. 2010. Academically Adrift . Chicago: The University of Chicago Press. [ Google Scholar ]
• Bakshy, Eytan, Solomon Messing, and Lada Adamic. 2015. Exposure to ideologically diverse news and opinion on Facebook. Science 348: 1130–32. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Bart, William. 2010. The Measurement and Teaching of Critical Thinking Skills . Tokyo: Invited colloquium given at the Center for Research on Education Testing. [ Google Scholar ]
• Bruine de Bruin, Wandi, Andrew Parker, and Baruch Fischhoff. 2007. Individual differences in adult decision-making competence. Journal of Personality and Social Psychology 92: 938–56. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Butler, Heather. 2012. Halpern Critical Thinking Assessment predicts real-world outcomes of critical thinking. Applied Cognitive Psychology 26: 721–29. [ Google Scholar ] [ CrossRef ]
• Butler, Heather, and Diane Halpern. 2020. Critical Thinking Impacts Our Everyday Lives. In Critical Thinking in Psychology , 2nd ed. Edited by Robert Sternberg and Diane Halpern. Cambridge, UK: Cambridge University Press. [ Google Scholar ] [ CrossRef ]
• Butler, Heather, Chris Dwyer, Michael Hogan, Amanda Franco, Silvia Rivas, Carlos Saiz, and Leandro Almeida. 2012. Halpern Critical Thinking Assessment and real-world outcomes: Cross-national applications. Thinking Skills and Creativity 7: 112–21. [ Google Scholar ] [ CrossRef ]
• Butler, Heather, Chris Pentoney, and Mabelle Bong. 2017. Critical thinking ability is a better predictor of life decisions than intelligence. Thinking Skills and Creativity 24: 38–46. [ Google Scholar ] [ CrossRef ]
• Ennis, Robert. 2005. The Ennis-Weir Critical Thinking Essay Test . Urbana: The Illinois Critical Thinking Project. Available online: http://faculty.ed.uiuc.edu/rhennis/supplewmanual1105.htm (accessed on 22 October 2023).
• Ennis, Robert, and Eric Weir. 2005. Ennis-Weir Critical Thinking Essay Test . Seaside: The Critical Thinking Company. Available online: https://www.academia.edu/1847582/The_Ennis_Weir_Critical_Thinking_Essay_Test_An_Instrument_for_Teaching_and_Testing (accessed on 22 October 2023).
• Facione, Peter. 1990. California Critical Thinking Dispositions Inventory . Millbrae: The California Academic Press. [ Google Scholar ]
• Facione, Peter, Noreen Facione, and Kathryn Winterhalter. 2012. The Test of Everyday Reasoning—(TER): Test Manual . Millbrae: California Academic Press. [ Google Scholar ]
• Forsyth, Carol, Philip Pavlik, Arthur C. Graesser, Zhiqiang Cai, Mae-lynn Germany, Keith Millis, Robert P. Dolan, Heather Butler, and Diane Halpern. 2012. Learning gains for core concepts in a serious game on scientific reasoning. In Proceedings of the 5th International Conference on Educational Data Mining . Edited by Kalina Yacef, Osmar Zaïane, Arnon Hershkovitz, Michael Yudelson and John Stamper. Chania: International Educational Data Mining Society, pp. 172–75. [ Google Scholar ]
• French, Brian, Brian Hand, William Therrien, and Juan Valdivia Vazquez. 2012. Detection of sex differential item functioning in the Cornell Critical Thinking Test. European Journal of Psychological Assessment 28: 201–7. [ Google Scholar ] [ CrossRef ]
• Frenkel, Sheera, and Mike Isaac. 2018. Facebook ‘Better Prepared’ to Fight Election Interference, Mark Zuckerberg Says . Manhattan: New York Times. Available online: https://www.nytimes.com/2018/09/13/technology/facebook-elections-mark-zuckerberg.html (accessed on 22 October 2023).
• Gheorghia, Olimpiu. 2018. Romania’s Measles Outbreak Kills Dozens of Children: Some Doctors Complain They Don’t Have Sufficient Stock of Vaccines . New York: Associated Press. Available online: https://www.nbcnews.com/health/health-news/romania-s-measles-outbreak-kills-dozens-children-n882771 (accessed on 13 November 2023).
• Giancarlo, Carol, Stephen Bloom, and Tim Urdan. 2004. Assessing secondary students’ disposition toward critical thinking: Development of the California Measure of Mental Motivation. Educational and Psychological Measurement 64: 347–64. [ Google Scholar ] [ CrossRef ]
• Halpern, Diane. 1998. Teaching critical thinking for transfer across domains: Dispositions, skills, structure training, and metacognitive monitoring. American Psychologist 53: 449–55. [ Google Scholar ] [ CrossRef ]
• Halpern, Diane. 2012. Halpern Critical Thinking Assessment . Mödling: Schuhfried (Vienna Test System). Available online: http://www.schuhfried.com/vienna-test-system-vts/all-tests-from-a-z/test/hcta-halpern-critical-thinking-assessment-1/ (accessed on 13 January 2013).
• Halpern, Diane. 2014. Thought and Knowledge: An Introduction to Critical Thinking , 5th ed. New York: Routledge Publishers. [ Google Scholar ]
• Halpern, Diane, Keith Millis, Arthur Graesser, Heather Butler, Carol Forsyth, and Zhiqiang Cai. 2012. Operation ARIES!: A computerized learning game that teaches critical thinking and scientific reasoning. Thinking Skills and Creativity 7: 93–100. [ Google Scholar ] [ CrossRef ]
• Huber, Christopher, and Nathan Kuncel. 2015. Does college teach critical thinking? A meta-analysis. Review of Educational Research 86: 431–68. [ Google Scholar ] [ CrossRef ]
• Insight Assessment, Inc. n.d. Critical Thinking Attribute Tests: Manuals and Assessment Information . Hermosa Beach: Insight Assessment. Available online: http://www.insightassessment.com (accessed on 22 October 2023).
• Jain, Anjali, Jaclyn Marshall, Ami Buikema, Tim Bancroft, Jonathan Kelly, and Craig Newschaffer. 2015. Autism occurrence by MMR vaccine status among US children with older siblings with and without autism. Journal of the American Medical Association 313: 1534–40. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Klee, Miles, and Nikki McCann Ramirez. 2023. AI Has Made the Israel-Hamas Misinformation Epidemic Much, Much Worse . New York: Rollingstone. Available online: https://www.rollingstone.com/politics/politics-features/israel-hamas-misinformation-fueled-ai-images-1234863586/amp/?fbclid=PAAabKD4u1FRqCp-y9z3VRA4PZZdX52DTQEn8ruvHeGsBrNguD_F2EiMrs3A4_aem_AaxFU9ovwsrXAo39I00d-8NmcpRTVBCsUd_erAUwlAjw16x1shqeC6s22OCpSSx2H-w (accessed on 27 October 2023).
• Klepper, David. 2022. Poll: Most in US Say Misinformation Spurs Extremism, Hate . New York: Associated Press-NORC Center for Public Affairs Research. Available online: https://apnorc.org/poll-most-in-us-say-misinformation-spurs-extremism-hate/ (accessed on 27 October 2023).
• Landis, Richard, and William Michael. 1981. The factorial validity of three measures of critical thinking within the context of Guilford’s Structure-of-Intellect Model for a sample of ninth grade students. Educational and Psychological Measurement 41: 1147–66. [ Google Scholar ] [ CrossRef ]
• Liedke, Jacob, and Luxuan Wang. 2023. Social Media and News Fact Sheet . Washington, DC: Pew Research Center. Available online: https://www.pewresearch.org/journalism/fact-sheet/social-media-and-news-fact-sheet/ (accessed on 15 November 2023).
• Lilienfeld, Scott, Rachel Ammirati, and Kristin Landfield. 2009. Giving debiasing away: Can psychological research on correcting cognitive errors promote human welfare? Perspective on Psychological Science 4: 390–98. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• Michael, Joan, Roberta Devaney, and William Michael. 1980. The factorial validity of the Cornell Critical Thinking Test for a junior high school sample. Educational and Psychological Measurement 40: 437–50. [ Google Scholar ] [ CrossRef ]
• National Center for Health Statistics. 2015. Health, United States, 2015, with Special Feature on Racial and Ethnic Health Disparities ; Washington, DC: U.S. Government Printing Office.
• NCS Pearson, Inc. 2009. Watson-Glaser II Critical Thinking Appraisal: Technical Manual and User’s Guide . London: Pearson. Available online: http://www.talentlens.com/en/downloads/supportmaterials/WGII_Technical_Manual.pdf (accessed on 22 October 2023).
• Stanovich, Keith, and Richard West. 2008. On the failure of cognitive ability to predict myside and one-sided thinking biases. Thinking & Reasoning 14: 129–67. [ Google Scholar ] [ CrossRef ]
• The Critical Thinking Company. n.d. Critical Thinking Company. Available online: www.criticalthinking.com (accessed on 13 October 2023).
• Tsipursky, Gleb. 2018. (Dis)trust in Science: Can We Cure the Scourge of Misinformation? New York: Scientific American. Available online: https://blogs.scientificamerican.com/observations/dis-trust-in-science/ (accessed on 11 April 2022).
• Walsh, Catherina, Lisa Seldomridge, and Karen Badros. 2007. California Critical Thinking Disposition Inventory: Further factor analytic examination. Perceptual and Motor Skills 104: 141–51. [ Google Scholar ] [ CrossRef ] [ PubMed ]
• World Health Organization. 2018. Europe Observes a 4-Fold Increase in Measles Cases in 2017 Compared to Previous Year . Geneva: World Health Organization. Available online: http://www.euro.who.int/en/media-centre/sections/press-releases/2018/europe-observes-a-4-fold-increase-in-measles-cases-in-2017-compared-to-previous-year (accessed on 22 October 2023).

Share and Cite

Butler, H.A. Predicting Everyday Critical Thinking: A Review of Critical Thinking Assessments. J. Intell. 2024 , 12 , 16. https://doi.org/10.3390/jintelligence12020016

Butler HA. Predicting Everyday Critical Thinking: A Review of Critical Thinking Assessments. Journal of Intelligence . 2024; 12(2):16. https://doi.org/10.3390/jintelligence12020016

Butler, Heather A. 2024. "Predicting Everyday Critical Thinking: A Review of Critical Thinking Assessments" Journal of Intelligence 12, no. 2: 16. https://doi.org/10.3390/jintelligence12020016

Article Metrics

Article access statistics, further information, mdpi initiatives, follow mdpi.

Subscribe to receive issue release notifications and newsletters from MDPI journals

Enhancing students’ critical thinking skills: is comparing correct and erroneous examples beneficial?

• Original Research
• Open access
• Published: 26 September 2021
• Volume 49 , pages 747–777, ( 2021 )

Cite this article

You have full access to this open access article

• Lara M. van Peppen   ORCID: orcid.org/0000-0002-1219-8267 1   nAff2 ,
• Peter P. J. L. Verkoeijen 1 , 3 ,
• Anita E. G. Heijltjes 3 ,
• Eva M. Janssen 4 &
• Tamara van Gog 4  

13k Accesses

18 Citations

5 Altmetric

Explore all metrics

There is a need for effective methods to teach critical thinking (CT). One instructional method that seems promising is comparing correct and erroneous worked examples (i.e., contrasting examples). The aim of the present study, therefore, was to investigate the effect of contrasting examples on learning and transfer of CT-skills, focusing on avoiding biased reasoning. Students ( N  = 170) received instructions on CT and avoiding biases in reasoning tasks, followed by: (1) contrasting examples, (2) correct examples, (3) erroneous examples, or (4) practice problems. Performance was measured on a pretest, immediate posttest, 3-week delayed posttest, and 9-month delayed posttest. Our results revealed that participants’ reasoning task performance improved from pretest to immediate posttest, and even further after a delay (i.e., they learned to avoid biased reasoning). Surprisingly, there were no differences in learning gains or transfer performance between the four conditions. Our findings raise questions about the preconditions of contrasting examples effects. Moreover, how transfer of CT-skills can be fostered remains an important issue for future research.

Similar content being viewed by others

Unraveling the effects of critical thinking instructions, practice, and self-explanation on students’ reasoning performance

The effects of learning from correct and erroneous examples in individual and collaborative settings, deliberate erring improves far transfer of learning more than errorless elaboration and spotting and correcting others’ errors.

Avoid common mistakes on your manuscript.

Introduction

Every day, we reason and make many decisions based on previous experiences and existing knowledge. To do so we often rely on a number of heuristics (i.e., mental shortcuts) that ease reasoning processes (Tversky & Kahneman, 1974 ). Usually, these decisions are inconsequential but sometimes they can lead to biases (i.e., deviating from ideal normative standards derived from logic and probability theory) with severe consequences. To illustrate, a forensic expert who misjudges fingerprint evidence because it verifies his or her preexisting beliefs concerning the likelihood of the guilt of a defendant, displays the so-called confirmation bias, which can result in a misidentification and a wrongful conviction (e.g., the Madrid bomber case; Kassin et al., 2013 ). Biases occur when people rely on heuristic reasoning (i.e., Type 1 processing) when that is not appropriate, do not recognize the need for analytical or reflective reasoning (i.e., Type 2 processing), are not willing to switch to Type 2 processing or unable to sustain it, or miss the relevant mindware to come up with a better response (e.g., Evans, 2003 ; Stanovich, 2011 ). Our primary tool for reasoning and making better decisions, and thus to avoid biases in reasoning and decision making, is critical thinking (CT), which is generally characterized as “purposeful, self-regulatory judgment that results in interpretation, analysis, evaluation, and inference, as well as explanation of the evidential, conceptual, methodological, criteriological, or contextual considerations on which that judgment is based” (Facione, 1990 , p. 2).

Because CT is essential for successful functioning in one’s personal, educational, and professional life, fostering students’ CT has become a central aim of higher education (Davies, 2013 ; Halpern, 2014 ; Van Gelder, 2005 ). However, several large-scale longitudinal studies were quite pessimistic that this laudable aim would be realized merely by following a higher education degree program. These studies revealed that CT-skills of many higher education graduates are insufficiently developed (e.g., Arum & Roksa, 2011 ; Flores et al., 2012 ; Pascarella et al., 2011 ; although a more recent meta-analytic study reached the more positive conclusion that students’ do improve their CT-skills over college years: Huber & Kuncel, 2016 ). Hence, there is a growing body of literature on how to teach CT (e.g., Abrami et al., 2008 , 2014 ; Van Peppen et al., 2018 , 2021 ; Angeli & Valanides, 2009 ; Niu et al., 2013 ; Tiruneh et al., 2014 , 2016 ).

However, there are different views on the best way to teach CT; the most well-known debate being whether CT should be taught in a general or content-specific manner (Abrami et al., 2014 ; Davies, 2013 ; Ennis, 1989 ; Moore, 2004 ). This debate has faded away during the last years, since most researchers nowadays commonly agree that CT can be seen in terms of both general skills (e.g., sound argumentation, evaluating statistical information, and evaluating the credibility of sources) and specific skills or knowledge used in the context of disciplines (e.g., diagnostic reasoning). Indeed, it has been shown that the most effective teaching methods combine generic instruction on CT with the opportunity to integrate the general principles that were taught with domain-specific subject matter. It is well established, for instance, that explicit teaching of CT combined with practice improves learning of CT-skills required for unbiased reasoning (e.g., Abrami et al., 2008 ; Heijltjes et al., 2014b ). However, while some effective teaching methods have been identified, it is as yet unclear under which conditions transfer of CT-skills across tasks or domains can be promoted, that is, the ability to apply acquired knowledge and skills to some new context of related materials (e.g., Barnett & Ceci, 2002 ).

Transfer has been described as existing on a continuum from near to far, with lower degrees of similarity between the initial and transfer situation along the way (Salomon & Perkins, 1989 ). Transferring knowledge or skills to a very similar situation, for instance problems in an exam of the same kind as practiced during the lessons, refers to ‘near’ transfer. By contrast, transferring between situations that share similar structural features but, on appearance, seem remote and alien to one another is considered ‘far’ transfer.

Previous research has shown that CT-skills required for unbiased reasoning consistently failed to transfer to novel problem types, i.e., far transfer, even when using instructional methods that proved effective for fostering transfer in various other domains (Van Peppen et al., 2018 , 2021 ; Heijltjes et al., 2014a , 2014b , 2015 , and this also applies to CT-skills more generally, see for example Halpern, 2014 ; Ritchhart & Perkins, 2005 ; Tiruneh et al., 2014 , 2016 ). This lack of transfer of CT-skills is worrisome because it would be unfeasible to train students on each and every type of reasoning bias they will ever encounter. CT-skills acquired in higher education should transfer to other domains and on-the-job and, therefore, it is crucial to acquire more knowledge on how transfer of these skills can be fostered (and this also applies to CT-skills more generally, see for example, Halpern, 2014 ; Beaulac & Kenyon, 2014 ; Lai, 2011 ; Ritchhart & Perkins, 2005 ). One instructional method that seems promising is comparing correct and erroneous worked examples (i.e., contrasting examples; e.g., Durkin & Rittle-Johnson, 2012 ).

Benefits of studying examples

Over the last decades, a large body of research has investigated learning from studying worked examples as opposed to unsupported problem solving. Worked examples consist of a problem statement and an entirely and correctly worked-out solution procedure (in this paper referred to as correct examples; Renkl, 2014 ; Renkl et al., 2009 ; Sweller et al., 1998 ; Van Gog et al., 2019 ). Typically, studying correct examples is more beneficial for learning than problem-solving practice, especially in initial skill acquisition (for reviews, see Atkinson et al., 2003 ; Renkl, 2014 ; Sweller et al., 2011 ; Van Gog et al., 2019 ). Although this worked example effect has been mainly studied in domains such as mathematics and physics, it has also been demonstrated in learning argumentation skills (Schworm & Renkl, 2007 ), learning to reason about legal cases (Nievelstein et al., 2013 ) and medical cases (Ibiapina et al., 2014 ), and novices’ learning to avoid biased reasoning (Van Peppen et al., 2021 ).

The worked example effect can be explained by cognitive load imposed on working memory (Paas et al., 2003a ; Sweller, 1988 ). Cognitive Load Theory (CLT) suggests that—given the limited capacity and duration of our working memory—learning materials should be designed so as to decrease unnecessary cognitive load related to the presentation of the materials (i.e., extraneous cognitive load). Instead, learners’ attention should be devoted towards processes that are directly relevant for learning (i.e., germane cognitive load). When solving practice problems, novices often use general and weak problem-solving strategies that impose high extraneous load. During learning from worked examples, however, the high level of instructional guidance provides learners with the opportunity to focus directly on the problem-solving principles and their application. Accordingly, learners can use the freed up cognitive capacity to engage in generative processing (Wittrock, 2010 ). Generative processing involves actively constructing meaning from to-be-learned information, by mentally organizing it into coherent knowledge structures and integrating these principles with one’s prior knowledge (i.e., Grabowski, 1996 ; Osborne & Wittrock, 1983 ; Wittrock, 1974 , 1990 , 1992 , 2010 ). These knowledge structures in turn can aid future problem solving (Kalyuga, 2011 ; Renkl, 2014 ; Van Gog et al., 2019 ).

A recent study showed that the worked example effect also applies to novices’ learning to avoid biased reasoning (Van Peppen et al., 2021 Footnote 1 ): participants’ performance on isomorphic tasks on a final test improved after studying correct examples, but not after solving practice problems. However, studying correct examples was not sufficient to establish transfer to novel tasks that shared similar features with the isomorphic tasks, but on which participants had not acquired any knowledge during instruction/practice. The latter finding might be explained by the fact that students sometimes process worked examples superficially and do not spontaneously use the freed up cognitive capacity to engage in generative processing needed for successful transfer (Renkl & Atkinson, 2010 ). Another possibility is that these examples did not sufficiently encourage learners to make abstractions of the underlying principles and explore possible connections between problems (e.g., Perkins & Salomon, 1992 ). It seems that to fully take advantage of worked examples in learning unbiased reasoning, students should be encouraged to be actively involved in the learning process and facilitated to focus on the underlying principles (e.g., Van Gog et al., 2004 ).

The potential of erroneous examples

While most of the worked-example research focuses on correct examples, recent research suggests that students learn at a deeper level and may come to understand the principles behind solution steps better when (also) provided with erroneous examples (e.g., Adams et al., 2014 ; Barbieri & Booth, 2016 ; Booth et al., 2013 ; Durkin & Rittle-Johnson, 2012 ; McLaren et al., 2015 ). In studies involving erroneous examples, which are often preceded by correct examples (e.g., Booth et al., 2015 ), students are usually prompted to locate the incorrect solution step and to explain why this step is incorrect or to correct it. This induces generative processing, such as comparison with internally represented correct examples and (self-)explaining (e.g., Chi et al., 1994 ; McLaren et al., 2015 ; Renkl, 1999 ). Students are encouraged to go beyond noticing surface characteristics and to think deeply about how erroneous steps differ from correct ones and why a solution step is incorrect (Durkin & Rittle-Johnson, 2012 ). This might help them to correctly update schemas of correct concepts and strategies and, moreover, to create schemas for erroneous strategies (Durkin & Rittle-Johnson, 2012 ; Große & Renkl, 2007 ; Siegler, 2002 ; Van den Broek & Kendeou, 2008 ; VanLehn, 1999 ), reducing the probability of recurring erroneous solutions in the future (Siegler, 2002 ).

However, erroneous examples are typically presented separately from correct examples, requiring learners to use mental resources to recall the gist of the no longer visible correct solutions (e.g., Große & Renkl, 2007 ; Stark et al., 2011 ). Splitting attention across time increases the likelihood that mental resources will be expended on activities extraneous to learning, which subsequently may hamper learning (i.e., temporal contiguity effect: e.g., Ginns, 2006 ). One could, therefore, argue that the use of erroneous examples could be optimized by providing them side by side with correct examples (e.g., Renkl & Eitel, 2019 ). This would allow learners to focus on activities directly relevant for learning, such as structural alignment and detection of meaningful commonalities and differences between the examples (e.g., Durkin & Rittle-Johnson, 2012 ; Roelle & Berthold, 2015 ). Indeed, studies on comparing correct and erroneous examples revealed positive effects in math learning (Durkin & Rittle-Johnson, 2012 ; Kawasaki, 2010 ; Loibl & Leuders, 2018 , 2019 ; Siegler, 2002 ).

The present study

We already indicated that it is still an important open question, which instructional strategy can be used to enhance transfer of CT skills. To reiterate, previous research demonstrated that practice consisting of worked example study was more effective for novices’ learning than practice problem solving, but it was not sufficient to establish transfer. Recent research has demonstrated the potential of erroneous examples, which are often preceded by correct examples. Comparing correct and erroneous examples (from here on referred to as contrasting examples) when presenting them side-by-side, seems to hold a considerable promise with respect to promoting generative processing and transfer. Hence, the purpose of the present study was to investigate whether contrasting examples of fictitious students’ solutions on ‘heuristics and biases tasks’ (a specific sub-category of CT skills: e.g., Tversky & Kahneman, 1974 ) would be more effective to foster learning and transfer than studying correct examples only, studying erroneous examples only, or solving practice problems. Performance was measured on a pretest, immediate posttest, 3-week delayed posttest, and 9-month delayed posttest (for half of the participants due to practical reasons), to examine effects on learning and transfer.

Based on the literature presented above, we hypothesized that studying correct examples would impose less cognitive load (i.e., lower investment of mental effort during learning ) than solving practice problems (i.e., worked example effect: e.g., Van Peppen et al., 2021 ; Renkl, 2014 ; Hypothesis 1). Whether there would be differences in invested mental effort between contrasting examples, studying erroneous examples, and solving practice problems, however, is an open question. That is, it is possible that these instructional formats impose a similar level of cognitive load, but originating from different processes: while practice problem solving may impose extraneous load that does not contribute to learning, generative processing of contrasting or erroneous examples may impose germane load that is effective for learning (Sweller et al., 2011 ). As such, it is important to consider invested mental effort (i.e., experienced cognitive load) in combination with learning outcomes. Secondly, we hypothesized that students in all conditions would benefit from the CT-instructions combined with the practice activities, as evidenced by pretest to immediate posttest gains in performance on instructed and practiced items (i.e., learning : Hypothesis 2). Furthermore, based on cognitive load theory, we hypothesized that studying correct examples would be more beneficial for learning than solving practice problems (i.e., worked example effect: e.g., Van Peppen et al., 2021 ; Renkl, 2014 ). Based on the aforementioned literature, we expected that studying erroneous examples would promote generative processing more than studying correct examples. Whether that generative processing would actually enhance learning, however, is an open question. This can only be expected to be the case if learners can actually remember and apply the previously studied information on the correct solution, which arguably involves higher cognitive load (i.e., temporal contiguity effect) than studying correct examples or contrasting examples. As contrasting can help learners to focus on key information and thereby induces generative processes directly relevant for learning (e.g., Durkin & Rittle-Johnson, 2012 ), we expected that contrasting examples would be most effective. Thus, we predict the following pattern of results regarding performance gains on learning items (Hypothesis 3): contrasting examples > correct examples > practice problems. As mentioned above, it is unclear how the erroneous examples condition would compare to the other conditions.

Furthermore, we expected that generative processing would promote transfer. Despite findings of previous studies in other domains (e.g., Paas, 1992 ), we found no evidence in a previous study that studying correct examples or solving practice problems would lead to a difference in transfer performance (Van Peppen et al., 2021 ). Therefore, we predict the following pattern of results regarding performance on non-practiced items of the immediate posttest (i.e., transfer , Hypothesis 4): contrasting examples > correct examples ≥ practice problems. Again, it is unclear how the erroneous examples condition would compare to the other conditions.

We expected these effects (Hypotheses 3 and 4) to persist on the delayed posttests. As effects of generative processing (relative to non-generative learning strategies) sometimes increase as time goes by (Dunlosky et al., 2013 ), they may be even greater after a delay. For a schematic overview of the hypotheses, see Table 1 .

We created an Open Science Framework (OSF) page for this project, where all materials, the dataset, and all script files of the experiment are provided (osf.io/8zve4/).

Participants and design

Participants were 182 first-year ‘Public Administration’ and ‘Safety and Security Management’ students of a Dutch university of applied sciences (i.e., higher professional education), both part of the Academy for Security and Governance. These students were approximately 20 years old ( M  = 19.53, SD  = 1.91) and most of them were male (120 male, 62 female). Before they were involved in these study programs, they completed secondary education (senior general secondary education: n  = 122, pre-university: n  = 7) or went to college (secondary vocational education: n  = 28, higher professional education: n  = 24, university education: n  = 1).

Of the 182 students (i.e., total number of students in these cohorts), 173 students (95%) completed the first experimental session (see Fig.  1 for an overview) and 158 students (87%) completed both the first and second experimental session. Additionally, 83 of these students (46%) of the Safety and Security Management program completed the 9-month delayed posttest during the first mandatory CT-lesson of their second study year (we had no access to another CT-lesson of the Public Administration program). The number of absentees during a lesson (about 15 in total) is quite common for mandatory lessons in these programs and often due to illness or personal circumstances. Students who were absent during the first experimental session and returned to the second experimental session could not participate in the study because they had missed the intervention phase.

Overview of the study design. The four conditions differed in practice activities during the practice phase

We defined a priori that participants would be excluded in case of excessively fast reading speed. Considering that even fast readers can read no more than 350 words per minute (e.g., Trauzettel-Klosinski & Dietz, 2012 ), and the text of our instructions additionally required understanding, we assumed that participants who spent < 0.17 s per word (i.e., 60 s/350 words) did not read the instructions seriously. These participants were excluded from the analyses. Due to drop-outs, we decided to split the analyses to include as many participants as possible. We had a final sample of 170 students ( M age  = 19.54, SD = 1.93; 57 female) for the pretest to immediate posttest analyses, a subsample of 155 students for the immediate to 3-week delayed posttest analyses ( M age  = 19.46, SD = 1.91; 54 female), and a subsample of 82 students (46%) for the 3-week delayed to 9-month delayed posttest ( M age  = 19.27, SD = 1.79; 25 female). We calculated a power function of our analyses using the G*Power software (Faul et al., 2009 ) based on these sample sizes. The power for the crucial Practice Type × Test Moment interaction—under a fixed alpha level of 0.05 and with a correlation between measures of 0.3 (e.g., Van Peppen et al., 2018 )—for detecting a small (η p 2  = .01), medium (η p 2  = .06), and large effect (η p 2  = .14) respectively, is estimated at .42, > .99, and 1.00 for the pretest to immediate posttest analyses; .39, > .99, and 1.00 for the immediate to 3-week delayed posttest analyses; and .21, .90, and > .99 for the 3-week to 9-month delayed posttest. Thus, the power of our study should be sufficient to pick up medium-sized interaction effects.

Students participated in a pretest-intervention–posttest design (see Fig.  1 ). After completing the pretest on learning items (i.e., instructed and practiced during the practice phase), all participants received succinct CT instructions and two correct worked examples. Thereafter, they were randomly assigned to one of four conditions that differed in practice activities during the practice phase: they either (1) compared correct and erroneous examples (‘contrasting examples’, n  = 41; n  = 35; n  = 20); (2) studied correct examples (i.e., step-by-step solutions to unbiased reasoning) and explained why these were right (‘correct examples’, n  = 43; n  = 40; n  = 21); (3) studied erroneous examples (i.e., step-by-step incorrect solutions including biased reasoning) and explained why these were wrong (‘erroneous examples’, n  = 43; n  = 40; n  = 18); or (4) solved practice problems and justified their answers (‘practice problems’, n  = 43; n  = 40; n  = 23). A detailed explanation of the practice activities can be found in the CT-practice subsection below. Immediately after the practice phase and after a 3-week delay, participants completed a posttest on learning items (i.e., instructed and practiced during the practice phase) and transfer items (i.e., not instructed and practiced during the practice phase). Additionally, some students took a posttest after a 9-month delay. Further CT-instructions were given (in three lessons of approx. 90 min) in-between the second session of the experiment and the 9-month follow up. In these lessons, for example, the origins of the concept of CT, inductive and deductive reasoning, and the Toulmin model of argument were discussed. Thus, these data were exploratively analyzed and need to be interpreted with caution.

In the following paragraphs, the used learning materials, instruments and associated measures, and characteristics of the experimental conditions are described.

CT-skills tests

The CT-skills tests consisted of classic heuristics and biases tasks that reflected important aspects of CT. In all tasks, belief bias played a role, that is, when the conclusion aligns with prior beliefs or real-world knowledge but is invalid or vice versa (Evans et al., 1983 ; Markovits & Nantel, 1989 ; Newstead et al., 1992 ). These tasks require that one recognizes the need for analytical and reflective reasoning (i.e. based on knowledge and rules of logical reasoning and statistical reasoning) and switches to this type of reasoning. This is only possible when heuristic responses are successfully inhibited.

The pretest consisted of six classic heuristics and biases items, across two categories (see Online Appendix A for an example of each category): syllogistic reasoning (i.e., logical reasoning) and conjunction (i.e., statistical reasoning) items. Three syllogistic reasoning items measured students’ tendency to be influenced by the believability of a conclusion that is inferred from two premises when evaluating the logical validity of that conclusion (adapted from Evans, 2002 ). For instance, the conclusion that cigarettes are healthy is logically valid given the premises that all things you can smoke are healthy and that you can smoke cigarettes. Most people, however, indicate that the conclusion is invalid because it does not align with their prior beliefs or real-world knowledge (i.e., belief bias, Evans et al., 1983 ). Three conjunction items examined to what extent the conjunction rule ( P (A&B) ≤  P (B))—which states that the probability of multiple specific events both occurring must be lower than the probability of one of these events occurring alone—is neglected (Tversky & Kahneman, 1983 ). To illustrate, people have the tendency to judge two things with a causal or correlational link, for example advanced age and occurrence of heart attacks, as more probable than one of these on its own.

The posttests consisted of parallel versions (i.e., structurally equivalent but different surface features) of the six pretest items which were instructed and practiced and, thus, served to assess differences in learning outcomes. Additionally, the posttests contained six items across two non-practiced categories that served to assess differences in transfer performance (see Online Appendix A for an example of each category). Three Wason selection items measured students’ tendency to disprove a hypothesis by verifying rules rather than falsifying them (i.e., confirmation bias, adapted from Stanovich, 2011 ). Three base-rate items examined students’ tendency to incorrectly judge the likelihood of individual-case evidence (e.g., from personal experience, a single case, or prior beliefs) by not considering all relevant statistical information (i.e., base-rate neglect, adapted from Fong et al., 1986 ; Stanovich & West, 2000 ; Stanovich et al., 2016 ; Tversky & Kahneman, 1974 ). These transfer items shared similar features with the learning categories, namely, one category requiring knowledge and rules of logic (i.e., Wason selection tasks can be solved by applying syllogism rules) and one category requiring knowledge and rules of statistics (i.e., base-rate tasks can be solved by appropriate probability and data interpretation).

The cover stories of all test items were adapted to the domain of participants’ study program (i.e., Public Administration and Safety and Security Management). A multiple-choice (MC) format with different numbers of alternatives per item was used, with only one correct alternative for each item.

CT-instructions

All participants received a 12 min video-based instruction that started with emphasizing the importance of CT in general, describing the features of CT, and explaining which skills and attitudes are needed to think critically. Thereafter, explicit instructions on how to avoid biases in syllogistic reasoning and conjunction fallacies followed, consisting of two worked examples that showed the correct line of reasoning. The purpose of these explicit instructions was to provide students with knowledge on CT and to allow them to mentally correct initially incorrect responses on the items seen in the pretest.

CT-practice

Participants performed practice activities on the task categories that they were given instructions on (i.e., syllogistic reasoning and conjunction tasks). The CT-practice consisted of four practice tasks, two of each of the task categories. Each practice task was again adapted to the study domain and started with the problem statement (see Online Appendix B for an example of a practice task of each condition). Participants in the correct examples condition were provided with a fictitious student’s correct solution and explanation to the problem, including auxiliary representations, and were prompted to explain why the solution steps were correct. Participants in the erroneous examples condition received a fictitious student’s erroneous solution to the problem, again including auxiliary representations. They were prompted to indicate the erroneous solution step and to provide the correct solution themselves. In the contrasting examples , participants were provided fictitious students’ correct and erroneous solutions to the problem and were prompted to compare the two solutions and to indicate the erroneous solution and the erroneous solution step. Participants in the practice problems condition had to solve the problems themselves, that is, they were instructed to choose the best answer option and were asked to explain how the answer was obtained. Participants in all conditions were asked to read the practice tasks thoroughly. To minimize differences in time investment (i.e., the contrasting examples consisted of considerably more text), we have added self-explanation prompts in the correct examples, erroneous examples, and practice problem conditions.

Mental effort

After each test item and practice-task, participants were asked to report how much effort they invested in completing that task or item on a 9-point subjective rating scale ranging from (1) very, very low effort to (9) very, very high effort (Paas, 1992 ). This widely used scale in educational research (for overviews, see Paas et al., 2003b ; Van Gog & Paas, 2008 ), is assumed to reflect the cognitive capacity actually allocated to accommodate the demands imposed by the task or item (Paas et al., 2003a ).

The study was run during the first two lessons of a mandatory first-year CT-course in two, very similar, Security and Governance study programs. Participants were not given CT-instructions in between these lessons. They completed the study in a computer classroom at the participants’ university with an entire class of students, their teacher, and the experiment leader (first author) present. When entering the classroom, participants were instructed to sit down at one of the desks and read an A4-paper containing some general instructions and a link to the computer-based environment (Qualtrics platform). The first experimental session (ca. 90 min) began with obtaining written consent from all participants. Then, participants filled out a demographic questionnaire and completed the pretest. Next, participants entered the practice phase in which they first viewed the video-based CT-instructions and then were assigned to one of the four practice conditions. Immediately after the practice phase, participants completed the immediate posttest. Approximately 3 weeks later, participants took the delayed posttest (ca. 20 min) in their computer classrooms. Additionally, students of the Safety and Security Management program took the 9-month delayed posttest during the first mandatory CT-lesson of their second study year, Footnote 2 which was exactly the same as the 3-week delayed posttest. During all experimental sessions, participants could work at their own pace and were allowed to use scrap paper. Time-on-task was logged during all phase and participants had to indicate after each test item and practice-task how much effort they invested. Participants had to wait (in silence) until the last participants had finished before they were allowed to leave the classroom.

Data analysis

All test items were MC-only questions, except for one learning item and one transfer items with only two alternatives (conjunction item and base-rate item) that were MC-plus-motivation questions to prevent participants from guessing. Items were scored for accuracy, that is, unbiased reasoning; 1 point for each correct alternative on the MC-only questions or a maximum of 1 point (increasing in steps of 0.5) for the correct explanation for the MC-plus-motivation question using a coding scheme that can be found on our OSF-page. Because two transfer items (i.e., one Wason selection item and one base-rate item) appeared to substantially reduce the reliability of the transfer performance measure, presumably as a result of low variance due to floor effects, we decided to omit these items from our analyses. As a result, participants could attain a maximum total score of 6 on the learning items and a maximum score of 4 on the transfer items. For comparability, learning and transfer outcomes were computed as percentage correct scores instead of total scores. Participants’ explanations on the open questions of the tests were coded by one rater and another rater (the first author) coded 25% of the explanations of the immediate posttest. Intra-class correlation coefficients were 0.990 for the learning test items and 0.957 for the transfer test items. After the discrepancies were resolved by discussion, the primary rater’s codes were used in the analyses.

Cronbach’s alpha on invested mental effort ratings during studying correct examples, studying erroneous examples, contrasting examples, and solving practice problems, respectively, was .87, .76, .77, and .65. Cronbach’s alpha on the learning items was .21, .42, .58, and .31 on the pretest, immediate posttest, 3-week delayed posttest, and 9-month delayed posttest, respectively. The low reliability on the pretest might be explained by the fact that a lack of prior knowledge requires guessing of answers. As such, inter-item correlations are low, resulting in a low Cronbach’s alpha. Cronbach’s alpha on the transfer items was .31, .12, and .29 on the immediate, 3-week delayed, and 9-month delayed posttest, respectively. Cronbach’s alpha on the mental effort items belonging to the learning items was .73, .79, .81, and .76 on the pretest, immediate posttest, 3-week delayed posttest, and 9-month delayed posttest, respectively. Cronbach’s alpha on the mental effort items belonging to the transfer items was .71, .75, and .64 on the immediate posttest, 3-week delayed posttest, and 9-month delayed posttest, respectively. However, caution is required in interpreting the above values because sample sizes as in studies like this do not seem to produce sufficiently precise alpha coefficients (e.g. Charter, 2003 ). Cronbach’s alpha is a statistic and therefore subject to sample fluctuations. Hence, one should be careful with drawing firm conclusions about the precision of Cronbach’s alpha in the population (the parameter) based on small sample sizes (i.e., in reliability literature, samples of 300–400 are considered small, see for instance Charter, 2003 ; Nunally & Bernstein, 1994 ; Segall, 1994 ).

There was no significant difference on pretest performance between participants who stayed in the study and those who dropped out after the first session, t (172) = .38, p  = .706, and those who dropped out after the second session, t (172) = − 1.46, p  = .146. Furthermore, there was no significant difference in educational background between participants who stayed in the study and those who dropped out after the first session, r (172) = .13, p  = .087, and those who dropped out after the second session, r (172) = − .01, p  = .860. Finally, there was no significant difference in age between participants who stayed in the study and those who dropped out after the first session, t (172) = − 1.51, p  = .134, but there was a difference between participants who stayed in the study and those who dropped out after the second session, t (172) = − 2.02, p  = .045. However, age did not correlate significantly with learning performance (minimum p  = .553) and was therefore not a confounding variable.

Additionally, participants’ performance during the practice phase was scored for accuracy, that is, unbiased reasoning. In each condition, participants could attain a maximum score of 2 points (increasing in steps of 0.5) for the correct answer on each problem (either MC-only answers or MC-plus-explanation answers), resulting in a maximum total score of 8. The explanations given during practice were coded for explicit relations to the principles that were communicated in the instructions (i.e., principle-based explanations; Renkl, 2014 ). For instance, participants earned the full 2 points if they explained in a conjunction task that the first statement is part of the second statement and that the first statement therefore can never be more likely than the two statements combined. Participants’ explanations were coded by the first author and another rater independently coded 25% of the explanations. Intra-class correlation coefficients were 0.941, 0.946, and 0.977 for performance in the correct examples, erroneous examples, and practice problems conditions respectively (contrasting examples consisted of MC-only questions). After a discussion between the raters about the discrepancies, the primary rater’s codes were updated and used in the exploratory analyses.

For all analyses in this paper, a p -value of .05 was used a threshold for statistical significance. Partial eta-squared (η p 2 ) is reported as an effect size for all ANOVAs (see Table 3 ) with η p 2  = .01, η p 2  = .06, and η p 2  = .14 denoting small, medium, and large effects, respectively (Cohen, 1988 ). Cramer’s V is reported as an effect size for chi-square tests with (having 2 degrees of freedom) V  = .07, V  = .21, and V  = .35 denoting small, medium, and large effects, respectively.

Preliminary analyses

Check on condition equivalence.

Before running any of the main analyses, we checked our conditions on equivalence. Preliminary analyses confirmed that there were no a-priori differences between the conditions in educational background, χ 2 (15) = 15.57, p  = .411, V  = .18; gender, χ 2 (3) = 1.21, p  = .750, V  = .08; performance on the pretest, F (3, 165) = 0.42, p  = .739, η p 2  = .01; time spent on the pretest, F (3, 165) = 0.16, p  = .926, η 2  < .01; and mental effort invested on the pretest, F (3, 165) = 0.80, p  = .498, η 2  = .01. Further, we estimated two multiple regression models (learning and transfer) with practice type and performance on the pretest as explanatory variables, including the interaction between practice type and performance on the pretest. There was no evidence of an interaction effect (learning: R 2  = .07, F (1, 166) = .296, p  = .587; transfer: R 2  = .07, F (1, 166) = .260, p  = .611) and we can, therefore, conclude that the relationship between practice type and performance on the posttest does not depend on performance on the pretest.

Check on time-on-task

The Levene’s test for equality of variances was significant, F (3, 166) = 9.57, p  < .001. Therefore, a Brown–Forsythe one-way ANOVA was conducted. This analysis revealed a significant time-on-task (in seconds) difference between the conditions during practice, F (3, 120.28) = 16.19, p  < .001, η 2  = .22. Pairwise comparisons showed that time-on-task was comparable between erroneous examples ( M  = 862.79, SD  = 422.43) and correct examples ( M  = 839.58, SD  = 298.33) and between contrasting examples ( M  = 512.29, SD  = 130.21) and practice problems ( M  = 500.41, SD  = 130.21). However, time-on-task was significantly higher in the first two conditions compared to the latter two conditions (erroneous examples = correct examples > contrasting examples = practice problems), all p ’s < .001. This should be considered when interpreting the results on effort and posttest performance.

Main analyses

Descriptive and test statistics are presented in Table 2 , 3 , and 4 . Correlations between several variables are presented in Table 5 . It is important to realize that we measured mental effort as an indicator of overall experienced cognitive load. It is known, though, that the relation with learning depends on the origin of the experienced cognitive load. That is, if it originates mainly from germane processes that contribute to learning, high load would positively correlate with test performance, if it originates from extraneous processes, it would negatively correlate with test performance. Caution is warranted in interpreting these correlations, however, because of the exploratory nature of these correlation analyses, which makes it impossible to control for the probability of type 1 errors. We also exploratively analyzed invested mental effort and time-on-task data on the posttest; however, these analyses did not have much added value for this paper and, therefore, are not reported here but will be provided on our OSF-project page.

Performance during the practice phase

As each condition received different prompts during practice, performance during the practice phase could not be meaningfully compared between conditions and, therefore, we decided to report descriptive statistics only to describe the level of performance during the practice phase per condition (see Table 2 ). Descriptive statistics showed that participants earned more than half of the maximum total score while studying correct examples or engaging in contrasting examples. Participants who studied erroneous examples or solved practice problems performed worse during practice.

Mental effort during learning

A one-way ANOVA revealed a significant main effect of Practice Type on mental effort invested in the practice tasks. Contrary to hypothesis 1, a Tukey post hoc test revealed that participants who solved practice problems invested significantly less effort ( M  = 4.28, SD  = 1.11) than participants who engaged in contrasting examples ( M  = 5.08, SD  = 1.29, p  = .022) or studied erroneous examples ( M  = 5.17, SD  = 1.19, p  = .008). There were no other significant differences in effort investment between conditions. Interestingly, invested mental effort during contrasting examples correlated negatively with pretest to posttest performance gains on learning items, indicating that the experienced load originated mainly from extraneous processes (see Table 5 ).

Test performance

The data on learning items were analyzed with two 2 × 4 mixed ANOVAs with Test Moment (pretest and immediate posttest/immediate posttest and 3-week delayed posttest) as within-subjects factor and Practice Type (correct examples, erroneous examples, contrasting examples, and practice problems) as between-subjects factor. Because transfer items were not included in the pretest, the data on transfer items were analyzed by a 2 × 4 mixed ANOVA with Test Moment (immediate posttest and 3-week delayed posttest) as within-subjects factor and Practice Type (correct examples, erroneous examples, contrasting examples, and practice problems) as between-subjects factor.

Performance on learning items

In line with Hypothesis 2, the pretest-immediate posttest analysis showed a main effect of Test Moment on performance on learning items: participants’ performance improved from pretest ( M  = 27.26, SE  = 1.43) to immediate posttest ( M  = 49.98, SE  = 1.87). In contrast to Hypothesis 3, the results did not reveal a main effect of Practice Type, nor an interaction between Practice Type and Test Moment. The second analysis ( N  = 154)—to test whether effects are still present after 3 weeks—showed a main effect of Test Moment: participants performed better on the delayed posttest ( M  = 55.54, SE  = 2.16) compared to the immediate posttest ( M  = 50.95, SE  = 2.00). Again, contrary to our hypothesis, there was no main effect of Practice Type, nor an interaction between Practice Type and Test Moment.

Performance on transfer items

The results revealed no main effect of Test Moment. Moreover, in contrast to Hypothesis 4, the results did not reveal a main effect of Practice Type, nor an interaction between Practice Type and Test Moment. Footnote 3

Exploratory analyses

Participants from one of the study programs were tested again after a 9-month delay. Regarding performance on learning items, a 2 × 4 mixed ANOVA with Test Moment (3-week delayed posttest or 9-month delayed posttest) as within-subjects factor and Practice Type (correct examples, erroneous examples, contrasting examples, and practice problems) as between-subjects factor revealed a main effect of Test Moment (see Table 2 ): participants’ performance improved from 3-week delayed posttest ( M  = 53.30, SE  = 2.69) to 9-month delayed posttest ( M  = 63.00, SE  = 2.24). The results did not reveal a main effect of Practice Type, nor an interaction between Practice Type and Test Moment.

Regarding performance on transfer items , a 2 × 4 mixed ANOVA with Test Moment (3-week delayed posttest and 9-month delayed posttest) as within-subjects factor and Practice Type (correct examples, erroneous examples, contrasting examples, and practice problems) as between-subjects factor revealed a main effect of Test Moment (see Table 2 ): participants performed lower on the 3-week delayed test ( M  = 19.25, SE  = 1.60) than the 9-month delayed test ( M  = 24.84, SE  = 1.67). The results did not reveal a main effect of Practice Type, nor an interaction between Practice Type and Test Moment.

Previous research has demonstrated that providing students with explicit instructions combined with practice on domain-relevant tasks was beneficial for learning to reason in an unbiased manner (Heijltjes et al., 2014a , 2014b , 2015 ), and that practice consisting of worked example study was more effective for novices’ learning than practice problem solving (Van Peppen et al., 2021 ). However, this was not sufficient to establish transfer to novel tasks. With the present study, we aimed to find out whether contrasting examples—which has been proven effective for promoting transfer in other learning domains—would promote learning and transfer of reasoning skills.

Findings and implications

Our results corroborate the finding of previous studies (e.g., Heijltjes et al., 2015 ; Van Peppen et al., 2018 , 2021 ) that providing students with explicit instructions and practice activities is effective for learning to avoid biased reasoning (Hypothesis 1), since we found considerable pretest to immediate posttest gains on practiced items. Moreover, our results revealed that participants’ performance improved even further after a 3-week and a 9-month delay, although the latter finding could also be attributed to the further instructions that were given in courses in-between the 3-week and 9-month follow up. That students improved in the longer term seems to indicate that our instructional intervention triggered active and deep processing and contributed to storage strength. Hence, our findings provide further evidence that a relatively brief instructional intervention including explicit instructions and practice opportunities is effective for learning of CT-skills, which is promising for educational practice.

In contrast to our expectations, however, we did not find any differences among conditions on either learning or transfer (Hypothesis 3). It is surprising that the present study did not reveal a beneficial effect of studying correct examples as opposed to practicing with problems, as this worked example effect has been demonstrated with many different tasks (Renkl, 2014 ; Van Gog et al., 2019 ), including heuristics-and-biases tasks (Van Peppen et al., 2021 ).

Given that most studies on the worked example effect use pure practice conditions or give minimal instructions prior to practice (e.g., Van Gog et al., 2019 ), whereas the current study was preceded by instructions including two worked examples, one might wonder whether this contributed to the lack of effect. That is, the effects are usually not investigated in a context in which elaborate processing of instructions precedes practice, as in the current (classroom) study, and this may have affected the results. It seems possible that the CT-instructions already had a substantial effect on learning unbiased reasoning, making it difficult to find differential effects of different types of practice activities. This suggestion, however, contradicts the relatively low performance during the practice phase. Moreover, one could argue that if these instructions would lead to higher prior knowledge, it should render the correct worked examples less useful (cf. research on the ‘expertise reversal effect’) and should help those in the other practice conditions perform better on the practice problems, but we did not find that either. Furthermore, these instructions were also provided in a previous study in which a worked example effect was found in two experiments (Van Peppen et al., 2021 ). A major difference between that prior study and this one, however, is that in the present study, participants were prompted to self-explain while studying examples or solving practice problems. Prompting self-explanations, however, seems to encourage students to engage in deep processing during learning (Chi et al., 1994 ), especially for students with sufficient prior knowledge (Renkl & Atkinson, 2010 ). In the present study, this might have interfered with the usual worked-example effect. However, the quality of the self-explanations was higher in the correct example condition than in the problem-solving condition (i.e., performance during the practice phase scores), making the absence of a worked example effect even more remarkable. Given that the worked example effect mainly occurs for novices, one could argue that participants in the current study had more prior knowledge than participants in that prior study; however, it concerned a similar group of students and descriptive statistics showed that students performed comparable on average in both studies.

Another potential explanation might lie in the number of practice tasks, which differed between the prior study (nine tasks: Van Peppen et al., 2021 ) and present study (four tasks), and which might moderate the effect of worked examples. The mean scores on the pretests as well as the performance progress in the practice problem condition was comparable with the previous study, but the progress of the worked example condition was considerably smaller. As it is crucial for a worked example effect that the worked-out solution procedures are understood, it might be that the effect did not emerge in the present study because participants did not get sufficient worked examples during practice.

This might perhaps also explain why contrasting examples did not benefit learning or transfer in the present study. Possibly, students first need to gain a better understanding of the subject matter with heuristics-and-biases tasks before they are able to benefit from aligning the examples (Rittle-Johnson et al., 2009 ). In particular the lack of transfer effects might be related to the duration or extensiveness of the practice activities; even though students learned to solve reasoning tasks, their subject knowledge may have been insufficient to solve novel tasks. As such, it can be argued that establishing transfer needs longer or more extensive practice. Contrasting examples seem to help students extend and refine their knowledge and skills through engaging in comparing activities and analyzing errors, that is, they seem to help them to correctly update schemas of correct concepts and strategies and to create schemas for erroneous strategies reducing the probability of recurring erroneous solutions in the future. However, more attention may need to be paid to the acquisition of the new knowledge and integration with wat students already know (see the Dimensions of Learning framework; Marzano et al., 1993 ). Potentially, having contrasting examples preceded by a more extensive instruction phase to guarantee a better understanding of logical and statistical reasoning would enhance learning and establish transfer. Another possibility would be to provide more guidance in the contrasting examples, as has been done in previous studies by explicitly marking the erroneous examples as incorrect and prompting students to reflect or elaborate on the examples (e.g., Durkin & Rittle-Johnson, 2012 ; Loibl & Leuders, 2018 , 2019 ). It should be noted though, that the lower time on task in the contrasting condition might also be indicative of a motivational problem; whereas the side-by-side presentation was intended to encourage deep processing, it might have had the opposite effect that students might have engaged in superficial processing, just scanning to see where differences in the examples lay, without thinking much about the underlying principles. This idea is confirmed by the finding that invested mental effort during comparing correct and erroneous examples correlated negatively with performance gains on learning items, indicating that the experienced load originated mainly from extraneous processes. It would be interesting in future research to manipulate knowledge gained during instruction to investigate whether prior knowledge indeed moderates the effect of contrasting examples and to examine the interplay between contrasting examples, reflection/elaboration prompts, and final test performance.

Another possible explanation for the lack of a beneficial effect of contrasting examples might be related to the self-explanations prompts that were provided in the correct examples, erroneous examples, and practice problems conditions. Although the prompts differ, it is important to note that the explicit instruction to compare the solution process likely evokes self-explaining processes as well. The reason we added self-explanation prompts to the other conditions was to rule out an effect of prompting as such, as well as a potential effect of time on task (i.e., the text length in the contrasting examples condition was considerably longer than in the other conditions). The positive effect of contrasting examples might have been negated by a positive effect of the self-explanation prompts given in the other conditions. However, had we found a positive effect of comparing, as we expected, our design would have increased the likelihood that this was due to the comparison process and not just to more in-depth processing or higher processing time through self-explaining. Unexpectedly, we did find time-on-task differences between conditions during practice, but this does not seem to affect our findings. Time-on-task during practice was not correlated with learning and transfer posttest performance. This also becomes apparent from the condition means, i.e., the conditions with the lowest time-on-task means did not differ on learning and transfer compared to the conditions with the highest time-on-task means.

The classroom setting might also explain why there were no differential effects of contrasting examples. This study was conducted as part of an existing course and the learning materials were relevant for the course/exam and. Because of that, students’ willingness to invest effort in their performance may have been higher than is generally the case in psychological laboratory studies: their performance on such tasks actually mattered (intrinsically or extrinsically) to them. As such, students in the control conditions may have engaged in generative processing themselves, for instance by trying to compare the given correct (or erroneous) examples with internally represented erroneous (or correct) solutions. Therefore, it is possible that effects of generative processing strategies such as comparing correct and erroneous examples found in the psychological laboratory—where students participate to earn required research credits and the learning materials are not part of their study program—might not readily transfer to field experiments conducted in real classrooms.

The absence of differential effects of the practice activities on learning and transfer may also be related to the affective and attitudinal dimension of CT. Attitudes and perceptions about learning affect learning (Marzano et al., 1993 ), probably even more so in the CT-domain than in other learning domains. Being able to think critically relies heavily on the extent to which one possesses the requisite skills and is able to use these skills, but also on whether one is inclined to use these skills (i.e., thinking dispositions; Perkins et al., 1993 ).

The present study raises further questions about how transfer of CT-skills can be promoted. Although several studies have shown that to enhance transfer of knowledge or skills, instructional strategies should contribute to storage strength by effortful learning conditions that trigger active and deep processing ( desirable difficulties ; e.g., Bjork & Bjork, 2011 ), the present study—once again (Van Peppen et al., 2018 , 2021 ; Heijltjes et al., 2014a , 2014b , 2015 )—showed that this may not apply to transfer of CT-skills. This lack of transfer could lie in inadequate recall of the acquired knowledge, recognition that the acquired knowledge is relevant to the new task, and/or the ability to actually map that knowledge onto the new task (Barnett & Ceci, 2002 ). Following this, a further study should elucidate what the underlying mechanism(s) is/are to shed more light on how to promote transfer of CT-skills.

Limitations and strengths

One limitation of this study is that our measures showed low levels of reliability. Under these circumstances, the probability of detecting a significant effect—given one exists—are low (e.g., Cleary et al., 1970 ; Rogers & Hopkins, 1988 ), and subsequently, the chance that Type 2 errors have occurred in the current study is relatively high. In our study, the low levels of reliability can probably be explained by the multidimensional nature of the CT-test, that is, it represents multiple constructs that do not correlate with each other. Performance on these tasks depends not only on the extent to which that task elicits a bias (resulting from heuristic reasoning), but also on the extent to which a person possesses the requisite mindware (e.g., rules or logic or probability). Thus, systematic variance in performance on such tasks can either be explained by a person’s use of heuristics or his/her available mindware. If it differs per item to what extent a correct answer depends on these two aspects, and if these aspects are not correlated, there may not be a common factor explaining all interrelationships between the measured items. Moreover, the reliability issue may have increased even more since multiple task types were included in the CT-skills tests, requiring different, and perhaps uncorrelated, types of mindware (e.g., rules of logic or probability). Future research, therefore, would need to find ways to improve CT measures (i.e., decrease measurement error), for instance by narrowing down the test into a single measurable construct, or should utilize measures known to have acceptable levels of reliability (LeBel & Paunonen, 2011 ). The latter option seems challenging, however, as multiple studies report rather low levels of reliability of tests consisting of heuristics and biases tasks (Aczel et al., 2015 ; West et al., 2008 ) and revealed concerns with the reliability of widely used standardized CT tests, particularly with regard to subscales (Bernard et al., 2008 ; Bondy et al., 2001 ; Ku, 2009 ; Leppa, 1997 ; Liu et al., 2014 ; Loo & Thorpe, 1999 ). This raises the question whether these issues are related to the general construct CT. To achieve further progress in research on instructional methods for teaching CT, more knowledge on the construct validity of CT in general and unbiased reasoning in particular is needed. When the aim is to evaluate CT as a whole, one should perhaps move towards a more holistic measurement method, for instance by performing pairwise comparisons (i.e., comparative judgment; Bramley & Vitello, 2018 ; Lesterhuis et al., 2017 ). If, however, the intention is to measure specific aspects of CT, one should indicate specifically which aspect of CT to measure and select a suitable test for that aspect. Mainly considering that individual aspects of CT may not be as strongly correlated as thought and then could not be included in one scale.

Another point worth mentioning, is that we opted for assessing invested mental effort, which reflects the amount of cognitive load students experienced. This is informative when combined with their performance (for a more elaborate discussion, see Van Gog & Paas, 2008 ). Moreover, research has shown that it is important to measure cognitive load immediately after each task (e.g., Schmeck et al., 2015 ; Van Gog et al., 2012 ) and the mental effort rating scale (Paas, 1992 ) is easy to apply after each task. However, it unfortunately does not allow us to distinguish between different types of load. It should be noted, though, that it seems very challenging to do this with other measurement instruments (e.g., Skulmowski & Rey, 2017 ). Also, instruments that might be suited for this purpose, for example the rating scale developed by Leppink et al. ( 2013 ), would have been too long to apply after each task in the present study.

A strength of the current study is that it was conducted in a real educational setting as part of an existing CT course, which increases ecological validity. Despite the wealth of worked examples research, classroom studies are relatively rare. Interestingly, (multi-session) classroom studies on math and chemistry have also failed to find the worked example effect, although—in contrast to the present study—worked examples often did show clear efficiency benefits compared to practice problems (McLaren et al., 2016 ; Van Loon-Hillen et al., 2012 ). In line with our finding, a classroom study by Isotani et al. ( 2011 ) indicated that (high prior knowledge) students did not benefit more from studying erroneous examples than from correct examples or practice problems. As discussed earlier in the discussion, the classroom setting might explain the absence of generative processing strategies on learning and transfer. This suggests a theoretical implication, namely that beneficial effects of such strategies might become smaller when the willingness to invest increases and vice versa.

To conclude, based on the findings of the present study, comparing correct and erroneous examples (i.e., contrasting examples) does not seem to be a promising instructional method to further enhance learning and transfer of specific—and specifically tested—CT skills. Consequently, our findings raise questions about the preconditions of contrasting examples effects and effects of generative processing strategies in general, such as the setting in which they are presented to students. Further research on the exact boundary conditions, through solid laboratory and classroom studies, is therefore recommended. Moreover, this study provides valuable insights for educational practice. That is, providing students with explicit CT-instruction and the opportunity to practice with domain-relevant problems in a relatively short instructional intervention has the potential to improve learning. The format of the practice tasks does not seem to matter much, although a prior study did find a benefit of studying correct examples, which might therefore be the safest bet. Finally, this study again underlines the great difficulty of designing instructions to enhance CT-skills in such a way that these would also transfer across tasks/domains.

Data availability

All data, script files, and materials are provided on the Open Science Framework (OSF) project page that we created for this study (anonymized view-only link: https://osf.io/8zve4/?view_only=ca500b3aeab5406290310de34323457b ).

Code availability

Not applicable.

This study investigated effects of interleaved practice (as opposed to blocked practice) on students’ learning and transfer of unbiased reasoning. Given that interleaved practice seems to impose high cognitive load, which may hinder learning, it was additionally tested whether this effect interacts with the format of the practice tasks (i.e., correct examples or practice problems).

We had no access to another CT-lesson of the Public Administration program, so due to practical reasons, students of this program were not administered to the 9-month delayed posttest.

We also exploratively analyzed the learning and transfer data for each individual measurement point and we analyzed performance on single learning and transfer items. The outcomes did not deviate markedly from the findings on sum scores (i.e., no effects of Practice Type were found). Test statistics can be found on our OSF-project page and the descriptive statistics of performance per single item can be found in Table 4 .

Abrami, P. C., Bernard, R. M., Borokhovski, E., Wade, A., Surkes, M. A., Tamim, R., & Zhang, D. (2008). Instructional interventions affecting critical thinking skills and dispositions: A stage 1 meta-analysis. Review of Educational Research, 78 , 1102–1134. https://doi.org/10.3102/0034654308326084

Article   Google Scholar  

Abrami, P. C., Bernard, R. M., Borokhovski, E., Waddington, D. I., Wade, C. A., & Persson, T. (2014). Strategies for teaching students to think critically: A meta-analysis. Review of Educational Research, 85 , 275–314. https://doi.org/10.3102/0034654314551063

Aczel, B., Bago, B., Szollosi, A., Foldes, A., & Lukacs, B. (2015). Measuring individual differences in decision biases: Methodological considerations. Frontiers in Psychology . https://doi.org/10.3389/fpsyg.2015.01770

Adams, D. M., McLaren, B. M., Durkin, K., Mayer, R. E., Rittle-Johnson, B., Isotani, S., & Van Velsen, M. (2014). Using erroneous examples to improve mathematics learning with a web-based tutoring system. Computers in Human Behavior, 36 , 401–411. https://doi.org/10.1016/j.chb.2014.03.053

Angeli, C., & Valanides, N. (2009). Instructional effects on critical thinking: Performance on ill-defined issues. Learning and Instruction, 19 , 322–334. https://doi.org/10.1016/j.learninstruc.2008.06.010

Arum, R., & Roksa, J. (2011). Limited learning on college campuses. Society, 48 , 203–207. https://doi.org/10.1007/s12115-011-9417-8

Atkinson, R. K., Renkl, A., & Merrill, M. M. (2003). Transitioning from studying examples to solving problems: Effects of self-explanation prompts and fading worked-out steps. Journal of Educational Psychology, 95 , 774–783. https://doi.org/10.1037/0022-0663.95.4.774

Barbieri, C., & Booth, J. L. (2016). Support for struggling students in algebra: Contributions of incorrect worked examples. Learning and Individual Differences, 48 , 36–44. https://doi.org/10.1016/j.lindif.2016.04.001

Barnett, S. M., & Ceci, S. J. (2002). When and where do we apply what we learn?: A taxonomy for far transfer. Psychological Bulletin, 128 , 612–636. https://doi.org/10.1037/0033-2909.128.4.612

Beaulac, G. & Kenyon, T. (2014). Critical thinking education and debiasing. Informal Logic, 34 , 341–363. https://doi.org/10.22329/il.v34i4.4203

Bernard, R. M., Zhang, D., Abrami, P. C., Sicoly, F., Borokhovski, E., & Surkes, M. A. (2008). Exploring the structure of the Watson-Glaser Critical Thinking Appraisal: One scale or many subscales? Thinking Skills and Creativity, 3 , 15–22. https://doi.org/10.1016/j.tsc.2007.11.001

Bjork, E. L., & Bjork, R. A. (2011). Making things hard on yourself, but in a good way: Creating desirable difficulties to enhance learning. In M. A. Gernsbacher, R. W. Pew, & J. R. Pomerantz (Eds.), Psychology and the real world: Essays illustrating fundamental contributions to society (pp. 59–68). Worth Publishers.

Google Scholar  

Bondy, K. N., Koenigseder, L. A., Ishee, J. H., & Williams, B. G. (2001). Psychometric properties of the California Critical Thinking Tests. Journal of Nursing Measurement, 9 , 309–328. https://doi.org/10.1891/1061-3749.9.3.309

Booth, J. L., Lange, K. E., Koedinger, K. R., & Newton, K. J. (2013). Using example problems to improve student learning in algebra: Differentiating between correct and incorrect examples. Learning and Instruction, 25 , 24–34. https://doi.org/10.1016/j.learninstruc.2009.10.001

Booth, J. L., Oyer, M. H., Paré-Blagoev, E. J., Elliot, A. J., Barbieri, C., Augustine, A., & Koedinger, K. R. (2015). Learning algebra by example in real-world classrooms. Journal of Research on Educational Effectiveness, 8 , 530–551. https://doi.org/10.1080/19345747.2015.1055636

Bramley, T., & Vitello, S. (2018). The effect of adaptivity on the reliability coefficient in adaptive comparative judgement. Assessment in Education: Principles, Policy & Practice, 2018 , 1–16. https://doi.org/10.1080/0969594X.2017.1418734

Charter, R. A. (2003). Study samples are too small to produce sufficiently precise reliability coefficients. The Journal of General Psychology, 130 , 117–129.

Chi, M. T. H., de Leeuw, N., Chiu, M., & LaVancher, C. (1994). Eliciting self-explanation improves understanding. Cognitive Science, 18 , 439–477. https://doi.org/10.1207/s15516709cog1803_3

Cleary, T. A., Linn, R. L., & Walster, G. W. (1970). Effect of reliability and validity on power of statistical tests. Sociological Methodology, 2 , 130–138. https://doi.org/10.1037/a0031026

Cohen, J (1988). Statistical power analysis for the behavioral sciences (2nd. ed., reprint). Psychology Press.

Davies, M. (2013). Critical thinking and the disciplines reconsidered. Higher Education Research & Development, 32 , 529–544. https://doi.org/10.1080/07294360.2012.697878

Dunlosky, J., Rawson, K. A., Marsh, E. J., Nathan, M. J., & Willingham, D. T. (2013). Improving students’ learning with effective learning techniques: Promising directions from cognitive and educational psychology. Psychological Science in the Public Interest, 14 , 4–58. https://doi.org/10.1177/1529100612453266

Durkin, K., & Rittle-Johnson, B. (2012). The effectiveness of using incorrect examples to support learning about decimal magnitude. Learning and Instruction, 22 , 206–214. https://doi.org/10.1016/j.learninstruc.2011.11.001

Ennis, R. H. (1989). Critical thinking and subject specificity: Clarification and needed research. Educational Researcher, 18 , 4–10. https://doi.org/10.3102/0013189X018003004

Evans, J. S. B. (2002). Logic and human reasoning: An assessment of the deduction paradigm. Psychological Bulletin, 128 , 978–996. https://doi.org/10.1037/0033-2909.128.6.978

Evans, J. S. B. (2003). In two minds: Dual-process accounts of reasoning. Trends in Cognitive Sciences, 7 , 454–459. https://doi.org/10.1016/j.tics.2003.08.012

Evans, J. S. B., Barston, J. L., & Pollard, P. (1983). On the conflict between logic and belief in syllogistic reasoning. Memory & Cognition, 11 , 295–306. https://doi.org/10.3758/BF03196976

Facione, P. A. (1990). Critical thinking: A statement of expert consensus for purposes of educational assessment and instruction . The California Academic Press.

Faul, F., Erdfelder, E., Buchner, A., & Lang, A. G. (2009). Statistical power analyses using G* Power 3.1: Tests for correlation and regression analyses. Behavior Research Methods, 41 , 1149–1160. https://doi.org/10.3758/BRM.41.4.1149 .

Flores, K. L., Matkin, G. S., Burbach, M. E., Quinn, C. E., & Harding, H. (2012). Deficient critical thinking skills among college graduates: Implications for leadership. Educational Philosophy and Theory, 44 , 212–230. https://doi.org/10.1111/j.1469-5812.2010.00672.x

Fong, G. T., Krantz, D. H., & Nisbett, R. E. (1986). The effects of statistical training on thinking about everyday problems. Cognitive Psychology, 18 , 253–292. https://doi.org/10.1016/0010-0285(86)90001-0

Ginns, P. (2006). Integrating information: A meta-analysis of the spatial contiguity and temporal contiguity effects. Learning and Instruction, 16 , 511–525. https://doi.org/10.1016/j.learninstruc.2006.10.001

Grabowski, B. (1996). Generative learning. Past, present, and future. In D. H. Jonassen (Ed.), Handbook of research for educational communications and technology (pp. 897–918). Macimillian Library Reference.

Große, C. S., & Renkl, A. (2007). Finding and fixing errors in worked examples: Can this foster learning outcomes? Learning and Instruction, 17 , 612–634. https://doi.org/10.1016/j.learninstruc.2007.09.008

Halpern, D. F. (2014). Critical thinking across the curriculum: A brief edition of thought & knowledge . Routledge.

Book   Google Scholar  

Heijltjes, A., Van Gog, T., Leppink, J., & Paas, F. (2014a). Improving critical thinking: Effects of dispositions and instructions on economics students’ reasoning skills. Learning and Instruction, 29 , 31–42. https://doi.org/10.1016/j.learninstruc.2013.07.003

Heijltjes, A., Van Gog, T., & Paas, F. (2014b). Improving students’ critical thinking: Empirical support for explicit instructions combined with practice. Applied Cognitive Psychology, 28 , 518–530. https://doi.org/10.1002/acp.3025

Heijltjes, A., Van Gog, T., Leppink, J., & Paas, F. (2015). Unraveling the effects of critical thinking instructions, practice, and self-explanation on students’ reasoning performance. Instructional Science, 43 , 487–506. https://doi.org/10.1002/acp.3025

Huber, C. R., & Kuncel, N. R. (2016). Does college teach critical thinking? A meta-analysis. Review of Educational Research, 86 , 431–468. https://doi.org/10.3102/0034654315605917

Ibiapina, C., Mamede, S., Moura, A., Elói-Santos, S., & van Gog, T. (2014). Effects of free, cued and modelled reflection on medical students’ diagnostic competence. Medical Education, 48 , 796–805. https://doi.org/10.1111/medu.12435

Isotani, S., Adams, D., Mayer, R. E., Durkin, K., Rittle-Johnson, B., & McLaren, B. M. (2011). Can erroneous examples help middle-school students learn decimals? In Proceedings of the sixth European conference on technology enhanced learning: Towards ubiquitous learning (EC-TEL-2011) .

Kalyuga, S. (2011). Cognitive load theory: How many types of load does it really need? Educational Psychology Review, 23 , 1–19. https://doi.org/10.1007/s10648-010-9150-7

Kassin, S. M., Dror, I. E., & Kukucka, J. (2013). The forensic confirmation bias: Problems, perspectives, and proposed solutions. Journal of Applied Research in Memory and Cognition, 2 , 42–52. https://doi.org/10.1016/j.jarmac.2013.01.001 .

Kawasaki, M. (2010). Learning to solve mathematics problems: The impact of incorrect solutions in fifth grade peers’ presentations. Japanese Journal of Developmental Psychology, 21 , 12–22.

Ku, K. Y. L. (2009). Assessing students’ critical thinking performance: Urging for measurements using multi-response format. Thinking Skills and Creativity, 4 , 70–76. https://doi.org/10.1016/j.tsc.2009.02.001

Lai, E. R. (2011). Critical thinking: A literature review. Pearson’s Research Reports, 6 , 40–41.

LeBel, E. P., & Paunonen, S. V. (2011). Sexy but often unreliable: The impact of unreliability on the replicability of experimental findings with implicit measures. Personality and Social Psychology Bulletin, 37 , 570–583. https://doi.org/10.1177/0146167211400619

Leppa, C. J. (1997). Standardized measures of critical thinking: Experience with the California Critical Thinking Tests. Nurse Education, 22 , 29–33.

Leppink, J., Paas, F., Van der Vleuten, C. P., Van Gog, T., & Van Merriënboer, J. J. (2013). Development of an instrument for measuring different types of cognitive load. Behavior Research Methods, 45 , 1058–1072. https://doi.org/10.3758/s13428-013-0334-1

Lesterhuis, M., Verhavert, S., Coertjens, L., Donche, V., & De Mayer, S. (2017). Comparative judgement as a promising alternative to score competences. In E. Cano & G. Ion (Eds.), Innovative practices for higher education assessment and measurement (pp. 119–138). IGI Global. https://doi.org/10.4018/978-1-5225-0531-0.ch007

Chapter   Google Scholar  

Liu, O. L., Frankel, L., & Roohr, K. C. (2014). Assessing critical thinking in higher education: Current state and directions for next-generation assessment. ETS Research Report Series, 2014 , 1–23. https://doi.org/10.1002/ets2.12009

Loibl, K., & Leuders, T. (2018). Errors during exploration and consolidation—The effectiveness of productive failure as sequentially guided discovery learning. Journal für Mathematik-Didaktik, 39 , 69–96. https://doi.org/10.1007/s13138-018-0130-7

Loibl, K., & Leuders, T. (2019). How to make failure productive: Fostering learning from errors through elaboration prompts. Learning and Instruction, 62 , 1–10. https://doi.org/10.1016/j.learninstruc.2019.03.002

Loo, R., & Thorpe, K. (1999). A psychometric investigation of scores on the Watson-Glaser critical thinking appraisal new forms. Educational and Psychological Measurement, 59 , 995–1003. https://doi.org/10.1177/00131649921970305

Markovits, H., & Nantel, G. (1989). The belief-bias effect in the production and evaluation of logical conclusions. Memory & Cognition, 17 , 11–17. https://doi.org/10.3758/BF03199552

Marzano, R. J., Pickering, D., & McTighe, J. (1993). Assessing student outcomes: Performance assessment using the Dimensions of Learning Model . Association for Supervision and Curriculum Development.

McLaren, B. M., Adams, D. M., & Mayer, R. E. (2015). Delayed learning effects with erroneous examples: A study of learning decimals with a web-based tutor. International Journal of Artificial Intelligence in Education, 25 , 520–542. https://doi.org/10.1007/s40593-015-0064-x

McLaren, B. M., Van Gog, T., Ganoe, C., Karabinos, M., & Yaron, D. (2016). The efficiency of worked examples compared to erroneous examples, tutored problem solving, and problem solving in computer-based learning environments. Computers in Human Behavior, 55 , 87–99. https://doi.org/10.1016/j.chb.2015.08.038

Moore, T. (2004). The critical thinking debate: How general are general thinking skills? Higher Education Research & Development, 23 , 3–18. https://doi.org/10.1080/0729436032000168469

Newstead, S. E., Pollard, P., Evans, J. St. B. T., & Allen, J. L. (1992). The source of belief bias effects in syllogistic reasoning. Cognition, 45 , 257–284. https://doi.org/10.1016/0010-0277(92)90019-E

Nievelstein, F., Van Gog, T., Van Dijck, G., & Boshuizen, H. P. (2013). The worked example and expertise reversal effect in less structured tasks: Learning to reason about legal cases. Contemporary Educational Psychology, 38 , 118–125. https://doi.org/10.1007/s11251-008-9076-3

Niu, L., Behar-Horenstein, L. S., & Garvan, C. W. (2013). Do instructional interventions influence college students’ critical thinking skills? A meta-analysis. Educational Research Review, 9 , 114–128. https://doi.org/10.1016/j.edurev.2012.12.002

Nunally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd ed.). McGraw-Hill.

Osborne, R. J., & Wittrock, M. C. (1983). Learning science: A generative process. Science Education, 67 , 489–508. https://doi.org/10.1002/sce.3730670406

Paas, F. (1992). Training strategies for attaining transfer or problem solving skills in statistics: A cognitive-load approach. Journal of Educational Psychology, 84 , 429–434. https://doi.org/10.1037/0022-0663.84.4.429

Paas, F., Renkl, A., & Sweller, J. (2003a). Cognitive load theory and instructional design: Recent developments. Educational Psychologist, 38 , 1–4. https://doi.org/10.1207/S15326985EP3801_1

Paas, F., Tuovinen, J. E., Tabbers, H., & Van Gerven, P. W. (2003b). Cognitive load measurement as a means to advance cognitive load theory. Educational Psychologist, 38 , 63–71. https://doi.org/10.1207/S15326985EP3801_8

Pascarella, E. T., Blaich, C., Martin, G. L., & Hanson, J. M. (2011). How robust are the findings of Academically Adrift? Change: The Magazine of Higher Learning, 43 , 20–24. https://doi.org/10.1080/00091383.2011.568898

Perkins, D. N., & Salomon, G. (1992). Transfer of learning. In T. Husen & T. N. Postelwhite (Eds.), The international encyclopedia of educational (2nd ed., Vol. 11, pp. 6452–6457). Pergamon Press.

Perkins, D. N., Jay, E., & Tishman, S. (1993). Beyond abilities: A dispositional theory of thinking. MerrillPalmer Quarterly, 39 , 1–21.

Renkl, A. (1999). Learning mathematics from worked-out examples: Analyzing and fostering self-explanations. European Journal of Psychology of Education, 14 , 477–488. https://doi.org/10.1007/BF03172974

Renkl, A. (2014). Toward an instructionally oriented theory of example-based learning. Cognitive Science, 38 , 1–37. https://doi.org/10.1111/cogs.12086

Renkl, A., & Atkinson, R. K. (2010). Learning from worked-out examples and problem solving. In J. Plass, R. Moreno, & R. Brünken (Eds.), Cognitive load theory and research in educational psychology (pp. 89–108). Cambridge University Press.

Renkl, A., & Eitel, A. (2019). Self-explaining: Learning about principles and their application. In J. Dunlosky & K. Rawson (Eds.), The Cambridge handbook of cognition and education (pp. 528–549). Cambridge University Press.

Renkl, A., Hilbert, T., & Schworm, S. (2009). Example-based learning in heuristic domains: A cognitive load theory account. Educational Psychology Review, 21 , 67–78. https://doi.org/10.1007/s10648-008-9093-4

Ritchhart, R., & Perkins, D. N. (2005). Learning to think: The challenges of teaching thinking. In K. J. Holyoak & R. G. Morrison (Eds.), The Cambridge handbook of thinking and reasoning (pp. 775–802). Cambridge University Press.

Rittle-Johnson, B., Star, J. R., & Durkin, K. (2009). The importance of prior knowledge when comparing examples: Influences on conceptual and procedural knowledge of equation solving. Journal of Educational Psychology, 101, 836–852. https://doi.org/10.1037/a0016026 .

Roelle, J., & Berthold, K. (2015). Effects of comparing contrasting cases on learning from subsequent explanations. Cognition and Instruction, 33 , 199–225. https://doi.org/10.1080/07370008.2015.1063636

Rogers, W. T., & Hopkins, K. D. (1988). Power estimates in the presence of a covariate and measurement error. Educational and Psychological Measurement, 48 , 647–656. https://doi.org/10.1177/0013164488483008

Salomon, G., & Perkins, D. N. (1989). Rocky roads to transfer: Rethinking mechanism of a neglected phenomenon. Educational Psychologist, 24 , 113–142. https://doi.org/10.1207/s15326985ep2402_1 .

Schmeck, A., Opfermann, M., Van Gog, T., Paas, F., & Leutner, D. (2015). Measuring cognitive load with subjective rating scales during problem solving: Differences between immediate and delayed ratings. Instructional Science, 43 , 93–114. https://doi.org/10.1007/s11251-014-9328-3

Schworm, S., & Renkl, A. (2007). Learning argumentation skills through the use of prompts for self-explaining examples. Journal of Educational Psychology, 99, 285–296. https://doi.org/10.1037/0022-0663.99.2.285

Segall, D. O. (1994). The reliability of linearly equated tests. Psychometrika, 59 , 361–375.

Siegler, R. S. (2002). Microgenetic studies of self-explanations. In N. Grannot & J. Parziale (Eds.), Microdevelopment: Transition processs in development and learning (pp. 31–58). Cambridge University Press.

Skulmowski, A., & Rey, G. D. (2017). Measuring cognitive load in embodied learning settings. Frontiers in Psychology, 8 , 1191. https://doi.org/10.3389/fpsyg.2017.01191

Stanovich, K. E. (2011). Rationality and the reflective mind . Oxford University Press.

Stanovich, K. E., & West, R. F. (2000). Individual differences in reasoning: Implications for the rationality debate? Behavioral and Brain Sciences, 23 , 645–665. https://doi.org/10.1017/S0140525X00003435

Stanovich, K. E., West, R. K., & Toplak, M. E. (2016). The rationality quotient: Toward a test of rational thinking . MIT Press.

Stark, R., Kopp, V., & Fischer, M. R. (2011). Case-based learning with worked examples in complex domains: Two experimental studies in undergraduate medical education. Learning and Instruction, 21 , 22–33. https://doi.org/10.1016/j.learninstruc.2009.10.001

Sweller, J. (1988). Cognitive load during problem solving: Effects on learning. Cognitive Science, 12 , 257–285. https://doi.org/10.1207/s15516709cog1202_4

Sweller, J., Van Merriënboer, J. J., & Paas, F. G. (1998). Cognitive architecture and instructional design. Educational Psychology Review, 10 , 251–296.

Sweller, J., Ayres, P., & Kalyuga, S. (Eds.). (2011). Measuring cognitive load. In Cognitive load theory (pp. 71–85). Springer.

Tiruneh, D. T., Verburgh, A., & Elen, J. (2014). Effectiveness of critical thinking instruction in higher education: A systematic review of intervention studies. Higher Education Studies, 4 , 1–17. https://doi.org/10.5539/hes.v4n1p1

Tiruneh, D. T., Weldeslassie, A. G., Kassa, A., Tefera, Z., De Cock, M., & Elen, J. (2016). Systematic design of a learning environment for domain-specific and domain-general critical thinking skills. Educational Technology Research and Development, 64 , 481–505. https://doi.org/10.1007/s11423-015-9417-2

Trauzettel-Klosinski, S., & Dietz, K. (2012). Standardized assessment of reading performance: The new International Reading Speed Texts IReST. Investigative Ophthalmology & Visual Science, 53 , 5452–5461. https://doi.org/10.1167/iovs.11-8284 .

Tversky, A., & Kahneman, D. (1974). Judgment under uncertainty: Heuristics and biases. Science, 185 , 1124–1131. https://doi.org/10.1126/science.185.4157.1124

Tversky, A., & Kahneman, D. (1983). Extensional versus intuitive reasoning: The conjunction fallacy in probability judgment. Psychological Review , 90, 293–315. https://psycnet.apa.org . https://doi.org/10.1037/0033-295X.90.4.293

Van den Broek, P., & Kendeou, P. (2008). Cognitive processes in comprehension of science texts: The role of co-activation in confronting misconceptions. Applied Cognitive Psychology, 22 , 335–351. https://doi.org/10.1002/acp.1418

Van Gelder, T. V. (2005). Teaching critical thinking: Some lessons from cognitive science. College Teaching, 53 , 41–48. https://doi.org/10.3200/CTCH.53.1.41-48

Van Gog, T., & Paas, F. (2008). Instructional efficiency: Revisiting the original construct in educational research. Educational Psychologist, 43 , 16–26. https://doi.org/10.1080/00461520701756248

Van Gog, T., Paas, F., & Van Merriënboer, J. J. (2004). Process-oriented worked examples: Improving transfer performance through enhanced understanding. Instructional Science, 32 , 83–98. https://doi.org/10.1023/B:TRUC.0000021810.70784.b0

Van Gog, T., Kirschner, F., Kester, L., & Paas, F. (2012). Timing and frequency of mental effort measurement: Evidence in favour of repeated measures. Applied Cognitive Psychology, 26 , 833–839. https://doi.org/10.1002/acp.2883

Van Gog, T., Rummel, N., & Renkl, A. (2019). Learning how to solve problems by studying examples. In J. Dunlosky & K. Rawson (Eds.), The Cambridge handbook of cognition and education (pp. 183–208). Cambridge University Press.

VanLehn, K. (1999). Rule-learning events in the acquisition of a complex skill: An evaluation of cascade. The Journal of the Learning Sciences, 8 , 71–125. https://doi.org/10.1207/s15327809jls0801_3

Van Loon-Hillen, N. H., Van Gog, T., & Brand-Gruwel, S. (2012). Effects of worked examples in a primary school mathematics curriculum. Interactive Learning Environments, 20 , 89–99. https://doi.org/10.1080/10494821003755510

Van Peppen, L. M., Verkoeijen P. P. J. L., Heijltjes, A. E. G., Janssen, E. M., Koopmans, D., & Van Gog, T. (2018). Effects of self-explaining on learning and transfer of critical thinking skills. Frontiers in Education, 3 , 100. https://doi.org/10.3389/feduc.2018.00100 .

Van Peppen, L. M., Verkoeijen, P. P., Kolenbrander, S. V., Heijltjes, A. E., Janssen, E. M., & van Gog, T. (2021). Learning to avoid biased reasoning: Effects of interleaved practice and worked examples. Journal of Cognitive Psychology, 33 , 304–326. https://doi.org/10.1080/20445911.2021.1890092 .

West, R. F., Toplak, M. E., & Stanovich, K. E. (2008). Heuristics and biases as measures of critical thinking: Associations with cognitive ability and thinking dispositions. Journal of Educational Psychology, 100 , 930–941. https://doi.org/10.1037/a0012842

Wittrock, M. C. (1974). Learning as a generative process. Educational Psychologist, 11 , 87–95. https://doi.org/10.1080/00461527409529129

Wittrock, M. C. (1990). Generative processes of comprehension. Educational Psychologist, 24 , 345–376. https://doi.org/10.1207/s15326985ep2404_2

Wittrock, M. C. (1992). Generative learning processes of the brain. Educational Psychologist, 27 , 531–541. https://doi.org/10.1207/s15326985ep2704_8

Wittrock, M. C. (2010). Learning as a generative process. Educational Psychologist, 45 , 40–45. https://doi.org/10.1080/00461520903433554

Download references

Acknowledgements

This research was funded by The Netherlands Organisation for Scientific Research (Project Number 409-15-203). The authors would like to thank Stefan V. Kolenbrander for his help with running this study and Esther Stoop and Marjolein Looijen for their assistance with coding the data.

This research was funded by The Netherlands Organisation for Scientific Research (Project Number 409-15-203).

Author information

Lara M. van Peppen

Present address: Institute of Medical Education Research, Erasmus University Medical Center Rotterdam, Doctor Molewaterplein 40, 3051 GD, Rotterdam, The Netherlands

Authors and Affiliations

Department of Psychology, Education and Child Studies, Erasmus University Rotterdam, Burgemeester Oudlaan 50, 3062 PA, Rotterdam, The Netherlands

Lara M. van Peppen & Peter P. J. L. Verkoeijen

Learning and Innovation Center, Avans University of Applied Sciences, Hogeschoollaan 1, 4818 CR, Breda, The Netherlands

Peter P. J. L. Verkoeijen & Anita E. G. Heijltjes

Department of Education, Utrecht University, Heidelberglaan 1, 3584 CS, Utrecht, The Netherlands

Eva M. Janssen & Tamara van Gog

You can also search for this author in PubMed   Google Scholar

Contributions

LP, PV, AH, and TG contributed to the conception and design of the study. LP and EM prepared the materials. LP collected the data, organized the database, and performed the statistical analyses. LP, PV, and TG interpreted the data. LP wrote the original draft of the manuscript and PV, AH, EM, and TG provided critical revision of the manuscript. All authors read and approved the submitted version of the manuscript.

Corresponding author

Correspondence to Lara M. van Peppen .

Ethics declarations

Conflict of interest.

The authors declare that there is no conflict of interest.

Ethical approval

In accordance with the guidelines of the ethical committee at the Department of Psychology, Education and Child studies, Erasmus University Rotterdam, the study was exempt from ethical approval procedures because the materials and procedures were not invasive. All subjects gave written informed consent prior to participating in this study.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (DOCX 227 kb)

Rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

van Peppen, L.M., Verkoeijen, P.P.J.L., Heijltjes, A.E.G. et al. Enhancing students’ critical thinking skills: is comparing correct and erroneous examples beneficial?. Instr Sci 49 , 747–777 (2021). https://doi.org/10.1007/s11251-021-09559-0

Download citation

Received : 14 February 2020

Accepted : 11 August 2021

Published : 26 September 2021

Issue Date : December 2021

DOI : https://doi.org/10.1007/s11251-021-09559-0

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Critical thinking
• Heuristics and biases
• Transfer of learning
• Example-based learning
• Erroneous examples
• Contrasting examples
• Find a journal
• Publish with us
• Track your research

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Front Psychol

Metacognitive Strategies and Development of Critical Thinking in Higher Education

Silvia f. rivas.

1 Departamento de Psicología Básica, Psicobiología y Metodología de CC, Facultad de Psicología, Universidad de Salamanca, Salamanca, Spain

Carlos Saiz

Carlos ossa.

2 Departamento de Ciencias de la Educación, Facultad de Educación y Humanidades, Universidad del Bío-Bío, Sede Chillán, Chile

Associated Data

The original contributions presented in the study are included in the article/supplementary material; further inquiries can be directed to the corresponding author.

More and more often, we hear that higher education should foment critical thinking. The new skills focus for university teaching grants a central role to critical thinking in new study plans; however, using these skills well requires a certain degree of conscientiousness and its regulation. Metacognition therefore plays a crucial role in developing critical thinking and consists of a person being aware of their own thinking processes in order to improve them for better knowledge acquisition. Critical thinking depends on these metacognitive mechanisms functioning well, being conscious of the processes, actions, and emotions in play, and thereby having the chance to understand what has not been done well and correcting it. Even when there is evidence of the relation between metacognitive processes and critical thinking, there are still few initiatives which seek to clarify which process determines which other one, or whether there is interdependence between both. What we present in this study is therefore an intervention proposal to develop critical thinking and meta knowledge skills. In this context, Problem-Based Learning is a useful tool to develop these skills in higher education. The ARDESOS-DIAPROVE program seeks to foment critical thinking via metacognition and Problem-Based Learning methodology. It is known that learning quality improves when students apply metacognition; it is also known that effective problem-solving depends not only on critical thinking, but also on the skill of realization, and of cognitive and non-cognitive regulation. The study presented hereinafter therefore has the fundamental objective of showing whether instruction in critical thinking (ARDESOS-DIAPROVE) influences students’ metacognitive processes. One consequence of this is that critical thinking improves with the use of metacognition. The sample was comprised of first-year psychology students at Public University of the North of Spain who were undergoing the aforementioned program; PENCRISAL was used to evaluate critical thinking skills and the Metacognitive Activities Inventory (MAI) for evaluating metacognition. We expected an increase in critical thinking scores and metacognition following this intervention. As a conclusion, we indicate actions to incentivize metacognitive work among participants, both individually via reflective questions and decision diagrams, and at the interactional level with dialogues and reflective debates which strengthen critical thinking.

Introduction

One of the principal objectives which education must cover is helping our students become autonomous and effective. Students’ ability to use strategies which help them direct their motivation toward action in the direction of the meta-proposal is a central aspect to keep at the front of our minds when considering education. This is where metacognition comes into play—knowledge about knowledge itself, a component which is in charge of directing, monitoring, regulating, organizing, and planning our skills in a helpful way, once these have come into operation. Metacognition helps form autonomous students, increasing consciousness about their own cognitive processes and their self-regulation so that they can regulate their own learning and transfer it to any area of their lives. As we see, it is a conscious activity of high-level thinking which allows us to look into and reflect upon how we learn and to control our own strategies and learning processes. We must therefore approach a problem which is increasing in our time, that of learning and knowledge from the perspective of active participation by students. To achieve these objectives of “learning to learn” we must use adequate cognitive learning strategies, among which we can highlight those oriented toward self-learning, developing metacognitive strategies, and critical thinking.

Metacognition is one of the research areas, which has contributed the most to the formation of the new conceptions of learning and teaching. In this sense, it has advanced within the constructivist conceptions of learning, which have attributed an increasing role to student consciousness and to the regulation which they exercise over their own learning ( Glaser, 1994 ).

Metacognition was initially introduced by John Flavell in the early 1970s. He affirmed that metacognition, on one side, refers to “the knowledge which one has about his own cognitive processes products, or any other matter related with them” and on the other, “to the active supervision and consequent regulation and organization of these processes in relation with the objects or cognitive data upon which they act” ( Flavell, 1976 ; p. 232). Based on this, we can differentiate two components of metacognition: one of a declarative nature, which is metacognitive knowledge, referring to knowledge of the person and the task, and another of a procedural nature, which is metacognitive control or self-regulated learning, which is always directed toward a goal and controlled by the learner.

Different authors have pointed out that metacognition presents these areas of thought or skills, aimed knowledge or toward the regulation of thought and action, mainly proposing a binary organization in which attentional processes are oriented, on occasions, toward an object or subject, and the other hand, toward to interact with objects and/or subjects ( Drigas and Mitsea, 2021 ). However, it is possible to understand metacognition from another approach that establishes more levels of use of metacognitive thinking to promote knowledge, awareness, and intelligence, known as the eight pillars of metacognition model ( Drigas and Mitsea, 2020 ). These pillars allow thought to promote the use of deep knowledge, cognitive processes, self-regulation, functional adaptation to society, pattern recognition and operations, and even meaningful memorization ( Drigas and Mitsea, 2020 ).

In addition to the above, Drigas and Mitsea’s model establishes different levels where metacognition could be used, in a complex sequence from stimuli to transcendental ideas, in which each of the pillars could manifest a different facet of the process metacognitive, thus establishing a dialectical and integrative approach to learning and knowledge, allowing it to be understood as an evolutionary and complex process in stages ( Drigas and Mitsea, 2021 ).

All this clarifies the importance of and need for metacognition, not only in education but also in our modern society, since this need to “teach how to learn” and the capacity to “learn how to learn” in order to achieve autonomous learning and transfer it to any area of our lives will let us face problems more successfully. This becomes a relevant challenge, especially today where it is required to have a broad view regarding reflection and consciousness, and to transcend simplistic and reductionist models that seek to center the problem of knowledge only around the neurobiological or the phenomenological scope ( Sattin et al., 2021 ).

Critical thinking depends largely on these mechanisms functioning well and being conscious of the processes used, since this gives us the opportunity to understand what has not been done well and correct it in the future. Consciousness for critical thinking would imply a continuous process of reuse of thought, in escalations that allow thinking to be oriented both toward the objects of the world and toward the subjective interior, allowing to determine the ideas that give greater security to the person, and in that perspective, the metacognitive process, represents this use of Awareness, also allowing the generation of an identity of knowing being ( Drigas and Mitsea, 2021 ).

We know that thinking critically involves reasoning and deciding to effectively solve a problem or reach goals. However, effective use of these skills requires a certain degree of consciousness and regulation of them. The ARDESOS-DIAPROVE program seeks precisely to foment critical thinking, in part, via metacognition ( Saiz and Rivas, 2011 , 2012 , 2016 ).

However, it is not only centered on developing cognitive components, as this would be an important limitation. Since the 1990s, it has been known that non-cognitive components play a crucial role in developing critical thinking. However, there are few studies focusing on this relation. This intervention therefore considers both dimensions, where metacognitive processes play an essential role by providing evaluation and control mechanisms over the cognitive dimension.

Metacognition and Critical Thinking

Critical Thinking is a concept without a firm consensus, as there have been and still are varying conceptions regarding it. Its nature is so complex that it is hard to synthesize all its aspects in a single definition. While there are numerous conceptions about critical thinking, it is necessary to be precise about which definition we will use. We understand that “ critical thinking is a knowledge-seeking process via reasoning skills to solve problems and make decisions which allows us to more effectively achieve our desired results” ( Saiz and Rivas, 2008 , p. 131). Thinking effectively is desirable in all areas of individual and collective action. Currently, the background of the present field of critical thinking is also based in argumentation. Reasoning is used as the fundamental basis for all activities labeled as thinking. In a way, thinking cannot easily be decoupled from reasoning, at least if our understanding of it is “deriving something from another thing.” Inference or judgment is what we essentially find behind the concept of thinking. The question, though, is whether it can be affirmed that thinking is only reasoning. Some defend this concept ( Johnson, 2008 ), while others believe the opposite, that solving problems and making decisions are activities which also form part of thinking processes ( Halpern, 2003 ; Halpern and Dunn, 2021 , 2022 ). To move forward in this sense, we will return to our previous definition. In that definition, we have specified intellectual activity with a goal intrinsic to all mental processes, namely, seeking knowledge. Achieving our ends depends not only on the intellectual dimension, as we may need our motor or perceptive activities, so it contributes little to affirm that critical thinking allows us to achieve our objectives as we can also achieve them by doing other activities. It is important for us to make an effort to identify the mental processes responsible for thinking and distinguish them from other things.

Normally, we think to solve our problems. This is the second important activity of thought. A problem can be solved by reasoning, but also by planning course of action or selecting the best strategy for the situation. Apart from reasoning, we must therefore also make decisions to resolve difficulties. Choosing is one of the most frequent and important activities which we do. Because of this, we prefer to give it the leading role it deserves in a definition of thinking. Solving problems demands multiple intellectual activities, including reasoning, deciding, planning, etc. The final characteristic goes beyond the mechanisms peculiar to inference. What can be seen at the moment of delineating what it means to think effectively is that concepts are grouped together which go beyond the nuclear ideas of what has to do with inferring or reasoning. The majority of theoreticians in the field ( APA, 1990 ; Ennis, 1996 ; Halpern, 1998 , 2003 ; Paul and Elder, 2001 ; Facione, 2011 ; Halpern and Dunn, 2021 , 2022 ) consider that, in order to carry out this type of thinking effectively, apart from having this skill set, the intervention of other types of components is necessary, such as metacognition and motivation. This is why we consider it necessary to speak about the components of critical thinking, as we can see in Figure 1 :

Components of critical thinking ( Saiz, 2020 ).

In the nature of thinking, there are two types of components: the cognitive and the non-cognitive. The former include perception, learning, and memory processes. Learning is any knowledge acquisition mechanism, the most important of which is thinking. The latter refer to motivation and interests (attitudes tend to be understood as dispositions, inclinations…something close to motives); with metacognition remaining as a process which shares cognitive and non-cognitive aspects as it incorporates aspects of both judgment (evaluation) and disposition (control/efficiency) about thoughts ( Azevedo, 2020 ; Shekhar and Rahnev, 2021 ). Both the cognitive and non-cognitive components are essential to improve critical thinking, as one component is incomplete without the other, that is, neither cognitive skills nor dispositions on their own suffice to train a person to think critically. In general, relations are bidirectional, although for didactic reasons only unidirectional relations appear in Figure 1 ( Rivas et al., 2017 ). This is because learning is a dynamic process which is subject to all types of influence. For instance, if a student is motivated, they will work more and better—or at least, this is what is hoped for. If they can achieve good test scores as well, it can be supposed that motivation is reinforced, so that they will continue existing behaviors in the same direction that is, working hard and well on their studies. This latter point appears to arise at least because of an adjustment between expectations and reality which the student achieves thanks to metacognition, which allows them to effectively attribute their achievements to their efforts ( Ugartetxea, 2001 ).

Metacognition, which is our interest in this paper, should also have bidirectional relations with critical thinking. Metacognition tends to be understood as the degree of consciousness which we have about our own mental processes and similar to the capacity for self-regulation, that is, planning and organization ( Mayor et al., 1993 ). We observe that these two ideas have very different natures. The former is simpler, being the degree of consciousness which we reach about an internal mechanism or process. The latter is a less precise idea, since everything which has to do with self-regulation is hard to differentiate from a way of understanding motivation, such as the entire tradition of intrinsic motivation and self-determination from Deci, his collaborators, and other authors of this focus (see, e.g., Deci and Ryan, 1985 ; Ryan and Deci, 2000 ). The important thing is to emphasize the executive dimension of metacognition, more than the degree of consciousness, for practical reasons. It can be expected that this dimension has a greater influence on the learning process than that of consciousness, although there is little doubt that we have to establish both as necessary and sufficient conditions. However, the data must speak in this regard. Due to all of this, and as we shall see hereinafter, the intervention designed incorporates both components to improve critical thinking skills.

We can observe, though, that the basic core of critical thinking continues to be topics related to skills, in our case, reasoning, problem-solving, and decision-making. The fact that we incorporate concepts of another nature, such as motivation, in a description of critical thinking is justified because it has been proven that, when speaking about critical thinking, the fact of centering solely on skills does not allow for fully gathering its complexity. The purpose of the schematic in Figure 2 is to provide conceptual clarity to the adjective “critical” in the expression critical thinking . If we understand critical to refer to effective , we should also consider that effectiveness is not, as previously mentioned, solely achieved with skills. They must be joined together with other mechanisms during different moments. Intellectual skills alone cannot achieve the effectiveness assumed within the term “critical.” First, for said skills to get underway, we must want to do so. Motivation therefore comes into play before skills and puts them into operation. For its part, metacognition allows us to take advantage of directing, organizing, and planning our skills and act once they have begun to work. Motivation thus activates our abilities, while metacognition lets them be more effective. The final objective should always be to gain proper knowledge of reality to resolve our problems.

Purpose of critical thinking ( Saiz, 2020 , p.27).

We consider that the fact of referring to components of critical thinking while differentiating the skills of motivation and metacognition aids with the conceptual clarification we seek. On one side, we specify the skills which we discuss, and on another, we mention which other components are related to, and even overlap with them. We must be conscious of how difficult it is to find “pure” mental processes. Planning a course of action, an essential trait of metacognition, demands reflection, prediction, choice, comparison, and evaluation… And this, evidently, is thinking. The different levels or dimensions of our mental activity must be related and integrated. Our aim is to be able to identify what is substantial in thinking to know what we are able to improve and evaluate.

It is widely known that for our personal and professional functioning, thinking is necessary and useful. When we want to change a situation or gain something, all our mental mechanisms go into motion. We perceive the situation, identify relevant aspects of the problem, analyze all the available information, and appraise everything we analyze. We make judgments about the most relevant matters, decide about the options or pathways for resolution, execute the plan, obtain results, evaluate the results, estimate whether we have achieved our purpose and, according to the level of satisfaction following this estimation, consider our course of action good, or not.

The topic we must pose now is what things are teachable. It is useful to specify that what is acquired is clearly cognitive and some of the non-cognitive, because motivation can be stimulated or promoted, but not taught. The concepts of knowledge and wisdom are its basis. Mental representation and knowledge only become wisdom when we can apply it to reality, when we take it out of our mind and adequately situate it in the world. For our teaching purposes, we only have to take a position about whether knowledge is what makes critical thinking develop, or vice versa. For us, skills must be directly taught, and dominion is secondary. Up to now, we have established the components of critical thinking, but these elements still have to be interrelated properly. What we normally find are skills or components placed side by side or overlapping, but not the ways in which they influence each other. Lipman (2003) may have developed the most complete theory of critical and creative thinking, along Paul and his group, in second place, with their universal thought structures ( Paul and Elder, 2006 ). However, a proposal for the relation between the elements is lacking.

To try to explain the relation between the components of thought, we will use Figure 2 as an aid.

The ultimate goal of critical thinking is change that is, passing from one state of wellbeing into a better state. This change is only the fruit of results, which must be the best. Effectiveness is simple achieving our goals in the best way possible. There are many possible results, but for our ends, there are always some which are better than others. Our position must be for effectiveness, the best response, the best solution. Reaching a goal is resolving or achieving something, and for this, we have mechanisms available which tell us which are the best course of action. Making decisions and solving problems are fundamental skills which are mutually interrelated. Decision strategies come before a solution. Choosing a course of action always comes before its execution, so it is easy to understand that decisions contribute to solutions.

Decisions must not come before reflection, although this often can and does happen. As we have already mentioned, the fundamental skills of critical thinking, in most cases, have been reduced to reasoning, and to a certain degree, this is justified. There is an entire important epistemological current behind this, within which the theory of argumentation makes no distinction, at least syntactically, between argumentation and explanation. However, for us this distinction is essential, especially in practice ( Saiz, 2020 ). We will only center on an essential difference for our purpose. Argumentation may have to do with values and realities, but explanation only has to do with the latter. We can argue about beliefs, convictions, and facts, but we can only explain realities. Faced with an explanation of reality, any argumentation would be secondary. Thus, explanation will always be the central skill in critical thinking.

The change which is sought is always expressed in reality. Problems always are manifested and resolved with actions, and these are always a reality. An argument about realities aids in explaining them. An argument about values upholds a belief or a conviction. However, beliefs always influence behavior; thus, indirectly, the argument winds up being about realities. One may argue, for example, only for or against the death penalty, and reach the conviction that it is good or bad and ultimately take a position for or against allowing it. This is why we say that deciding always comes before resolving; furthermore, resolution always means deciding about something in a particular direction—it always means choosing and taking an option; furthermore, deciding is often only from two possibilities, the better or that which is not better, or which is not as good. Decisions are made based on the best option possible of all those which can be presented. Resolution is a dichotomy. Since our basic end lies within reality, explanation must be constituted as the basic pillar to produce change. Argumentation must therefore be at the service of causality (explanation), and both must be in the service of solid decisions leading us to the best solution or change of situation. We now believe that the relation established in Figure 2 can be better understood. From this relation, we propose that thinking critically means reaching the best explanation for an event, phenomenon, or problem in order to know how to effectively resolve it ( Saiz, 2017 , p.19). This idea, to our judgment, is the best summary of the nature of critical thinking. It clarifies details and makes explicit the components of critical thinking.

Classroom Activities to Develop Metacognition

We will present a set of strategies to promote metacognitive work in the classroom in this section, aimed at improving critical thinking skills. These strategies can be applied both at the university level and the secondary school level; we will thus focus on these two levels, although metacognitive strategies can be worked on from an earlier age ( Jaramillo and Osses, 2012 ; Tamayo-Alzate et al., 2019 ) and some authors have indicated that psychological maturity has a greater impact on effectively achieving metacognition ( Sastre-Riba, 2012 ; García et al., 2016 ).

At the individual level, metacognition can be worked on via applying questions aimed at the relevant tasks which must be undertaken regarding a task (meta-knowledge questions), for example:

• Do I know how much I know about this subject?
• Do I have clear instructions and know what action is expected from me?
• How much time do I have?
• Am I covering the proper and necessary subjects, or is there anything important left out?
• How do I know that my work is right?
• Have I covered every point of the rubric for the work to gain a good grade or a sufficient level?

These reflective questions facilitate supervising knowledge level, resource use, and the final product achieved, so that the decisions taken for said activities are the best and excellent learning results are achieved.

Graphs or decision diagrams can also be used to aid in organizing these questions during the different phases of executing a task (planning, progress, and final evaluation), which is clearly linked with the knowledge and control processes of metacognition ( Mateos, 2001 ). These diagrams are more complex and elaborate strategies than the questions, but are effective when monitoring the steps considered in the activity ( Ossa et al., 2016 ). Decision diagrams begin from a question or task, detailing the principal steps to take, and associating an alternative (YES or NO) to each step, which leads to the next step whenever the decision is affirmative, or to improve or go further into the step taken if the decision is negative.

Finally, we can work on thinking aloud, a strategy which facilitates making the thoughts explicit and conscious, allowing us to monitor their knowledge, decisions, and actions to promote conscious planning, supervision and evaluation ( Ávila et al., 2017 ; Dahik et al., 2019 ). For example:

• While asking a question, the student thinks aloud: I am having problems with this part of the task, and I may have to ask the teacher to know whether I am right.

Thinking aloud can be done individually or in pairs, allowing for active monitoring of decisions and questions arising from cognitive and procedural work done by the student.

Apart from the preceding strategies, it is also possible to fortify metacognitive development via personal interactions based on dialogue between both the students themselves and between the teacher and individual students. One initial strategy, similar to thinking out loud in pairs, is reflective dialogue between teacher and student, a technique which allows for exchanging deep questions and answers, where the student becomes conscious of their knowledge and practice thanks to dialogical interventions by the teacher ( Urdaneta, 2014 ).

Reflective dialogue can also be done via reflective feedback implemented by the teacher for the students to learn by themselves about the positive and negative aspects of their performance on a task.

Finally, another activity based on dialogue and interaction is related to metacognitive argumentation ( Sánchez-Castaño et al., 2015 ), a strategy which uses argumentative resources to establish a valid argumentative structure to facilitate responding to a question or applying it to a debate. While argumentative analysis is based on logic and the search for solid reasons, these can have higher or lower confidence and reliability as a function of the data which they provide. Thus, if a reflective argumentative process is performed, via questioning reasons or identifying counterarguments, there is more depth and density in the argumentative structure, achieving greater confidence and validity.

We can note that metacognition development strategies are based on reflective capacity, which allow thought to repeatedly review information and decisions to consider, without immediately taking sides or being carried away by superficial or biased ideas or data. Critical thought benefits strongly from applying this reflective process, which guides both data management and cognitive process use. These strategies can also be developed in various formats (written, graphic, oral, individual, and dialogical), providing teachers a wide range of tools to strengthen learning and thinking.

Metacognitive Strategies to Improve Critical Thinking

In this section, we will describe the fundamental metacognitive strategies addressed in our critical thinking skills development program ARDESOS-DIAPROVE.

First, one of the active learning methodologies applied is Problem-Based Learning (PBL). This pedagogical strategy is student-centered and encourages autonomous and participative learning, orienting students toward more active and decisive learning. In PBL each situation must be approached as a problem-solving task, making it necessary to investigate, understand, interpret, reason, decide, and resolve. It is presented as a methodology which facilitates joint knowledge acquisition and skill learning. It is also good for working on daily problems via relevant situations, considerably reducing the distance between learning context and personal/professional life and aiding the connection between theory and practice, which promote the highly desired transference. It favors organization and the capacity to decide about problem-solving, which also improves performance and knowledge about the students’ own learning processes. Because of all this, this methodology aids in reflection and analysis processes, which in turn promotes metacognitive skill development.

The procedure which we carried out in the classroom with all the activities is based on the philosophy of gradual learning control transference ( Mateos, 2001 ). During instruction, the teacher takes on the role of model and guide for students’ cognitive and metacognitive activity, gradually bringing them into participating in an increasing level of competency, and slowly withdrawing support in order to attain control over the students’ learning process. This methodology develops in four phases: (1) explicit instruction, where the teacher directly explains the skills which will be worked on; (2) guided practice, where the teacher acts as a collaborator to guide and aid students in self-regulation; and (3) cooperative practice, where cooperative group work facilitates interaction with a peer group collaborating to resolve the problem. By explaining, elaborating, and justifying their own points of view and alternative solutions, greater consciousness, reflection, and control over their own cognitive processes is promoted. Finally, (4) individual practice is what allows students to place their learning into practice in individual evaluation tasks.

Regarding the tasks, it is important to highlight that the activities must be aimed not only at acquiring declarative knowledge, but also at procedural knowledge. The objective of practical tasks, apart from developing fundamental knowledge, is to develop CT skills among students in both comprehension and expression in order to favor their learning and its transference. The problems used must be common situations, close to our students’ reality. The important thing in our task of teaching critical thinking is its usefulness to our students, which can only be achieved during application since we only know something when we are capable of applying it. We are not interested in students merely developing critical skills; they must also be able to generalize their intellectual skills, for which they must perceive them as useful in order to want to acquire them. Finally, they will have to actively participate to apply them to solving problems. Furthermore, if we study the different ways of reasoning without context, via overly academic problems, their application to the personal sphere becomes impossible, leading them to be considered hardly useful. This makes it important to contextualize skills within everyday problems or situations which help us get students to use them regularly and understand their usefulness.

Reflecting on how one carries things out in practice and analyzing mistakes are ways to encourage success and autonomy in learning. These self-regulation strategies are the properly metacognitive part of our study. The teacher has various resources to increase these strategies, particularly feedback oriented toward task resolution. Similarly, one of the most effective instruments to achieve it is using rubrics, a central tool for our methodology. These guides, used in student performance evaluations, describe the specific characteristics of a task at various performance levels, in order to clarify expectations for students’ work, evaluate their execution, and facilitate feedback. This type of technique also allows students to direct their own activity. We use them with this double goal in mind; on the one hand, they aid students in carrying out tasks, since they help divide the complex tasks they have to do into simpler jobs, and on the other, they help evaluate the task. Rubrics guide students in the skills and knowledge they need to acquire as well as facilitating self-evaluation, thereby favoring responsibility in their learning. Task rubrics are also the guide for evaluation which teachers carry out in classrooms, where they specify, review, and correctly resolve the tasks which students do according to the rubric criteria. Providing complete feedback to students is a crucial aspect for the learning process. Thus, in all sessions time is dedicated to carrying it out. This is what will allow them to move ahead in self-regulated skill learning.

According to what we have seen, there is a wide range of positions when it comes to defining critical thinking. However, there is consensus in the fact that critical thinking involves cognitive, attitudinal, and metacognitive components, which together favor proper performance in critical thinking ( Ennis, 1987 ; Facione, 1990 ). This important relation between metacognition and critical thinking has been widely studied in the literature ( Berardi-Coletta et al., 1995 ; Antonietti et al., 2000 ; Kuhn and Dean, 2004 ; Black, 2005 ; Coutinho et al., 2005 ; Orion and Kali, 2005 ; Schroyens, 2005 ; Akama, 2006 ; Choy and Cheah, 2009 ; Magno, 2010 ; Arslan, 2014 ) although not always in an applied way. Field studies indicate the existence of relations between teaching metacognitive strategies and progress in students’ higher-order thinking processes ( Schraw, 1998 ; Kramarski et al., 2002 ; Van der Stel and Veenman, 2010 ). Metacognition is thus considered one of the most relevant predictors of achieving a complex higher-order thought process.

Along the same lines, different studies show the importance of developing metacognitive skills among students as it is related not only with developing critical thinking, but also with academic achievement and self-regulated learning ( Klimenko and Alvares, 2009 ; Magno, 2010 ; Doganay and Demir, 2011 ; Özsoy, 2011 ). Klimenko and Alvares (2009) indicated that one way for students to acquire necessary tools to encourage autonomous learning is making cognitive and metacognitive strategies explicit and well-used and that teachers’ role is to be mediators and guides. Inspite of this evidence, there is less research about the use of metacognitive strategies in encouraging critical thinking. The principal reason is probably that it is methodologically difficult to gather direct data about active metacognitive processes which are complex by nature. Self-reporting is also still very common in metacognition evaluation, and there are few studies which have included objective measurements aiding in methodological precision for evaluating metacognition.

However, in recent years, greater importance has been assigned to teaching metacognitive skills in the educational system, as they aid students in developing higher-order thinking processes and improving their academic success ( Flavell, 2004 ; Larkin, 2009 ). Because of this, classrooms have seen teaching and learning strategies emphasizing metacognitive knowledge and regulation. Returning to our objective, which is to improve critical thinking via the ARDESOS-DIAPROVE program, we have achieved our goal in an acceptable way ( Saiz and Rivas, 2011 , 2012 , 2016 ).

However, we need to know which specific factors contribute to this improvement. We have covered significant ground through different studies, one of which we present here. In this one, we attempt to find out the role of metacognition in critical thinking. This is the central objective of the study. Our program includes motivational and metacognitive variables. Therefore, we seek to find out whether metacognition improves after this instruction program focused on metacognition. Therefore, our hypothesis is simple: we expect that the lesson will improve our students’ metacognition. The idea is to know whether applying metacognition helps us achieve improved critical thinking and whether after this change metaknowledge itself improves. In other words, improved critical thinking performance will make us think better about thinking processes themselves. If this can be improved, we can expect that in the future it will have a greater influence on critical thinking. The idea is to be able to demonstrate that applying specifically metacognitive techniques, the processes themselves will subsequently improve in quality and therefore contribute better volume and quality to reasoning tasks, decision-making and problem-solving.

Materials and Methods

Participants.

In the present study, we used a sample of 89 students in a first-year psychology course at Public University of the North of Spain. 82% (73) were women, and the other 18% (16) were men. Participants’ median age was 18.93 ( SD 1.744).

Instruments

Critical thinking test.

To measure critical thinking skills, we applied the PENCRISAL test ( Saiz and Rivas, 2008 ; Rivas and Saiz, 2012 ). The PENCRISAL is a battery consisting of 35 production problem situations with an open-answer format, composed of five factors: Deductive Reasoning , Inductive Reasoning , Practical Reasoning , Decision-Making , and Problem-Solving , with seven items per factor. Items for each factor gather the most representative structures of fundamental critical thinking skills.

The items’ format is open, so that the person has to answer a concrete question, adding a justification for the reasons behind their answer. Because of this, there are standardized correction criteria assigning values between 0 and 2 points as a function of answer quality. This test offers us a total score of critical thinking skills and another five scores referring to the five factors. The value range is located between 0 and 72 points as a maximum limit for total test scoring, and between 0 and 14 for each of the five scales. The reliability measures present adequate precision levels according to the scoring procedures, with the lowest Cronbach’s alpha values at 0.632, and the test–retest correlation at 0.786 ( Rivas and Saiz, 2012 ). PENCRISAL administration was done over the Internet via the evaluation platform SelectSurvey.NET V5: http://24.selectsurvey.net/pensamiento-critico/Login.aspx .

Metacognitive Skill Inventory

Metacognitive skill evaluation was done via the metacognitive awareness inventory from Schraw and Dennison (1994) (MAI; Huertas Bustos et al., 2014 ). This questionnaire has 52 Likert scale-type items with five points. The items are distributed in two general dimensions: cognitive knowledge (C) and regulation of cognition (R). This provides ample coverage for the two aforementioned ideas about metaknowledge. There are also eight defined subcategories within each general dimension. For C, these are: declarative knowledge (DK), procedural knowledge (PK), and conditional knowledge (CK). In R, we find: organization (O), monitoring (M), and evaluation (E). This instrument comprehensively, and fairly clearly, brings together essential aspects of metacognition. On one side, there is the level of consciousness, containing types of knowledge—declarative, procedural, and strategic. On the other, it considers everything important in the processes of self-regulation, planning, organization, direction or control (monitoring), adjustment (troubleshooting), and considering the results achieved (evaluation). It provides a very complete vision of everything important in this dimension. Cronbach’s alpha for this instrument is 0.94, showing good internal consistency.

Intervention Program

As previously mentioned, in this study, we applied the third version of the ARDESOS_DIAPROVE program ( Saiz and Rivas, 2016 ; Saiz, 2020 ), with the objective of improving thinking skills. This program is centered on directly teaching the skills which we consider essential to develop critical thinking and for proper performance in our daily affairs. For this, we must use reasoning and good problem-solving and decision-making strategies, with one of the most fundamental parts of our intervention being the use of everyday situations to develop these abilities.

DIAPROVE methodology incorporates three new and essential aspects: developing observation, the combined use of facts and deduction, and effective management of de-confirmation procedures, or discarding hypotheses. These are the foundation of our teaching, which requires specific teaching–learning techniques.

The intervention took place over 16 weeks and is designed to be applied in classrooms over a timeframe of 55–60 h. The program is applied in classes of around 30–35 students divided into groups of four for classwork in collaborative groups, and organized into six activity blocks: (1) nature of critical thinking, (2) problem-solving and effectiveness, (3) explanation and causality, (4) deduction and explanation, (5) argumentation and deduction, and (6) problem-solving and decision-making. These blocks are assembled maintaining homogeneity, facilitating a global integrated skill focus which helps form comprehension and use of the different structures in any situation as well as a greater degree of ability within the domain of each skill.

Our program made an integrated use of problem-based learning (PBL) and cooperative learning (CL) as didactic teaching and learning strategies in the critical thinking program. These methodologies jointly exert a positive influence on the students, allowing them to participate more actively in the learning process, achieve better results in contextualizing content and developing skills and abilities for problem-solving, and improve motivation.

To carry out our methodology in the classrooms, we have designed a teaching system aligned with these directives. Two types of tasks are done: (1) comprehension and (2) production. The materials we used to carry out these activities are the same for all the program blocks. One key element in our aim of teaching how to think critically must be its usefulness to our students, which is only achieved through application. This makes it important to contextualize reasoning types within common situations or problems, aiding students to use them regularly and understand their usefulness. Our intention with the materials we use is to face the problems of transference, usefulness, integrated skills, and how to produce these things. Accordingly, the materials used for the tasks are: (1) common situations and (2) professional/personal problems.

The tasks which the students perform take place over a week. They work in cooperative groups in class, and then review, correct, and clarify together, promoting reflection on their achievements and errors, which fortifies metacognition. Students get the necessary feedback on the work performed which will help them progressively acquire fundamental procedural contents. Our goal here is that students become conscious of their own thought processes in order to improve them. In this way, via the dialogue achieved between teachers and students as well as between the students themselves in their cooperative work, metacognition is developed. For conscious performance of tasks, the students will receive rubrics for each and every task to guide them in their completion.

Application of the ARDESOS-DIAPROVE program was done across a semester in the Psychology Department of the Public University of the North of Spain. One week before teaching began; critical thinking and metacognition evaluations were done. This was also done 1 week after the intervention ended, in order to gather the second measurement for PENCRISAL and MAI. The timelapse between the pre-treatment and post-treatment measurements was 4 months. The intervention was done by instructors with training and good experience in the program.

To test our objective, we used a quasi-experimental pre-post design with repeated measurements.

Statistical Analysis

For statistical analysis, we used the IBM SPSS Statistics 26 statistical packet. The statistical tools and techniques used were: frequency and percentage tables for qualitative variables, exploratory and descriptive analysis of quantitative variables with a goodness of fit test to the normal Gaussian model, habitual descriptive statistics (median, SD, etc.) for numerical variables, and Student’s t -tests for significance of difference.

To begin, a descriptive analysis of the study variables was carried out. Tables 1 , ​ ,2 2 present the summary of descriptions for the scores obtained by students in the sample, as well as the asymmetry and kurtosis coefficients for their distribution.

Description of critical thinking measurement (PENCRISAL).

TOT_PRE, PENCRISAL pre-test; RD_PRE, Deductive reasoning pre-test; RI_PRE, Inductive reasoning pre-test; RP_PRE, Practical reasoning pre-test; TD_PRE, Decision making pre-test; SP_PRE, Problem solving pre-test; TOT_POST, PENCRISAL post-test; RD_ POST, Deductive reasoning post-test; RI_ POST, Inductive reasoning post-test; RP_ POST, Practical reasoning post-test; TD_ POST, Decision making post-test; SP_ POST, Problem solving post-test; Min, minimum, Max, maximum, Asym, asymmetry; and Kurt, kurtosis.

Description of metacognition measurement (MAI).

TOT_MAI_PRE, MAI pre-test; Decla_PRE, Declarative pre-test; Proce_PRE, Procedural pre-test; Condi_PRE, Conditional pre-test; CONO_PRE, Knowledge pre-test; Plani_PRE, Planning pre-test; Orga_PRE, Organization pre-test; Moni_PRE, Monitoring pre-test; Depu_PRE, Troubleshooting pre-test; Eva_PRE, Evaluation pre-test; REGU_PRE, Regulation pre-test; TOT_MAI_POST, MAI post-test; Decla_ POST, Declarative post-test; Proce_ POST, Procedural post-test; Condi_ POST, Conditional post-test; CONO_ POST, Knowledge post-test; Plani_ POST, Planning post-test; Orga_POST, Organization post-test; Moni_ POST, Monitoring post-test; Depu_ POST, Troubleshooting post-test; Eva_ POST, Evaluation post-test; and REGU_ POST, Regulation post-test;

As we see in the description of all study variables, the evidence is that the majority of them adequately fit the normal model, although some present significant deviations which can be explained by sample size.

Next, to verify whether there were significant differences in the metacognition variable based on measurements before and after the intervention, we contrasted medians for samples related with Student’s t -test (see Table 3 ).

Comparison of the METAKNOWLEDGE variable as a function of PRE-POST measurements.

The results show that there are significant differences in the metaknowledge scale total and in most of its dimensions, where all the post medians for both the scale overall and for the three dimensions of the knowledge factor (declarative, procedural, and conditional) are higher than the pre-medians. However, in the cognition regulation dimension, there are only significant differences in the total and in the planning, organization, and monitoring dimensions. The medians are also greater in the post-test than the pre-test. However, the troubleshooting and evaluation dimensions do not differ significantly after intervention.

Finally, for critical thinking skills, the results show significant differences in the scale total and in the five factors regarding the measurement time, where performance medians rise after intervention (see Table 4 ).

Comparison of the CRITICAL THINKING variable as a function of PRE-POST measurements.

These results show how metacognition improves due to CT intervention, as well as how critical thinking also improves with metacognitive intervention and CT skills intervention. Thus, it improves how people think about thinking as well as about the results achieved, since metacognition supports decision-making and final evaluation about proper strategies to solve problems.

Discussion and Conclusions

The general aim of our study was to know whether a critical thinking intervention program can also influence metacognitive processes. We know that our teaching methodology improves cross-sectional skills in argumentation, explanation, decision-making, and problem-solving, but we do not know if this intervention also directly or indirectly influences metacognition. In our study, we sought to shed light on this little-known point. If we bear in mind the centrality of how we think about thinking for our cognitive machinery to function properly and reach the best results possible in the problems we face, it is hard to understand the lack of attention given to this theme in other research. Our study aimed to remedy this deficiency somewhat.

As said in the introduction, metacognition has to do with consciousness, planning, and regulation of our activities. These mechanisms, as understood by many authors, have a blended cognitive and non-cognitive nature, which is a conceptual imprecision; what is known, though, is the enormous influence they exert on fundamental thinking processes. However, there is a large knowledge gap about the factors which make metacognition itself improve. This second research lacuna is what we have partly aimed to shrink here as well with this study. Our guide has been the idea of knowing how to improve metacognition from a teaching initiative and from the improvement of fundamental critical thinking skills.

Our study has shed light in both directions, albeit in a modest way, since its design does not allow us to unequivocally discern some of the results obtained. However, we believe that the data provide relevant information to know more about existing relations between skills and metacognition, something which has seen little contrast. These results allow us to better describe these relations, guiding the design of future studies which can better discern their roles. Our data have shown that this relation is bidirectional, so that metacognition improves thinking skills and vice versa. It remains to establish a sequence of independent factors to avoid this confusion, something which the present study has aided with to be able to design future research in this area.

As the results show, total differences in almost all metaknowledge dimensions are higher after intervention; specifically, we see how in the knowledge factor the declarative, procedural, and conditional dimensions improve in post-measurements. This improvement moves in the direction we predicted. However, the cognitive regulation dimension only shows differences in the total, and in the planning, organization, and regulation dimensions. We can see how the declarative knowledge dimensions are more sensitive than the procedural ones to change, and within the latter, the dimensions over which we have more control are also more sensitive. With troubleshooting and evaluation, no changes are seen after intervention. We may interpret this lack of effects as being due to how everything referring to evaluating results is highly determined by calibration capacity, which is influenced by personality factors not considered in our study. Regarding critical thinking, we found differences in all its dimensions, with higher scores following intervention. We can tentatively state that this improved performance can be influenced not only by interventions, but also by the metacognitive improvement observed, although our study was incapable of separating these two factors, and merely established their relation.

As we know, when people think about thinking they can always increase their critical thinking performance. Being conscious of the mechanisms used in problem-solving and decision-making always contributes to improving their execution. However, we need to go into other topics to identify the specific determinants of these effects. Does performance improve because skills are metacognitively benefited? If so, how? Is it only the levels of consciousness which aid in regulating and planning execution, or do other factors also have to participate? What level of thinking skills can be beneficial for metacognition? At what skill level does this metacognitive change happen? And finally, we know that teaching is always metacognitive to the extent that it helps us know how to proceed with sufficient clarity, but does performance level modify consciousness or regulation level of our action? Do bad results paralyze metacognitive activity while good ones stimulate it? Ultimately, all of these open questions are the future implications which our current study has suggested. We believe them to be exciting and necessary challenges, which must be faced sooner rather than later. Finally, we cannot forget the implications derived from specific metacognitive instruction, as presented at the start of this study. An intervention of this type should also help us partially answer the aforementioned questions, as we cannot obviate what can be modified or changed by direct metacognition instruction.

Data Availability Statement

Ethics statement.

Ethical review and approval was not required for the study on human participants in accordance with the local legislation and institutional requirements. The patients/participants provided their written informed consent to participate in this study.

Author Contributions

SR and CS contributed to the conception and design of the study. SR organized the database, performed the statistical analysis, and wrote the first draft of the manuscript. SR, CS, and CO wrote sections of the manuscript. All authors contributed to the article and approved the submitted version.

This study was partly financed by the Project FONDECYT no. 11220056 ANID-Chile.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Publisher’s Note

All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.

• Akama K. (2006). Relations among self-efficacy, goal setting, and metacognitive experiences in problem solving . Psychol. Rep. 98 , 895–907. doi: 10.2466/pr0.98.3.895-907, PMID: [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Antonietti A., Ignazi S., Perego P. (2000). Metacognitive knowledge about problem solving methods . Br. J. Educ. Psychol. 70 , 1–16. doi: 10.1348/000709900157921, PMID: [ PubMed ] [ CrossRef ] [ Google Scholar ]
• APA (1990). Critical thinking: A statement of expert consensus for purposes of educational assessment and instruction. Executive Summary “The Delphi Report.”
• Arslan S. (2014). An investigation of the relationships between metacognition and self-regulation with structural equation . Int. Online J. Educ. Sci. 6 , 603–611. doi: 10.15345/IOJES.2014.03.009 [ CrossRef ] [ Google Scholar ]
• Ávila M., Bianchetti M., González A. (2017). Uso del método “Think Aloud” en la investigación cualitativa . Pistas Educ. 39 , 26–38. [ Google Scholar ]
• Azevedo R. (2020). Reflections on the field of metacognition: issues, challenges, and opportunities . Metacogn. Learn. 15 , 91–98. doi: 10.1007/s11409-020-09231-x [ CrossRef ] [ Google Scholar ]
• Berardi-Coletta B., Buyer L. S., Dominowski R. L., Rellinger E. R. (1995). Metacognition and problem solving: a process-oriented approach . J. Consult. Clin. Psychol. 21 , 205–223. [ Google Scholar ]
• Black S. (2005). Teaching students to think critically . Educ. Digest 70 , 42–47. [ Google Scholar ]
• Choy S. C., Cheah P. K. (2009). Teacher perceptions of critical thinking among students and its influence on higher education . Int. J. Teach. Learn. Higher Educ. 20 , 198–206. [ Google Scholar ]
• Coutinho S., Wiemer-Hastings K., Skowronski J. J., Britt M. A. (2005). Metacognition, need for cognition and use of explanations during ongoing learning and problem solving . Learn. Individ. Differ. 15 , 321–337. doi: 10.1016/j.lindif.2005.06.001 [ CrossRef ] [ Google Scholar ]
• Dahik S., Cáneppa C., Dahik C., Feijoò K. (2019). Estrategias de Think-Aloud para mejorar la habilidad de lectura en estudiantes en el centro de idiomas en la universidad técnica de Babahoyo . Rev. Magaz. Ciencias 4 , 65–83. doi: 10.5281/zenodo.3239552 [ CrossRef ] [ Google Scholar ]
• Deci E. L., Ryan R. M. (1985). The general causality orientations scale: self-determination in personality . J. Res. Pers. 19 , 109–134. doi: 10.1016/0092-6566(85)90023-6, PMID: [ CrossRef ] [ Google Scholar ]
• Doganay A., Demir O. (2011). Comparison of the level of using metacognitive strategies during study between high achieving and low achieving prospective teachers . Educ. Sci. Theor. Pract. 11 , 2036–2043. [ Google Scholar ]
• Drigas A., Mitsea E. (2020). The 8 pillars of metacognition . Int. J. Emerg. Technol. Learn. 15 , 162–178. doi: 10.3991/ijet.v15i21.14907 [ CrossRef ] [ Google Scholar ]
• Drigas A., Mitsea E. (2021). 8 pillars X 8 layers model of metacognition: educational strategies, exercises and trainings . Int. J. Online Biomed. Eng. 17 , 115–134. doi: 10.3991/ijoe.v17i08.23563 [ CrossRef ] [ Google Scholar ]
• Ennis R. H. (1987). “ A taxonomy of critical thinking dispositions and abilities ,” in Teaching Thinking Skills. eds. Baron J. B., Sternberg R. J. (New York: Freeman and Company; ), 9–26. [ Google Scholar ]
• Ennis R. H. (1996). Critical Thinking. Upper Saddle River, NJ: Prentice-Hall [ Google Scholar ]
• Facione P. A. (1990). Critical Thinking: A Statement of expert consensus for Purposes of Educational Assessment and Instruction—Executive Summary of the delphi Report. Millbrae: California Academic Press [ Google Scholar ]
• Facione P. A. (2011). Think Critically. New York: Prentice-Hall. [ Google Scholar ]
• Flavell J. H. (1976). “ Metacognitive aspects of problem solving ,” in The Nature of Intelligence. ed. Resnik L. B. (Hillsdale, N.J: Erlbaum; ), 231–235. [ Google Scholar ]
• Flavell J. H. (2004). Theory of the mind development: retrospect and prospect . Merrill-Palmer Q. 50 , 274–290. doi: 10.1353/mpq.2004.0018 [ CrossRef ] [ Google Scholar ]
• García T., Rodríguez C., González-Castro P., Álvarez-García D., González-Pienda J.-A. (2016). Metacognición y funcionamiento ejecutivo en Educación Primaria [Metacognition and executive functioning in Elementary School] . Ann. Psychol. 32 , 474–483. doi: 10.6018/analesps.32.2.202891 [ CrossRef ] [ Google Scholar ]
• Glaser R. (1994). “ Learning theory and instruction ,” in International Perspectives on Psychological Science. Vol. 2 . eds. D’Ydewalle G., Eelen P., Bertelson B. (NJ: Erlbaum; ) [ Google Scholar ]
• Halpern D. (1998). Teaching critical thinking for transfer across domains—dispositions, skills, structure training, and metacognitive monitoring . Am. Psychol. 53 , 449–455. doi: 10.1037/0003-066X.53.4.449, PMID: [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Halpern D. (2003). Halpern Critical Thinking Assessment Using Everyday Situations: Background and Scoring Standards. Claremont, CA: Claremont McKenna College. [ Google Scholar ]
• Halpern D. F., Dunn D. S. (2021). Critical thinking: A model of intelligence for solving real-world problems . J. Intellig. 9 :22. doi: 10.3390/jintelligence9020022, PMID: [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Halpern D. F., Dunn D. S. (2022). Thought and Knowledge. An Introduction to Critical-Thinking . 6th Edn. New York: Taylor and Francis. [ Google Scholar ]
• Huertas Bustos A. P., Vesga Bravo G. J., Gilando León M. (2014). Validación del instrumento “Inventario de Habilidades Metacognitivas (MAI)” con estudiantes colombianos . Praxis Saber 5 , 55–74. doi: 10.19053/22160159.3022 [ CrossRef ] [ Google Scholar ]
• Jaramillo S., Osses S. (2012). Validación de un Instrumento sobre Metacognición para Estudiantes de Segundo Ciclo de Educación General Básica . Estud. Pedag. 38 , 117–131. doi: 10.4067/S0718-07052012000200008 [ CrossRef ] [ Google Scholar ]
• Johnson R. H. (2008). “Critical thinking, logic and argumentation,” in Paper presented at the Conferencia Internacional: Lógica, Argumentación y Pensamiento Crítico. Santiago de Chile. January 8–11. [ Google Scholar ]
• Klimenko O., Alvares J. L. (2009). Aprender cómo aprendo: la enseñanza de estrategias metacognitivas . Educ. Educ. 12 , 11–28. [ Google Scholar ]
• Kramarski B., Mevarceh Z. R., Arami M. (2002). The effect of metacognitive instruction on solving mathematical authentic tasks . Educ. Stud. Math. 49 , 225–250. doi: 10.1023/A:1016282811724 [ CrossRef ] [ Google Scholar ]
• Kuhn D., Dean D. (2004). Metacognition: a bridge between cognitive psychology and educational practice . Theory Pract. 43 , 268–274. doi: 10.1207/s15430421tip4304_4 [ CrossRef ] [ Google Scholar ]
• Larkin S. (2009). Metacognition in Young Children. New York, NY: Routledge [ Google Scholar ]
• Lipman M. (2003). Thinking in Education (2nd Edn.). Cambridge, MA: Cambridge University Press [ Google Scholar ]
• Magno C. (2010). The role of metacognitive skills in developing critical thinking . Metacogn. Learn. 5 , 137–156. doi: 10.1007/s11409-010-9054-4, PMID: [ CrossRef ] [ Google Scholar ]
• Mateos M. (2001). Metacognición y Educación. Buenos Aires: Aique [ Google Scholar ]
• Mayor J., Suengas A., González Marqués J. (1993). Estrategias Metacognitivas. Aprender a Aprendery Aprender a Pensar. Madrid: Síntesis [ Google Scholar ]
• Orion N., Kali Y. (2005). The effect of an earth-science learning program on students’ scientific thinking skills . J. Geosci. Educ. 53 , 387–394. doi: 10.5408/1089-9995-53.4.387 [ CrossRef ] [ Google Scholar ]
• Ossa C., Rivas S.F., Saiz C. (2016). Estrategias metacognitivas en el desarrollo del análisis argumentativo En IV Seminário Internacional Cognição, aprendizagem e desempenho. eds. Casanova J., Bisinoto C., Almeida L. (Braga: Livro de atas; ), 30–47. [ Google Scholar ]
• Özsoy G. (2011). An investigation of the relationship between metacognition and mathematics achievement . Asia Pac. Educ. Rev. 12 , 227–235. doi: 10.1007/s12564-010-9129-6 [ CrossRef ] [ Google Scholar ]
• Paul R., Elder L. (2001). Critical Thinking Handbook: Basic Theory and Instructional Structures. Dillon Beach, CA: Foundation for Critical Thinking [ Google Scholar ]
• Paul R., Elder A. D. (2006). Critical Thinking. Learn the Tools the Best Thinkers Use. Upper Saddle River, NJ: Pearson/Prentice Hall [ Google Scholar ]
• Rivas S. F., Saiz C. (2012). Validación y propiedades psicométricas de la prueba de pensamiento crítico PENCRISAL . Rev. Electrón. Metodol. Aplic. 17 , 18–34. [ Google Scholar ]
• Rivas S. F., Saiz C., Ossa C. (2017). “ Desarrollo de las estrategias metacognitivas mediante el programa de instrucción en pensamiento crítico ARDESOS .” in II Seminario Internacional de 660 Pensamiento Crítico. Manizales (Colombia). de octubre de 11–13, 2017. [ Google Scholar ]
• Ryan R. M., Deci E. L. (2000). Intrinsic and extrinsic motivations: classic definitions and new directions . Contemp. Educ. Psychol. 21 , 54–67. doi: 10.1006/ceps.1999.1020, PMID: [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Saiz C. (2017). Pensamiento Crítico y Cambio. Madrid: Pirámide. [ Google Scholar ]
• Saiz C. (2020). Pensamiento Crítico y Eficacia. 2ª Edn. Madrid: Pirámide. [ Google Scholar ]
• Saiz C., Rivas S. F. (2008). Evaluación en pensamiento crítico: una propuesta para diferenciar formasde pensar . Ergo. Nueva Época 22-23 , 25–66. [ Google Scholar ]
• Saiz C., Rivas S. F. (2011). Evaluation of the ARDESOS program: an initiative to improve criticalthinking skills . J. Scholar. Teach. Learn. 11 , 34–51. [ Google Scholar ]
• Saiz C., Rivas S. F. (2012). Pensamiento crítico y aprendizaje basado en problemas . Rev. Docenc. Univ. 10 , 325–346. doi: 10.4995/redu.2012.6026 [ CrossRef ] [ Google Scholar ]
• Saiz C., Rivas S. F. (2016). New teaching techniques to improve critical thinking . DIAPROVE Methodol. 40 , 3–36. [ Google Scholar ]
• Sánchez-Castaño J. A., Castaño-Mejía O. Y., Tamayo-Alzate O. E. (2015). La argumentación metacognitiva en el aula de ciencias . Rev. Latin. Cienc. Soc. 13 , 1153–1168. doi: 10.11600/1692715x.13242110214 [ CrossRef ] [ Google Scholar ]
• Sastre-Riba S. (2012). Alta capacidad intelectual: perfeccionismo y regulación metacognitiva . Rev. Neurol. 54 , S21–S29. doi: 10.33588/rn.54S01.2012011, PMID: [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Sattin D., Magnani F. G., Bartesaghi L., Caputo M., Fittipaldo A. V., Cacciatore M., et al.. (2021). Theoretical models of consciousness: a scoping review . Brain Sci. 11 :535. doi: 10.3390/brainsci11050535, PMID: [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Schraw G. (1998). Promoting general metacognitive awareness . Instr. Sci. 26 , 113–125. doi: 10.1023/A:1003044231033, PMID: [ CrossRef ] [ Google Scholar ]
• Schraw G., Dennison R. (1994). Assessing metacognitive awareness . Contemp. Educ. Psychol. 19 , 460–475. doi: 10.1006/ceps.1994.1033, PMID: [ CrossRef ] [ Google Scholar ]
• Schroyens W. (2005). Knowledge and thought: an introduction to critical thinking . Exp. Psychol. 52 , 163–164. doi: 10.1027/1618-3169.52.2.163, PMID: [ CrossRef ] [ Google Scholar ]
• Shekhar M., Rahnev D. (2021). Sources of metacognitive inefficiency . Trends Cogn. Sci. 25 , 12–23. doi: 10.1016/j.tics.2020.10.007, PMID: [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Tamayo-Alzate O., Cadavid-Alzate V., Montoya-Londoño D. (2019). Análisis metacognitivo en estudiantes de básica, durante la resolución de dos situaciones experimentales en la clase de Ciencias Naturales . Rev. Colomb. Educ. 76 , 117–141. doi: 10.17227/rce.num76-4188 [ CrossRef ] [ Google Scholar ]
• Ugartetxea J. (2001). Motivación y metacognición, más que una relación . Relieve 7 :4442. doi: 10.7203/relieve.7.2.4442 [ CrossRef ] [ Google Scholar ]
• Urdaneta M. (2014). Diálogo para la reflexión: compartiendo la experiencia de aula desde el proyecto pedagógico . Innov. Educ. 16 , 43–49. doi: 10.22458/ie.v16i21.902 [ CrossRef ] [ Google Scholar ]
• Van der Stel M., Veenman M. V. J. (2010). Development of metacognitive skillfulness: a longitudinal study . Learn. Individ. Differ. 20 , 220–224. doi: 10.1016/j.lindif.2009.11.005 [ CrossRef ] [ Google Scholar ]

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings
• My Bibliography
• Collections
• Citation manager

Save citation to file

Email citation, add to collections.

• Create a new collection
• Add to an existing collection

Add to My Bibliography

Your saved search, create a file for external citation management software, your rss feed.

• Search in PubMed
• Search in NLM Catalog
• Add to Search

Critical thinking in nursing clinical practice, education and research: From attitudes to virtue

Affiliations.

• 1 Department of Fundamental Care and Medical Surgital Nursing, Faculty of Medicine and Health Sciences, School of Nursing, Consolidated Research Group Quantitative Psychology (2017-SGR-269), University of Barcelona, Barcelona, Spain.
• 2 Department of Fundamental Care and Medical Surgital Nursing, Faculty of Medicine and Health Sciences, School of Nursing, Consolidated Research Group on Gender, Identity and Diversity (2017-SGR-1091), University of Barcelona, Barcelona, Spain.
• 3 Department of Fundamental Care and Medical Surgital Nursing, Faculty of Medicine and Health Sciences, School of Nursing, University of Barcelona, Barcelona, Spain.
• 4 Multidisciplinary Nursing Research Group, Vall d'Hebron Research Institute (VHIR), Vall d'Hebron Hospital, Barcelona, Spain.
• PMID: 33029860
• DOI: 10.1111/nup.12332

Critical thinking is a complex, dynamic process formed by attitudes and strategic skills, with the aim of achieving a specific goal or objective. The attitudes, including the critical thinking attitudes, constitute an important part of the idea of good care, of the good professional. It could be said that they become a virtue of the nursing profession. In this context, the ethics of virtue is a theoretical framework that becomes essential for analyse the critical thinking concept in nursing care and nursing science. Because the ethics of virtue consider how cultivating virtues are necessary to understand and justify the decisions and guide the actions. Based on selective analysis of the descriptive and empirical literature that addresses conceptual review of critical thinking, we conducted an analysis of this topic in the settings of clinical practice, training and research from the virtue ethical framework. Following JBI critical appraisal checklist for text and opinion papers, we argue the need for critical thinking as an essential element for true excellence in care and that it should be encouraged among professionals. The importance of developing critical thinking skills in education is well substantiated; however, greater efforts are required to implement educational strategies directed at developing critical thinking in students and professionals undergoing training, along with measures that demonstrate their success. Lastly, we show that critical thinking constitutes a fundamental component in the research process, and can improve research competencies in nursing. We conclude that future research and actions must go further in the search for new evidence and open new horizons, to ensure a positive effect on clinical practice, patient health, student education and the growth of nursing science.

Keywords: critical thinking; critical thinking attitudes; nurse education; nursing care; nursing research.

© 2020 John Wiley & Sons Ltd.

PubMed Disclaimer

Similar articles

• Student and educator experiences of maternal-child simulation-based learning: a systematic review of qualitative evidence protocol. MacKinnon K, Marcellus L, Rivers J, Gordon C, Ryan M, Butcher D. MacKinnon K, et al. JBI Database System Rev Implement Rep. 2015 Jan;13(1):14-26. doi: 10.11124/jbisrir-2015-1694. JBI Database System Rev Implement Rep. 2015. PMID: 26447004
• Health professionals' experience of teamwork education in acute hospital settings: a systematic review of qualitative literature. Eddy K, Jordan Z, Stephenson M. Eddy K, et al. JBI Database System Rev Implement Rep. 2016 Apr;14(4):96-137. doi: 10.11124/JBISRIR-2016-1843. JBI Database System Rev Implement Rep. 2016. PMID: 27532314 Review.
• Strategies to overcome obstacles in the facilitation of critical thinking in nursing education. Mangena A, Chabeli MM. Mangena A, et al. Nurse Educ Today. 2005 May;25(4):291-8. doi: 10.1016/j.nedt.2005.01.012. Epub 2005 Apr 12. Nurse Educ Today. 2005. PMID: 15896414
• Ethics in nursing education: learning to reflect on care practices. Vanlaere L, Gastmans C. Vanlaere L, et al. Nurs Ethics. 2007 Nov;14(6):758-66. doi: 10.1177/0969733007082116. Nurs Ethics. 2007. PMID: 17901186 Review.
• Teaching strategies and outcome assessments targeting critical thinking in bachelor nursing students: a scoping review protocol. Westerdahl F, Carlson E, Wennick A, Borglin G. Westerdahl F, et al. BMJ Open. 2020 Feb 2;10(1):e033214. doi: 10.1136/bmjopen-2019-033214. BMJ Open. 2020. PMID: 32014875 Free PMC article. Review.
• Higher Vocational Nursing Students' Clinical Core Competence in China: A Cross-Sectional Study. Wang S, Huang S, Yan L. Wang S, et al. SAGE Open Nurs. 2024 Mar 1;10:23779608241233147. doi: 10.1177/23779608241233147. eCollection 2024 Jan-Dec. SAGE Open Nurs. 2024. PMID: 38435341 Free PMC article.
• Effect of the case-based learning method combined with virtual reality simulation technology on midwifery laboratory courses: A quasi-experimental study. Zhao L, Dai X, Chen S. Zhao L, et al. Int J Nurs Sci. 2023 Dec 16;11(1):76-82. doi: 10.1016/j.ijnss.2023.12.009. eCollection 2024 Jan. Int J Nurs Sci. 2023. PMID: 38352279 Free PMC article.
• Translation, validation and psychometric properties of the Albanian version of the Nurses Professional Competence Scale Short form. Duka B, Stievano A, Caruso R, Prendi E, Ejupi V, Spada F, De Maria M, Rocco G, Notarnicola I. Duka B, et al. Acta Biomed. 2023 Aug 3;94(4):e2023197. doi: 10.23750/abm.v94i4.13575. Acta Biomed. 2023. PMID: 37539614 Free PMC article.
• A study of the effects of blended learning on university students' critical thinking: A systematic review. Haftador AM, Tehranineshat B, Keshtkaran Z, Mohebbi Z. Haftador AM, et al. J Educ Health Promot. 2023 Mar 31;12:95. doi: 10.4103/jehp.jehp_665_22. eCollection 2023. J Educ Health Promot. 2023. PMID: 37288404 Free PMC article. Review.
• Multilevel Modeling of Individual and Group Level Influences on Critical Thinking and Clinical Decision-Making Skills among Registered Nurses: A Study Protocol. Zainal NH, Musa KI, Rasudin NS, Mamat Z. Zainal NH, et al. Healthcare (Basel). 2023 Apr 19;11(8):1169. doi: 10.3390/healthcare11081169. Healthcare (Basel). 2023. PMID: 37108003 Free PMC article.
• Alfaro-Lefevre, R. (2019). Critical thinking, clinical reasoning and clinical judgment. A practical approach, 7th ed. Elsevier.
• Armstrong, A. (2006). Towards a strong virtue ethics for nursing practice. Nursing Philosophy, 7(3), 110-124.
• Armstrong, A. (2007). Nursing ethics. A virtue-based approach. Palgrave Macmillian.
• Banks-Wallace, J., Despins, L., Adams-Leander, S., McBroom, L., & Tandy, L. (2008). Re/Affirming and re/conceptualizing disciplinary knowledge as the foundations for doctoral education. Advances in Nursing Sciencies, 31(1), 67-78. https://doi.org/10.1097/01.ANS.0000311530.81188.88
• Banning, M. (2008). Clinical reasoning and its application to nursing: Concepts and research studies. Nurse Education in Practice, 8(3), 177-183.
• Search in MeSH

Grants and funding

• PREI-19-007-B/School of Nursing. Faculty of Medicine and Health Sciences. University of Barcelona

LinkOut - more resources

Full text sources.

• Ovid Technologies, Inc.

• Citation Manager

NCBI Literature Resources

MeSH PMC Bookshelf Disclaimer

The PubMed wordmark and PubMed logo are registered trademarks of the U.S. Department of Health and Human Services (HHS). Unauthorized use of these marks is strictly prohibited.

Promoting critical thinking through an evidence-based skills fair intervention

Journal of Research in Innovative Teaching & Learning

ISSN : 2397-7604

Article publication date: 23 November 2020

Issue publication date: 1 April 2022

The lack of critical thinking in new graduates has been a concern to the nursing profession. The purpose of this study was to investigate the effects of an innovative, evidence-based skills fair intervention on nursing students' achievements and perceptions of critical thinking skills development.

Design/methodology/approach

The explanatory sequential mixed-methods design was employed for this study.

The findings indicated participants perceived the intervention as a strategy for developing critical thinking.

Originality/value

The study provides educators helpful information in planning their own teaching practice in educating students.

Critical thinking

Evidence-based practice, skills fair intervention.

Gonzalez, H.C. , Hsiao, E.-L. , Dees, D.C. , Noviello, S.R. and Gerber, B.L. (2022), "Promoting critical thinking through an evidence-based skills fair intervention", Journal of Research in Innovative Teaching & Learning , Vol. 15 No. 1, pp. 41-54. https://doi.org/10.1108/JRIT-08-2020-0041

Emerald Publishing Limited

Copyright © 2020, Heidi C. Gonzalez, E-Ling Hsiao, Dianne C. Dees, Sherri R. Noviello and Brian L. Gerber

Published in Journal of Research in Innovative Teaching & Learning . Published by Emerald Publishing Limited. This article is published under the Creative Commons Attribution (CC BY 4.0) licence. Anyone may reproduce, distribute, translate and create derivative works of this article (for both commercial and non-commercial purposes), subject to full attribution to the original publication and authors. The full terms of this licence may be seen at http://creativecommons.org/licences/by/4.0/legalcode

Introduction

Critical thinking (CT) was defined as “cognitive skills of analyzing, applying standards, discriminating, information seeking, logical reasoning, predicting, and transforming knowledge” ( Scheffer and Rubenfeld, 2000 , p. 357). Critical thinking is the basis for all professional decision-making ( Moore, 2007 ). The lack of critical thinking in student nurses and new graduates has been a concern to the nursing profession. It would negatively affect the quality of service and directly relate to the high error rates in novice nurses that influence patient safety ( Arli et al. , 2017 ; Saintsing et al. , 2011 ). It was reported that as many as 88% of novice nurses commit medication errors with 30% of these errors due to a lack of critical thinking ( Ebright et al. , 2004 ). Failure to rescue is another type of error common for novice nurses, reported as high as 37% ( Saintsing et al. , 2011 ). The failure to recognize trends or complications promptly or take action to stabilize the patient occurs when health-care providers do not recognize signs and symptoms of the early warnings of distress ( Garvey and CNE series, 2015 ). Internationally, this lack of preparedness and critical thinking attributes to the reported 35–60% attrition rate of new graduate nurses in their first two years of practice ( Goodare, 2015 ). The high attrition rate of new nurses has expensive professional and economic costs of $82,000 or more per nurse and negatively affects patient care ( Twibell et al. , 2012 ). Facione and Facione (2013) reported the failure to utilize critical thinking skills not only interferes with learning but also results in poor decision-making and unclear communication between health-care professionals, which ultimately leads to patient deaths.

Due to the importance of critical thinking, many nursing programs strive to infuse critical thinking into their curriculum to better prepare graduates for the realities of clinical practice that involves ever-changing, complex clinical situations and bridge the gap between education and practice in nursing ( Benner et al. , 2010 ; Kim et al. , 2019 ; Park et al. , 2016 ; Newton and Moore, 2013 ; Nibert, 2011 ). To help develop students' critical thinking skills, nurse educators must change the way they teach nursing, so they can prepare future nurses to be effective communicators, critical thinkers and creative problem solvers ( Rieger et al. , 2015 ). Nursing leaders also need to redefine teaching practice and educational guidelines that drive innovation in undergraduate nursing programs.

Evidence-based practice has been advocated to promote critical thinking and help reduce the research-practice gap ( Profetto-McGrath, 2005 ; Stanley and Dougherty, 2010 ). Evidence-based practice was defined as “the conscientious, explicit, and judicious use of current best evidence in making decisions about the care of the individual patient” ( Sackett et al. , 1996 , p. 71). Skills fair intervention, one type of evidence-based practice, can be used to engage students, promote active learning and develop critical thinking ( McCausland and Meyers, 2013 ; Roberts et al. , 2009 ). Skills fair intervention helps promote a consistent teaching practice of the psychomotor skills to the novice nurse that decreased anxiety, gave clarity of expectations to the students in the clinical setting and increased students' critical thinking skills ( Roberts et al. , 2009 ). The researchers of this study had an opportunity to create an active, innovative skills fair intervention for a baccalaureate nursing program in one southeastern state. This intervention incorporated evidence-based practice rationale with critical thinking prompts using Socratic questioning, evidence-based practice videos to the psychomotor skill rubrics, group work, guided discussions, expert demonstration followed by guided practice and blended learning in an attempt to promote and develop critical thinking in nursing students ( Hsu and Hsieh, 2013 ; Oermann et al. , 2011 ; Roberts et al. , 2009 ). The effects of an innovative skills fair intervention on senior baccalaureate nursing students' achievements and their perceptions of critical thinking development were examined in the study.

Literature review

The ability to use reasoned opinion focusing equally on processes and outcomes over emotions is called critical thinking ( Paul and Elder, 2008 ). Critical thinking skills are desired in almost every discipline and play a major role in decision-making and daily judgments. The roots of critical thinking date back to Socrates 2,500 years ago and can be traced to the ancient philosopher Aristotle ( Paul and Elder, 2012 ). Socrates challenged others by asking inquisitive questions in an attempt to challenge their knowledge. In the 1980s, critical thinking gained nationwide recognition as a behavioral science concept in the educational system ( Robert and Petersen, 2013 ). Many researchers in both education and nursing have attempted to define, measure and teach critical thinking for decades. However, a theoretical definition has yet to be accepted and established by the nursing profession ( Romeo, 2010 ). The terms critical literacy, CT, reflective thinking, systems thinking, clinical judgment and clinical reasoning are used synonymously in the reviewed literature ( Clarke and Whitney, 2009 ; Dykstra, 2008 ; Jones, 2010 ; Swing, 2014 ; Turner, 2005 ).

Watson and Glaser (1980) viewed critical thinking not only as cognitive skills but also as a combination of skills, knowledge and attitudes. Paul (1993) , the founder of the Foundation for Critical Thinking, offered several definitions of critical thinking and identified three essential components of critical thinking: elements of thought, intellectual standards and affective traits. Brunt (2005) stated critical thinking is a process of being practical and considered it to be “the process of purposeful thinking and reflective reasoning where practitioners examine ideas, assumptions, principles, conclusions, beliefs, and actions in the contexts of nursing practice” (p. 61). In an updated definition, Ennis (2011) described critical thinking as, “reasonable reflective thinking focused on deciding what to believe or do” (para. 1).

The most comprehensive attempt to define critical thinking was under the direction of Facione and sponsored by the American Philosophical Association ( Scheffer and Rubenfeld, 2000 ). Facione (1990) surveyed 53 experts from the arts and sciences using the Delphi method to define critical thinking as a “purposeful, self-regulatory judgment which results in interpretation, analysis, evaluation, and inference, as well as an explanation of the evidential, conceptual, methodological, criteriological, or contextual considerations upon which judgment, is based” (p. 2).

To come to a consensus definition for critical thinking, Scheffer and Rubenfeld (2000) also conducted a Delphi study. Their study consisted of an international panel of nurses who completed five rounds of sequenced questions to arrive at a consensus definition. Critical thinking was defined as “habits of mind” and “cognitive skills.” The elements of habits of mind included “confidence, contextual perspective, creativity, flexibility, inquisitiveness, intellectual integrity, intuition, open-mindedness, perseverance, and reflection” ( Scheffer and Rubenfeld, 2000 , p. 352). The elements of cognitive skills were recognized as “analyzing, applying standards, discriminating, information seeking, logical reasoning, predicting, and transforming knowledge” ( Scheffer and Rubenfeld, 2000 , p. 352). In addition, Ignatavicius (2001) defined the development of critical thinking as a long-term process that must be practiced, nurtured and reinforced over time. Ignatavicius believed that a critical thinker required six cognitive skills: interpretation, analysis, evaluation, inference, explanation and self-regulation ( Chun-Chih et al. , 2015 ). According to Ignatavicius (2001) , the development of critical thinking is difficult to measure or describe because it is a formative rather than summative process.

Fero et al. (2009) noted that patient safety might be compromised if a nurse cannot provide clinically competent care due to a lack of critical thinking. The Institute of Medicine (2001) recommended five health care competencies: patient-centered care, interdisciplinary team care, evidence-based practice, informatics and quality improvement. Understanding the development and attainment of critical thinking is the key for gaining these future competencies ( Scheffer and Rubenfeld, 2000 ). The development of a strong scientific foundation for nursing practice depends on habits such as contextual perspective, inquisitiveness, creativity, analysis and reasoning skills. Therefore, the need to better understand how these critical thinking habits are developed in nursing students needs to be explored through additional research ( Fero et al. , 2009 ). Despite critical thinking being listed since the 1980s as an accreditation outcome criteria for baccalaureate programs by the National League for Nursing, very little improvement has been observed in practice ( McMullen and McMullen, 2009 ). James (2013) reported the number of patient harm incidents associated with hospital care is much higher than previously thought. James' study indicated that between 210,000 and 440,000 patients each year go to the hospital for care and end up suffering some preventable harm that contributes to their death. James' study of preventable errors is attributed to other sources besides nursing care, but having a nurse in place who can advocate and critically think for patients will make a positive impact on improving patient safety ( James, 2013 ; Robert and Peterson, 2013 ).

Adopting teaching practice to promote CT is a crucial component of nursing education. Research by Nadelson and Nadelson (2014) suggested evidence-based practice is best learned when integrated into multiple areas of the curriculum. Evidence-based practice developed its roots through evidence-based medicine, and the philosophical origins extend back to the mid-19th century ( Longton, 2014 ). Florence Nightingale, the pioneer of modern nursing, used evidence-based practice during the Crimean War when she recognized a connection between poor sanitary conditions and rising mortality rates of wounded soldiers ( Rahman and Applebaum, 2011 ). In professional nursing practice today, a commonly used definition of evidence-based practice is derived from Dr. David Sackett: the conscientious, explicit and judicious use of current best evidence in making decisions about the care of the individual patient ( Sackett et al. , 1996 , p. 71). As professional nurses, it is imperative for patient safety to remain inquisitive and ask if the care provided is based on available evidence. One of the core beliefs of the American Nephrology Nurses' Association's (2019) 2019–2020 Strategic Plan is “Anna must support research to develop evidence-based practice, as well as to advance nursing science, and that as individual members, we must support, participate in, and apply evidence-based research that advances our own skills, as well as nursing science” (p. 1). Longton (2014) reported the lack of evidence-based practice in nursing resulted in negative outcomes for patients. In fact, when evidence-based practice was implemented, changes in policies and procedures occurred that resulted in decreased reports of patient harm and associated health-care costs. The Institute of Medicine (2011) recommendations included nurses being leaders in the transformation of the health-care system and achieving higher levels of education that will provide the ability to critically analyze data to improve the quality of care for patients. Student nurses must be taught to connect and integrate CT and evidence-based practice throughout their program of study and continue that practice throughout their careers.

One type of evidence-based practice that can be used to engage students, promote active learning and develop critical thinking is skills fair intervention ( McCausland and Meyers, 2013 ; Roberts et al. , 2009 ). Skills fair intervention promoted a consistent teaching approach of the psychomotor skills to the novice nurse that decreased anxiety, gave clarity of expectations to the students in the clinical setting and increased students' critical thinking skills ( Roberts et al. , 2009 ). The skills fair intervention used in this study is a teaching strategy that incorporated CT prompts, Socratic questioning, group work, guided discussions, return demonstrations and blended learning in an attempt to develop CT in nursing students ( Hsu and Hsieh, 2013 ; Roberts et al. , 2009 ). It melded evidence-based practice with simulated CT opportunities while students practiced essential psychomotor skills.

Research methodology

Context – skills fair intervention.

According to Roberts et al. (2009) , psychomotor skills decline over time even among licensed experienced professionals within as little as two weeks and may need to be relearned within two months without performing a skill. When applying this concept to student nurses for whom each skill is new, it is no wonder their competency result is diminished after having a summer break from nursing school. This skills fair intervention is a one-day event to assist baccalaureate students who had taken the summer off from their studies in nursing and all faculty participated in operating the stations. It incorporated evidence-based practice rationale with critical thinking prompts using Socratic questioning, evidence-based practice videos to the psychomotor skill rubrics, group work, guided discussions, expert demonstration followed by guided practice and blended learning in an attempt to promote and develop critical thinking in baccalaureate students.

Students were scheduled and placed randomly into eight teams based on attributes of critical thinking as described by Wittmann-Price (2013) : Team A – Perseverance, Team B – Flexibility, Team C – Confidence, Team D – Creativity, Team E – Inquisitiveness, Team F – Reflection, Team G – Analyzing and Team H – Intuition. The students rotated every 20 minutes through eight stations: Medication Administration: Intramuscular and Subcutaneous Injections, Initiating Intravenous Therapy, ten-minute Focused Physical Assessment, Foley Catheter Insertion, Nasogastric Intubation, Skin Assessment/Braden Score and Restraints, Vital Signs and a Safety Station. When the students completed all eight stations, they went to the “Check-Out” booth to complete a simple evaluation to determine their perceptions of the effectiveness of the innovative intervention. When the evaluations were complete, each of the eight critical thinking attribute teams placed their index cards into a hat, and a student won a small prize. All Junior 2, Senior 1 and Senior 2 students were required to attend the Skills Fair. The Skills Fair Team strove to make the event as festive as possible, engaging nursing students with balloons, candy, tri-boards, signs and fun pre and postactivities. The Skills Fair rubrics, scheduling and instructions were shared electronically with students and faculty before the skills fair intervention to ensure adequate preparation and continuous resource availability as students move forward into their future clinical settings.

Research design

Institutional review board (IRB) approval was obtained from XXX University to conduct this study and protect human subject rights. The explanatory sequential mixed-methods design was employed for this study. The design was chosen to identify what effects a skills fair intervention that had on senior baccalaureate nursing students' achievements on the Kaplan Critical Thinking Integrated Test (KCTIT) and then follow up with individual interviews to explore those test results in more depth. In total, 52 senior nursing students completed the KCTIT; 30 of them participated in the skills fair intervention and 22 of them did not participate. The KCTIT is a computerized 85-item exam in which 85 equates to 100%, making each question worth one point. It has high reliability and validity ( Kaplan Nursing, 2012 ; Swing, 2014 ). The reliability value of the KCTIT ranged from 0.72 to 0.89. A t -test was used to analyze the test results.

A total of 11 participants were purposefully selected based on a range of six high achievers and five low achievers on the KCTIT for open-ended one-on-one interviews. Each interview was conducted individually and lasted for about 60 minutes. An open-ended interview protocol was used to guide the flow of data collection. The interviewees' ages ranged from 21 to 30 years, with an average of 24 years. One of 11 interviewees was male. Among them, seven were White, three were Black and one was Indian American. The data collected were used to answer the following research questions: (1) What was the difference in achievements on the KCTIT among senior baccalaureate nursing students who participated in the skills fair intervention and students who did not participate? (2) What were the senior baccalaureate nursing students' perceptions of internal and external factors impacting the development of critical thinking skills during the skills fair intervention? and (3) What were the senior baccalaureate nursing students' perceptions of the skills fair intervention as a critical thinking developmental strategy?

Inductive content analysis was used to analyze interview data by starting with the close reading of the transcripts and writing memos for initial coding, followed by an analysis of patterns and relationships among the data for focused coding. The intercoder reliability was established for qualitative data analysis with a nursing expert. The lead researcher and the expert read the transcript several times and assigned a code to significant units of text that corresponded with answering the research questions. The codes were compared based on differences and similarities and sorted into subcategories and categories. Then, headings and subheadings were used based on similar comments to develop central themes and patterns. The process of establishing intercoder reliability helped to increase dependability, conformability and credibility of the findings ( Graneheim and Lundman, 2004 ). In addition, methods of credibility, confirmability, dependability and transferability were applied to increase the trustworthiness of this study ( Graneheim and Lundman, 2004 ). First, reflexivity was observed by keeping journals and memos. This practice allowed the lead researcher to reflect on personal views to minimize bias. Data saturation was reached through following the recommended number of participants as well as repeated immersion in the data during analysis until no new data surfaced. Member checking was accomplished through returning the transcript and the interpretation to the participants to check the accuracy and truthfulness of the findings. Finally, proper documentation was conducted to allow accurate crossreferencing throughout the study.

Quantitative results

Results for the quantitative portion showed there was no difference in scores on the KCTIT between senior nursing students who participated in the skills fair intervention and senior nursing students who did not participate, t (50) = −0.174, p  = 0.86 > 0.05. The test scores between the nonparticipant group ( M  = 67.59, SD = 5.81) and the participant group ( M  = 67.88, SD = 5.99) were almost equal.

Qualitative results

Initial coding.

The results from the initial coding and generated themes are listed in Table 1 . First, the participants perceived the skills fair intervention as “promoting experience” and “confidence” by practicing previously learned knowledge and reinforcing it with active learning strategies. Second, the participants perceived the skills fair intervention as a relaxed, nonthreatening learning environment due to the festive atmosphere, especially in comparison to other learning experiences in the nursing program. The nonthreatening environment of the skills fair intervention allowed students to learn without fear. Third, the majority of participants believed their critical thinking was strengthened after participating. Several participants believed their perception of critical thinking was “enhanced” or “reinforced” rather than significantly changed.

Focused coding results

The final themes were derived from the analysis of patterns and relationships among the content of the data using inductive content analysis ( Saldana, 2009 ). The following was examined across the focused coding process: (1) factors impacting critical thinking skills development during skills fair intervention and (2) skills fair intervention a critical thinking skills developmental strategy.

Factors impacting critical thinking skills development . The factors impacting the development of critical thinking during the skills fair intervention were divided into two themes: internal factors and external factors. The internal factors were characteristics innate to the students. The identified internal factors were (1) confidence and anxiety levels, (2) attitude and (3) age. The external factors were the outside influences that affected the students. The external factors were (1) experience and practice, (2) faculty involvement, (3) positive learning environment and (4) faculty prompts.

I think that confidence and anxiety definitely both have a huge impact on your ability to be able to really critically think. If you start getting anxious and panicking you cannot think through the process like you need too. I do not really think gender or age necessarily would have anything to do with critical thinking.

Definitely the confidence level, I think, the more advanced you get in the program, your confidence just keeps on growing. Level of anxiety, definitely… I think the people who were in the Skills Fair for the first time, had more anxiety because they did not really know to think, they did not know how strict it was going to be, or if they really had to know everything by the book. I think the Skills Fair helped everyone's confidence levels, but especially the Jr. 2's.

Attitude was an important factor in the development of critical thinking skills during the skills fair intervention as participants believed possessing a pleasant and positive attitude meant a student was eager to learn, participate, accept responsibility for completing duties and think seriously. Participant 6 believed attitude contributed to performance in the Skills Fair.

I feel like, certain things bring critical thinking out in you. And since I'm a little bit older than some of the other students, I have had more life experiences and am able to figure stuff out better. Older students have had more time to learn by trial and error, and this and that.

Like when I had clinical with you, you'd always tell us to know our patients' medications. To always know and be prepared to answer questions – because at first as a Junior 1 we did not do that in the clinical setting… and as a Junior 2, I did not really have to know my medications, but with you as a Senior 1, I started to realize that the patients do ask about their meds, so I was making sure that I knew everything before they asked it. And just having more practice with IVs – at first, I was really nervous, but when I got to my preceptorship – I had done so many IVs and with all of the practice, it just built up my confidence with that skill so when I performed that skill during the Fair, I was confident due to my clinical experiences and able to think and perform better.

I think teachers will always affect the ability to critically think just because you want [to] get the right answer because they are there and you want to seem smart to them [Laugh]. Also, if you are leading in the wrong direction of your thinking – they help steer you back to [in] the right direction so I think that was very helpful.

You could tell the faculty really tried to make it more laid back and fun, so everybody would have a good experience. The faculty had a good attitude. I think making it fun and active helped keep people positive. You know if people are negative and not motivated, nothing gets accomplished. The faculty did an amazing job at making the Skills Fair a positive atmosphere.

However, for some of the participants, a positive learning environment depended on their fellow students. The students were randomly assigned alphabetically to groups, and the groups were assigned to starting stations at the Skills Fair. The participants claimed some students did not want to participate and displayed cynicism toward the intervention. The participants believed their cynicism affected the positive learning environment making critical thinking more difficult during the Skills Fair.

Okay, when [instructor name] was demonstrating the Chevron technique right after we inserted the IV catheter and we were trying to secure the catheter, put on the extension set, and flush the line at what seemed to be all at the same time. I forgot about how you do not want to put the tape right over the hub of the catheter because when you go back in and try to assess the IV site – you're trying to assess whether or not it is patent or infiltrated – you have to visualize the insertion site. That was one of the things that I had been doing wrong because I was just so excited that I got the IV in the vein in the first place – that I did not think much about the tape or the tegaderm for sterility. So I think an important part of critical thinking is to be able to recognize when you've made a mistake and stop, stop yourself from doing it in the future (see Table 2 ).

Skills fair intervention as a developmental strategy for critical thinking . The participants identified the skills fair intervention was effective as a developmental strategy for critical thinking, as revealed in two themes: (1) develops alternative thinking and (2) thinking before doing (See Table 3 ).

Develops alternative thinking . The participants perceived the skills fair intervention helped enhance critical thinking and confidence by developing alternative thinking. Alternative thinking was described as quickly thinking of alternative solutions to problems based on the latest evidence and using that information to determine what actions were warranted to prevent complications and prevent injury. It helped make better connections through the learning of rationale between knowledge and skills and then applying that knowledge to prevent complications and errors to ensure the safety of patients. The participants stated the learning of rationale for certain procedures provided during the skills fair intervention such as the evidence and critical thinking prompts included in the rubrics helped reinforce this connection. The participants also shared they developed alternative thinking after participating in the skills fair intervention by noticing trends in data to prevent potential complications from the faculty prompts. Participant 1 stated her instructor prompted her alternative thinking through questioning about noticing trends to prevent potential complications. She said the following:

Another way critical thinking occurred during the skills fair was when [instructor name] was teaching and prompted us about what it would be like to care for a patient with a fractured hip – I think this was at the 10-minute focused assessment station, but I could be wrong. I remember her asking, “What do you need to be on the look-out for? What can go wrong?” I automatically did not think critically very well and was only thinking circulation in the leg, dah, dah, dah. But she was prompting us to think about mobility alterations and its effect on perfusion and oxygenation. She was trying to help us build those connections. And I think that's a lot of the aspects of critical thinking that gets overlooked with the nursing student – trouble making connections between our knowledge and applying it in practice.

Thinking before doing . The participants perceived thinking before doing, included thinking of how and why certain procedures, was necessary through self-examination prior to taking action. The hands-on situational learning allowed the participants in the skills fair intervention to better notice assessment data and think at a higher level as their previous learning of the skills was perceived as memorization of steps. This higher level of learning allowed participants to consider different future outcomes and analyze pertinent data before taking action.

I think what helped me the most is considering outcomes of my actions before I do anything. For instance, if you're thinking, “Okay. Well, I need to check their blood pressure before I administer this blood pressure medication – or the blood pressure could potentially bottom out.” I really do not want my patient to bottom out and get hypotensive because I administered a medication that was ordered, but not safe to give. I could prevent problems from happening if I know what to be on alert for and act accordingly. So ultimately knowing that in the clinical setting, I can prevent complications from happening and I save myself, my license, and promote patient safety. I think knowing that I've seen the importance of critical thinking already in practice has helped me value and understand why I should be critically thinking. Yes, we use the 5-rights of medication safety – but we also have to think. For instance, if I am going to administer insulin – what do I need to know or do to give this safely? What is the current blood sugar? Has the patient been eating? When is the next meal scheduled? Is the patient NPO for a procedure? Those are examples of questions to consider and the level of thinking that needs to take place prior to taking actions in the clinical setting.

Although the results of quantitative data showed no significant difference in scores on the KCTIT between the participant and nonparticipant groups, during the interviews some participants attributed this result to the test not being part of a course grade and believed students “did not try very hard to score well.” However, the participants who attended interviews did identify the skills fair intervention as a developmental strategy for critical thinking by helping them develop alternative thinking and thinking before doing. The findings are supported in the literature as (1) nurses must recognize signs of clinical deterioration and take action promptly to prevent potential complications ( Garvey and CNE series 2015 ) and (2) nurses must analyze pertinent data and consider all possible solutions before deciding on the most appropriate action for each patient ( Papathanasiou et al. , 2014 ).

The skills fair intervention also enhanced the development of self-confidence by participants practicing previously learned skills in a controlled, safe environment. The nonthreatening environment of the skills fair intervention allowed students to learn without fear and the majority of participants believed their critical thinking was strengthened after participating. The interview data also revealed a combination of internal and external factors that influenced the development of critical thinking during the skills fair intervention including confidence and anxiety levels, attitude, age, experience and practice, faculty involvement, positive learning environment and faculty prompts. These factors should be considered when addressing the promotion and development of critical thinking.

Conclusions, limitations and recommendations

A major concern in the nursing profession is the lack of critical thinking in student nurses and new graduates, which influences the decision-making of novice nurses and directly affects patient care and safety ( Saintsing et al. , 2011 ). Nurse educators must use evidence-based practice to prepare students to critically think with the complicated and constantly evolving environment of health care today ( Goodare, 2015 ; Newton and Moore, 2013 ). Evidence-based practice has been advocated to promote critical thinking ( Profetto-McGrath, 2005 ; Stanley and Dougherty, 2010 ). The skills fair intervention can be one type of evidence-based practice used to promote critical thinking ( McCausland and Meyers, 2013 ; Roberts et al. , 2009 ). The Intervention used in this study incorporated evidence-based practice rationale with critical thinking prompts using Socratic questioning, evidence-based practice videos to the psychomotor skill rubrics, group work, guided discussions, expert demonstration followed by guided practice and blended learning in an attempt to promote and develop critical thinking in nursing students.

The explanatory sequential mixed-methods design was employed to investigate the effects of the innovative skills fair intervention on senior baccalaureate nursing students' achievements and their perceptions of critical thinking skills development. Although the quantitative results showed no significant difference in scores on the KCTIT between students who participated in the skills fair intervention and those who did not, those who attended the interviews perceived their critical thinking was reinforced after the skills fair intervention and believed it was an effective developmental strategy for critical thinking, as it developed alternative thinking and thinking before doing. This information is useful for nurse educators who plan their own teaching practice to promote critical thinking and improve patient outcomes. The findings also provide schools and educators information that helps review their current approach in educating nursing students. As evidenced in the findings, the importance of developing critical thinking skills is crucial for becoming a safe, professional nurse. Internal and external factors impacting the development of critical thinking during the skills fair intervention were identified including confidence and anxiety levels, attitude, age, experience and practice, faculty involvement, positive learning environment and faculty prompts. These factors should be considered when addressing the promotion and development of critical thinking.

There were several limitations to this study. One of the major limitations of the study was the limited exposure of students' time of access to the skills fair intervention, as it was a one-day learning intervention. Another limitation was the sample selection and size. The skills fair intervention was limited to only one baccalaureate nursing program in one southeastern state. As such, the findings of the study cannot be generalized as it may not be representative of baccalaureate nursing programs in general. In addition, this study did not consider students' critical thinking achievements prior to the skills fair intervention. Therefore, no baseline measurement of critical thinking was available for a before and after comparison. Other factors in the nursing program could have affected the students' scores on the KCTIT, such as anxiety or motivation that was not taken into account in this study.

The recommendations for future research are to expand the topic by including other regions, larger samples and other baccalaureate nursing programs. In addition, future research should consider other participant perceptions, such as nurse educators, to better understand the development and growth of critical thinking skills among nursing students. Finally, based on participant perceptions, future research should include a more rigorous skills fair intervention to develop critical thinking and explore the link between confidence and critical thinking in nursing students.

Initial coding results

Factors impacting critical thinking skill development during skills fair intervention

Skills fair intervention as a developmental strategy for critical thinking

American Nephrology Nurses Association (ANNA) ( 2019 ), “ Learning, leading, connecting, and playing at the intersection of nephrology and nursing-2019–2020 strategic plan ”, viewed 3 Aug 2019, available at: https://www.annanurse.org/download/reference/association/strategicPlan.pdf .

Arli , S.D. , Bakan , A.B. , Ozturk , S. , Erisik , E. and Yildirim , Z. ( 2017 ), “ Critical thinking and caring in nursing students ”, International Journal of Caring Sciences , Vol. 10 No. 1 , pp. 471 - 478 .

Benner , P. , Sutphen , M. , Leonard , V. and Day , L. ( 2010 ), Educating Nurses: A Call for Radical Transformation , Jossey-Bass , San Francisco .

Brunt , B. ( 2005 ), “ Critical thinking in nursing: an integrated review ”, The Journal of Continuing Education in Nursing , Vol. 36 No. 2 , pp. 60 - 67 .

Chun-Chih , L. , Chin-Yen , H. , I-Ju , P. and Li-Chin , C. ( 2015 ), “ The teaching-learning approach and critical thinking development: a qualitative exploration of Taiwanese nursing students ”, Journal of Professional Nursing , Vol. 31 No. 2 , pp. 149 - 157 , doi: 10.1016/j.profnurs.2014.07.001 .

Clarke , L.W. and Whitney , E. ( 2009 ), “ Walking in their shoes: using multiple-perspectives texts as a bridge to critical literacy ”, The Reading Teacher , Vol. 62 No. 6 , pp. 530 - 534 , doi: 10.1598/RT.62.6.7 .

Dykstra , D. ( 2008 ), “ Integrating critical thinking and memorandum writing into course curriculum using the internet as a research tool ”, College Student Journal , Vol. 42 No. 3 , pp. 920 - 929 , doi: 10.1007/s10551-010-0477-2 .

Ebright , P. , Urden , L. , Patterson , E. and Chalko , B. ( 2004 ), “ Themes surrounding novice nurse near-miss and adverse-event situations ”, The Journal of Nursing Administration: The Journal of Nursing Administration , Vol. 34 , pp. 531 - 538 , doi: 10.1097/00005110-200411000-00010 .

Ennis , R. ( 2011 ), “ The nature of critical thinking: an outline of critical thinking dispositions and abilities ”, viewed 3 May 2017, available at: https://education.illinois.edu/docs/default-source/faculty-documents/robert-ennis/thenatureofcriticalthinking_51711_000.pdf .

Facione , P.A. ( 1990 ), Critical Thinking: A Statement of Expert Consensus for Purposes of Educational Assessment and Instruction , The California Academic Press , Millbrae .

Facione , N.C. and Facione , P.A. ( 2013 ), The Health Sciences Reasoning Test: Test Manual , The California Academic Press , Millbrae .

Fero , L.J. , Witsberger , C.M. , Wesmiller , S.W. , Zullo , T.G. and Hoffman , L.A. ( 2009 ), “ Critical thinking ability of new graduate and experienced nurses ”, Journal of Advanced Nursing , Vol. 65 No. 1 , pp. 139 - 148 , doi: 10.1111/j.1365-2648.2008.04834.x .

Garvey , P.K. and CNE series ( 2015 ), “ Failure to rescue: the nurse's impact ”, Medsurg Nursing , Vol. 24 No. 3 , pp. 145 - 149 .

Goodare , P. ( 2015 ), “ Literature review: ‘are you ok there?’ The socialization of student and graduate nurses: do we have it right? ”, Australian Journal of Advanced Nursing , Vol. 33 No. 1 , pp. 38 - 43 .

Graneheim , U.H. and Lundman , B. ( 2014 ), “ Qualitative content analysis in nursing research: concepts, procedures, and measures to achieve trustworthiness ”, Nurse Education Today , Vol. 24 No. 2 , pp. 105 - 12 , doi: 10.1016/j.nedt.2003.10.001 .

Hsu , L. and Hsieh , S. ( 2013 ), “ Factors affecting metacognition of undergraduate nursing students in a blended learning environment ”, International Journal of Nursing Practice , Vol. 20 No. 3 , pp. 233 - 241 , doi: 10.1111/ijn.12131 .

Ignatavicius , D. ( 2001 ), “ Six critical thinking skills for at-the-bedside success ”, Dimensions of Critical Care Nursing , Vol. 20 No. 2 , pp. 30 - 33 .

Institute of Medicine ( 2001 ), Crossing the Quality Chasm: A New Health System for the 21st Century , National Academy Press , Washington .

James , J. ( 2013 ), “ A new, evidence-based estimate of patient harms associated with hospital care ”, Journal of Patient Safety , Vol. 9 No. 3 , pp. 122 - 128 , doi: 10.1097/PTS.0b013e3182948a69 .

Jones , J.H. ( 2010 ), “ Developing critical thinking in the perioperative environment ”, AORN Journal , Vol. 91 No. 2 , pp. 248 - 256 , doi: 10.1016/j.aorn.2009.09.025 .

Kaplan Nursing ( 2012 ), Kaplan Nursing Integrated Testing Program Faculty Manual , Kaplan Nursing , New York, NY .

Kim , J.S. , Gu , M.O. and Chang , H.K. ( 2019 ), “ Effects of an evidence-based practice education program using multifaceted interventions: a quasi-experimental study with undergraduate nursing students ”, BMC Medical Education , Vol. 19 , doi: 10.1186/s12909-019-1501-6 .

Longton , S. ( 2014 ), “ Utilizing evidence-based practice for patient safety ”, Nephrology Nursing Journal , Vol. 41 No. 4 , pp. 343 - 344 .

McCausland , L.L. and Meyers , C.C. ( 2013 ), “ An interactive skills fair to prepare undergraduate nursing students for clinical experience ”, Nursing Education Perspectives , Vol. 34 No. 6 , pp. 419 - 420 , doi: 10.5480/1536-5026-34.6.419 .

McMullen , M.A. and McMullen , W.F. ( 2009 ), “ Examining patterns of change in the critical thinking skills of graduate nursing students ”, Journal of Nursing Education , Vol. 48 No. 6 , pp. 310 - 318 , doi: 10.3928/01484834-20090515-03 .

Moore , Z.E. ( 2007 ), “ Critical thinking and the evidence-based practice of sport psychology ”, Journal of Clinical Sport Psychology , Vol. 1 , pp. 9 - 22 , doi: 10.1123/jcsp.1.1.9 .

Nadelson , S. and Nadelson , L.S. ( 2014 ), “ Evidence-based practice article reviews using CASP tools: a method for teaching EBP ”, Worldviews on Evidence-Based Nursing , Vol. 11 No. 5 , pp. 344 - 346 , doi: 10.1111/wvn.12059 .

Newton , S.E. and Moore , G. ( 2013 ), “ Critical thinking skills of basic baccalaureate and accelerated second-degree nursing students ”, Nursing Education Perspectives , Vol. 34 No. 3 , pp. 154 - 158 , doi: 10.5480/1536-5026-34.3.154 .

Nibert , A. ( 2011 ), “ Nursing education and practice: bridging the gap ”, Advance Healthcare Network , viewed 3 May 2017, available at: https://www.elitecme.com/resource-center/nursing/nursing-education-practice-bridging-the-gap/ .

Oermann , M.H. , Kardong-Edgren , S. , Odom-Maryon , T. , Hallmark , B.F. , Hurd , D. , Rogers , N. and Smart , D.A. ( 2011 ), “ Deliberate practice of motor skills in nursing education: CPR as exemplar ”, Nursing Education Perspectives , Vol. 32 No. 5 , pp. 311 - 315 , doi: 10.5480/1536-5026-32.5.311 .

Papathanasiou , I.V. , Kleisiaris , C.F. , Fradelos , E.C. , Kakou , K. and Kourkouta , L. ( 2014 ), “ Critical thinking: the development of an essential skill for nursing students ”, Acta Informatica Medica , Vol. 22 No. 4 , pp. 283 - 286 , doi: 10.5455/aim.2014.22.283-286 .

Park , M.Y. , Conway , J. and McMillan , M. ( 2016 ), “ Enhancing critical thinking through simulation ”, Journal of Problem-Based Learning , Vol. 3 No. 1 , pp. 31 - 40 , doi: 10.24313/jpbl.2016.3.1.31 .

Paul , R. ( 1993 ), Critical Thinking: How to Prepare Students for a Rapidly Changing World , The Foundation for Critical Thinking , Santa Rosa .

Paul , R. and Elder , L. ( 2008 ), “ Critical thinking: the art of socratic questioning, part III ”, Journal of Developmental Education , Vol. 31 No. 3 , pp. 34 - 35 .

Paul , R. and Elder , L. ( 2012 ), Critical Thinking: Tools for Taking Charge of Your Learning and Your Life , 3rd ed. , Pearson/Prentice Hall , Boston .

Profetto-McGrath , J. ( 2005 ), “ Critical thinking and evidence-based practice ”, Journal of Professional Nursing , Vol. 21 No. 6 , pp. 364 - 371 , doi: 10.1016/j.profnurs.2005.10.002 .

Rahman , A. and Applebaum , R. ( 2011 ), “ What's all this about evidence-based practice? The roots, the controversies, and why it matters ”, American Society on Aging , viewed 3 May 2017, available at: https://www.asaging.org/blog/whats-all-about-evidence-based-practice-roots-controversies-and-why-it-matters .

Rieger , K. , Chernomas , W. , McMillan , D. , Morin , F. and Demczuk , L. ( 2015 ), “ The effectiveness and experience of arts‐based pedagogy among undergraduate nursing students: a comprehensive systematic review protocol ”, JBI Database of Systematic Reviews and Implementation Reports , Vol. 13 No. 2 , pp. 101 - 124 , doi: 10.11124/jbisrir-2015-1891 .

Robert , R.R. and Petersen , S. ( 2013 ), “ Critical thinking at the bedside: providing safe passage to patients ”, Medsurg Nursing , Vol. 22 No. 2 , pp. 85 - 118 .

Roberts , S.T. , Vignato , J.A. , Moore , J.L. and Madden , C.A. ( 2009 ), “ Promoting skill building and confidence in freshman nursing students with a skills-a-thon ”, Educational Innovations , Vol. 48 No. 8 , pp. 460 - 464 , doi: 10.3928/01484834-20090518-05 .

Romeo , E. ( 2010 ), “ Quantitative research on critical thinking and predicting nursing students' NCLEX-RN performance ”, Journal of Nursing Education , Vol. 49 No. 7 , pp. 378 - 386 , doi: 10.3928/01484834-20100331-05 .

Sackett , D. , Rosenberg , W. , Gray , J. , Haynes , R. and Richardson , W. ( 1996 ), “ Evidence-based medicine: what it is and what it isn't ”, British Medical Journal , Vol. 312 No. 7023 , pp. 71 - 72 , doi: 10.1136/bmj.312.7023.71 .

Saintsing , D. , Gibson , L.M. and Pennington , A.W. ( 2011 ), “ The novice nurse and clinical decision-making: how to avoid errors ”, Journal of Nursing Management , Vol. 19 No. 3 , pp. 354 - 359 .

Saldana , J. ( 2009 ), The Coding Manual for Qualitative Researchers , Sage , Los Angeles .

Scheffer , B. and Rubenfeld , M. ( 2000 ), “ A consensus statement on critical thinking in nursing ”, Journal of Nursing Education , Vol. 39 No. 8 , pp. 352 - 359 .

Stanley , M.C. and Dougherty , J.P. ( 2010 ), “ Nursing education model. A paradigm shift in nursing education: a new model ”, Nursing Education Perspectives , Vol. 31 No. 6 , pp. 378 - 380 , doi: 10.1043/1536-5026-31.6.378 .

Swing , V.K. ( 2014 ), “ Early identification of transformation in the proficiency level of critical thinking skills (CTS) for the first-semester associate degree nursing (ADN) student ”, doctoral thesis , Capella University , Minneapolis , viewed 3 May 2017, ProQuest Dissertations & Theses database .

Turner , P. ( 2005 ), “ Critical thinking in nursing education and practice as defined in the literature ”, Nursing Education Perspectives , Vol. 26 No. 5 , pp. 272 - 277 .

Twibell , R. , St Pierre , J. , Johnson , D. , Barton , D. , Davis , C. and Kidd , M. ( 2012 ), “ Tripping over the welcome mat: why new nurses don't stay and what the evidence says we can do about it ”, American Nurse Today , Vol. 7 No. 6 , pp. 1 - 10 .

Watson , G. and Glaser , E.M. ( 1980 ), Watson Glaser Critical Thinking Appraisal , Psychological Corporation , San Antonio .

Wittmann-Price , R.A. ( 2013 ), “ Facilitating learning in the classroom setting ”, in Wittmann-Price , R.A. , Godshall , M. and Wilson , L. (Eds), Certified Nurse Educator (CNE) Review Manual , Springer Publishing , New York, NY , pp. 19 - 70 .

Corresponding author

Related articles, all feedback is valuable.

Please share your general feedback

Report an issue or find answers to frequently asked questions

Contact Customer Support

A Bibliometric Analysis of Virtual Reality Research on Critical Thinking Skills and Student Responses using PLS-SEM

New citation alert added.

This alert has been successfully added and will be sent to:

You will be notified whenever a record that you have chosen has been cited.

To manage your alert preferences, click on the button below.

New Citation Alert!

Please log in to your account

Information & Contributors

Bibliometrics & citations, index terms.

Applied computing

Interactive learning environments

Recommendations

Personal space in virtual reality.

Improving the sense of “presence” is a common goal of three-dimensional (3D) display technology for film, television, and virtual reality. However, there are instances in which 3D presentations may elicit unanticipated negative responses. For example, ...

Bibliometric analysis of bioeconomy research in South Africa

This document provides an analysis of bioeconomy research in South Africa and it discusses sources of growth in the country’s bioeconomy literature in general. We performed bibliometric analysis as indexed in the Web of Science (WoS) for number of ...

Extending Virtual Reality Display Wall Environments Using Augmented Reality

Two major form factors for virtual reality are head-mounted displays and large display environments such as CAVE®and the LCD-based successor CAVE2®. Each of these has distinct advantages and limitations based on how they’re used. This work explores ...

Information

Published in.

Association for Computing Machinery

New York, NY, United States

Publication History

Permissions, check for updates, author tags.

• Bibliometric
• Critical Thinkings Skills
• Student Response’
• and Virtual Reality
• Research-article
• Refereed limited

Funding Sources

• DAPT-EQUITY Program, Indonesia Endowment Fund for Education, Ministry of Finance, the Republic of Indonesia

Contributors

Other metrics, bibliometrics, article metrics.

• 0 Total Citations
• 0 Total Downloads
• Downloads (Last 12 months) 0
• Downloads (Last 6 weeks) 0

View Options

Login options.

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

View options.

View or Download as a PDF file.

View online with eReader .

HTML Format

View this article in HTML Format.

Share this Publication link

Copying failed.

Share on social media

Affiliations, export citations.

• Please download or close your previous search result export first before starting a new bulk export. Preview is not available. By clicking download, a status dialog will open to start the export process. The process may take a few minutes but once it finishes a file will be downloadable from your browser. You may continue to browse the DL while the export process is in progress. Download
• Download citation
• Copy citation

We are preparing your search results for download ...

We will inform you here when the file is ready.

Your file of search results citations is now ready.

Your search export query has expired. Please try again.

• Search Menu
• Sign in through your institution
• Advance articles
• Editor's Choice
• Key Concepts
• The View From Here
• Author Guidelines
• Submission Site
• Open Access
• Why Publish?
• About ELT Journal
• Editorial Board
• Advertising and Corporate Services
• Journals Career Network
• Self-Archiving Policy
• Dispatch Dates
• Terms and Conditions
• Journals on Oxford Academic
• Books on Oxford Academic

Article Contents

Introduction, language teaching: more than teaching language, introducing the books, critical thinking, taboos and controversial issues in foreign language education, gender diversity and sexuality in english language education, antisocial language teaching: english and the pervasive pathology of whiteness, social justice and the language classroom, final thoughts, the reviewer, beyond elt: more than just teaching language.

• Article contents
• Figures & tables
• Supplementary Data

Steve Brown, Beyond ELT: more than just teaching language, ELT Journal , 2024;, ccae038, https://doi.org/10.1093/elt/ccae038

• Permissions Icon Permissions

Most people would agree that the world is currently facing significant problems: we have not yet recovered from a global pandemic, wars are killing thousands every day, climate change is starting to damage people’s livelihoods and well-being, wealth inequality is increasing, minority groups are being oppressed for a variety of reasons, the spread of disinformation and fake news makes it increasingly difficult to know what is true and what is not—the list goes on. While these issues are causing real concern for people’s current welfare and for the future of the planet, and are the focus of everyday discussion all around the world, whether they have a place in the ELT classroom seems to be a different matter. There have long been calls—including from academics within the native English-speaking ‘inner circle’—for issues such as these to be incorporated into ELT curriculum content (see, for example, Pennycook 1989 ), but these calls have tended to be stifled by an ostensible desire for the ELT profession to remain ‘politically neutral’. In recent years, however, the extent of the problems facing the world, and the sense of urgency that something needs to be done to reverse the current trajectory, have allowed discourses advocating more transformative pedagogies to move closer to the mainstream. To this end, a number of books have been published recently that explore some of the bigger issues of concern in the world today, and how they can (and, more to the point, should ) be addressed in ELT. This survey review examines five such books, offering individual reviews of each one and considering the contribution they make towards a significant shift in ELT—one that moves away from the preservation of the status quo, and becomes focused on the promotion of a critically aware, transformative, social-justice-oriented agenda.

As Pennycook (2021) suggests, since its emergence as an academic discipline, the dominant view in applied linguistics in the United Kingdom and the United States—which has tended to play a significant role in informing English language teaching practice—was that it was, or should be, a ‘value-free’ discipline. As such, it drew on theories in SLA to develop language teaching approaches and methodologies from a position of what was claimed to be ‘political neutrality’. This desire for an ‘unbiased’ approach manifests itself in the content of many widely recognized ELT teacher training courses, which historically have focused on methodology and classroom processes, but which devote less attention to discussion of, or reflection on, the impact that English language teaching can have on wider society. This ostensibly neutral position is also adopted by some of the bigger global publishing companies that produce ELT materials, which tend to focus on rather bland topics that are likely to be acceptable for discussion in any context or culture, such as sport, travel, and shopping. Rather than selecting topics that learners can critically engage with, the main focus has tended to be on developing understanding of systems of language, providing and practising lexical items that are deemed most likely to be of some use to people’s everyday, academic or professional lives, and developing skills that allow learners to use this language and lexis within some kind of meaningful context.

However, an alternative school of thought within applied linguistics and ELT is that a position of political neutrality is in fact impossible, let alone preferable. For Pennycook, wherever in the world it is being taught, ‘English is bound up in a wealth of local social, cultural, economic and political complexities’ (Pennycook 2017: 7), making the inclusion of such complexities within the curriculum unavoidable. The question, then, is not whether social and political issues should be incorporated into ELT, but which ones. Besides, the practice of communicating across cultures, which is, after all, the main purpose of learning an additional language, brings with it ‘the opportunity for emancipation from the confines of learners’ native habitat and culture, with the development of new perceptions and insights into foreign and native cultures alike’ ( Byram 1988 : 15). This implies that any kind of language learning should entail some critical engagement with cultural norms and values, with a view to developing the learner’s understanding of their position within society.

The above argument suggests, then, that there is more to language teaching than teaching language. Language cannot be taught in a vacuum; there needs to be content, and decisions need to be made about what that content should be. These decisions determine what learners are (and are not) able to talk about most proficiently in the language they are learning. What is more, if the same content is being used across the world through the use of globally published textbooks, or if ELT is required to comply with global outcomes and standards, the likelihood is that the world’s English-learning population will become highly proficient in using English in some contexts, and hugely deficient in using it in others. Littlejohn claimed that ‘one of the most worrying aspects of standardisation and centralisation is that by setting out what needs to be done, what should not be done is simultaneously dictated’ ( Littlejohn 2012 : 294). Given the narrow range of topics and issues that tend to be covered in most ELT curricula, it is fair to assume that the ELT profession is very effectively preparing people to talk about their favourite room or their last shopping trip, to write a short description of information presented in a chart, or successfully chair a business meeting. However, when it comes to critically engaging with issues that are having a damaging impact on people or the world, or exploring ways to transform society in ways that will address the problems we currently face, it seems that our profession is less effective. This is not to say that ELT textbooks fail completely to include some mention of social justice issues, or to allow scope for teachers to incorporate such discussions into their teaching if they wish to. However, such affordances are a far cry from making these big issues central to the learning process, and pushing them to the periphery of the curriculum—or erasing them altogether (how many trans people are represented in published ELT materials, for example?)—is, in itself, a political act. As Freire famously put it in his seminal book Pedagogy of the Oppressed : ‘washing one’s hands of the conflict between the powerful and the powerless means to side with the powerful, not to be neutral’ ( Freire 1996 : 122).

Over the years, concepts such as linguistic imperialism, native-speakerism, and, more recently, translanguaging have exposed some of the more problematic underpinnings of applied linguistics and led to the emergence of critical applied linguistics ( Pennycook 2021 ) as a kind of parallel discipline. Yet much of the teaching practice that goes on in ELT still seems inclined to take a very cautious approach towards anything that might challenge learners’ existing beliefs and attitudes. This makes ELT something of an outlier in the field of education. While the ELT profession has, by and large, sought to avoid ‘controversial’ issues in the classroom, ostensibly so as not to upset learners’ sensibilities, the world of teacher education beyond TESOL has a more sophisticated understanding of the non-neutral nature of education, and it is widely accepted that the adoption of some kind of ideological position is unavoidable. Schiro (2013) identifies four different ideological positions that can be taken in teacher education: a scholar academic position, which promotes the transmission of cultural knowledge via the institution’s academic disciplines; a learner-centred position, which supports learners developing according to their needs and interests; a social reconstruction position, which is concerned with developing an understanding of social issues, with a view to transforming society; and a social efficiency position, which focuses on creating competent individuals who can meet the social and economic needs of the society in which they live. It is accepted, though, that subjective partiality is a requirement in any educational approach. Any attempts to ‘remove’ ideology or ‘political interference’ from education and focus instead on evidence-based ‘facts’ (see, for example, Morris 2014 ) are merely attempts to impose one ideological position over others.

It is curious, then, that the myth of neutrality should be so prevalent in ELT. After all, it is well documented that education has been complicit in the Eurocentric colonization of thought, and the resulting epistemicide, or eradication of alternative thought systems, that has taken place in the Global South as a result of this colonization ( de Sousa Santos 2014 ). One might expect an area of education with such a global reach as ELT to take more interest in its own—potentially very damaging—ideological position in global society. In a context in which the impact of corporate globalization on societies and the climate is facing widespread criticism, such critical examination might lead to a more social-transformation-oriented approach in ELT, rather than an approach that has tended to facilitate, rather than challenge, the spread of neoliberal ideology ( Block et al . 2012 ).

With this in mind, it does seem that things are changing. As the precarious nature of humanity highlighted above in the introduction requires the legitimacy of the status quo to come under increasing scrutiny, many ELT professionals and academics have started to reject the fallacy of a politically neutral pedagogy. As dominant ideologies are exposed as having a damaging impact on the ELT profession and wider global society, there are increasing calls for the adoption of a position that seeks to effect positive social change, in line with Schiro’s (2013) social reconstruction position. It is in this climate, then, that this survey review is presented, covering five recently published books, all of which challenge the premise that language teaching is ‘simply’ a question of ‘teaching language’. Instead, these books all hold the view that ignoring key issues in global society is, to a large extent, a dereliction of duty. Like all educators, language teachers, materials developers, and curriculum planners have a responsibility to include within their content the development of capacities to gain a critical understanding of the world, with a view to transforming it for the better.

The first book in this review is Critical Thinking , by Gregory Hadley and Andrew Boon. Unlike the other books, this one focuses on a specific learning skill which, the authors argue, should be incorporated into ELT in all its forms. I then move on to provide reviews of two edited books: Taboos and Controversial Issues in Foreign Language Education , edited by Christian Ludwig and Theresa Summer, and Gender Diversity and Sexuality in English Language Education: New Transnational Voices , edited by Darío Luis Banegas and Navan Govender. Each of these books explores various ways in which ELT can incorporate issues that have hitherto been marginalized in most ELT contexts. This is followed by a review of Antisocial Language Teachi ng: English and the Pervasive Pathology of Whiteness by J. P. B. Gerald, a single-author monograph which uses the construct of whiteness as a lens through which to analyse the ELT profession. Finally, I review Deniz Ortaçtepe Hart’s Social Justice and the Language Classroom: Reflection, Action and Transformation , which explores language teaching from a social justice perspective. I conclude by drawing together some of the common themes in the five books and reflecting on the contribution that they make to the ELT profession.

Before I begin this review, it is perhaps important to make a clear distinction between critical thinking and critical theory. Critical theory is an ontological position that views reality as socially constructed by hegemonic forces designed to locate power in ways that privilege some groups over others. Critical thinking (CT), on the other hand, is a discipline that is concerned with analysing data, claims, arguments, etc., with a view to identifying their underlying premises and assumptions, and, therefore, the extent to which they can be believed. Although CT is often applied by critical theorists, CT does not necessarily support critical theory, and can even be applied to critique it. Although critical theory heavily informs the epistemological assumptions underpinning the other books in this review, Hadley and Boon’s Critical Thinking is less concerned with adopting a specific epistemological or ideological position in ELT. Rather, the focus is on giving students the skills to understand the logic—or lack of logic—underpinning the texts and other sources of language content that they are exposed to. This book is part of a series entitled ‘Research and Resources in Language Teaching’, which seeks to bridge the often-criticized gap between research and language teaching practice. Although the series and book titles do not specify this, it is very much a book for teachers of English, with references made to English language, ELT, and EAP at various points throughout. Compared to the other books in this review, it has a very practical focus. Its main aim is to provide ideas for encouraging critical thinking among learners within a language teaching context, following what it calls on the back cover a ‘dynamic framework’.

As with all the books in this series, this volume is organized in four parts. Part I, ‘From Research to Implications’, starts with an introduction that offers a definition of critical thinking followed by a section that answers some common questions about CT, allowing the authors to address common sceptical views that readers may have about the importance of including CT within the language curriculum. Three key issues in CT are then addressed: argumentation, logical fallacies, and the role of problem-solving in externalizing or actualizing CT. These sections provide an overview of the research underpinning the principles of CT, with diagrams and tables supporting examples of different types of argumentation and analysis. Part II, entitled ‘From Implications to Application’, then takes some of the concepts and frameworks for CT introduced in Part I, and provides no fewer than 93 practical activities that aim to introduce these various concepts to learners.

Part III, ‘From Application to Implementation’, is concerned with methodology, and how the activities introduced in the previous part can be integrated into the curriculum. Two possible approaches are presented: first, how to create a self-standing CT course for English language learners; and second, how to supplement an existing course with CT activities. It also offers suggestions for adapting the activities to suit specific contexts and incorporate critical thinking more broadly into the teaching and learning process. A final section in this part offers some ideas for teachers to develop their own CT materials. Part IV, entitled ‘From Implementation to Research’, presents research and practice as part of a cycle and encourages teachers to use their own teaching to contribute to literature on the role and use of CT in ELT. This part offers a robust, mixed-methods research approach that teachers can apply to their own contexts, and offers some ideas for how to share their research findings. This part highlights the non-measurable nature of CT, and how an oversimplified, quantitative approach to research can only provide a partial, incomplete understanding of the phenomenon. It is a useful reminder that research into critical thinking is far from straightforward.

Although this book differs from the others in terms of its content and practical application, it still fits within the remit of this review as it is grounded in the principle that English language teachers have a responsibility to teach beyond language—to develop skills in critical literacy and analysis so that students can deal more effectively with the vast body of information that is thrust at them on a daily basis. ‘Our world’, the authors tell us in the introduction, ‘is experiencing a crisis of the mind’ (p. 4). They go on to say that ‘whether online, on the street, at home, or in the marketplace, we encounter arguments and propositions intended to prey upon those with an undeveloped sense of critical thinking’ (p. 5). As you might expect, this claim is then backed up with a solid, evidence-based argument which illustrates not only the prevalence of false or misleading information that is out there; it also reveals how CT features a lot less in education than it used to. In short, CT is needed more than ever, but it is being taught less.

The book introduces and uses a large amount of terminology about argumentation and different ways of analysing the truthfulness of certain statements. This is simultaneously very useful and somewhat bewildering to those readers who are not familiar with it (I include myself as one of those readers). Terms such as logical fallacy, false dilemma, and circular reasoning are used throughout the book, and may take some getting used to. However, these terms are clearly explained, and examples demonstrate that they relate to common features of everyday discourse—features that most readers will already be familiar with. The use of this terminology to describe strategies for argumentation and analysis highlights that CT is very much an established discipline in and of itself. It is clearly very useful, and in some ways reassuring, to know that teachers can draw on and apply these concepts in their teaching. The theory and research-focused parts of the book are supported with useful diagrams and figures that clearly illustrate the concepts being introduced and explored.

However, this book goes well beyond providing a theoretical overview of CT. The sheer number and range of ready-made practical activities make it very useful for any teacher who wishes to incorporate critical thinking into their practice. A possible problem with this is that it makes the book very lopsided in its structure, which is likely to affect how it is used by readers. Parts I, III, and IV make up a combined total of just 68 pages; in fact, Part III, which is concerned with how to implement CT into the ELT curriculum, is only eight pages long. By contrast Part II, which contains all the practical activities, is 226 pages long. This means that there may be a tendency for teachers to treat the book as if it was just a collection of activities: they might skim through Part I, select practical activities from Part II as stand-alone materials to supplement their own teaching, and never get to Parts III and IV. But the fact is that the book is much more than a collection of supplementary activities, and Parts I, III and IV are arguably the most valuable as they allow teachers to get a deep understanding of the importance of CT and its application to ELT. These parts also encourage teachers to develop their own skills in contributing to a more CT-oriented approach to ELT, by offering a research framework and practical tips on sharing ideas and findings. It would be a shame if these features of the book were to be overlooked. Perhaps the more theoretical and research-oriented ideas could have been woven throughout the practical section. This might have helped readers to develop an understanding of the theoretical underpinnings within each activity, and could also have illustrated how teachers themselves can contribute to the cycle from research to practice and back to research again.

Having said that, it would be unfair to criticize the book for being too practical—its practical application is a key strength in many respects, and it also distinguishes this book from the others in this review. Whether they read it all or dip into certain parts, this book offers scope for teachers to develop their capacities to understand and apply CT themselves, to introduce ready-made activities into the classroom to develop the CT of their students, to incorporate CT more coherently into their curriculum, and to conduct research related to CT in their own practice. The argument that the authors provide in Part I for incorporating critical thinking into ELT is a compelling one, and it justifies the book’s contribution to the field.

As previously mentioned, a long-standing criticism of ELT has been its tendency to avoid topics and issues that could be regarded as controversial or offensive. This tendency is compounded by a global approach to methodology and materials design; many large publishing corporations produce textbooks and other materials that aim to appeal to as wide a market as possible, in as many areas of the world as possible. The result of this is a tendency for ELT materials not only to follow a similar format and teaching approach, but also to stick to a very narrow range of topics. Materials published for a global market tend to avoid any mention of what are commonly referred to as the PARSNIP topics: politics, alcohol, religion, sex, narcotics, isms, and pork ( Gray 2002 ). An obvious problem with this universal, one-size-fits-all approach is that topics and issues which are not taboo in many parts of the world (and in some cases central to everyday life) are often omitted because they could cause offence in other countries or regions. For example, while the mention of pork may offend the sensibilities of many learners in the Middle East, it is a key element of people’s diets in a large number of other countries. In China the pig is even regarded as a symbol of prosperity and good fortune. Another example, from my own experience teaching ESOL learners in the United Kingdom, is the topic of narcotics. There is widespread, often public use of drugs in many areas of the United Kingdom, including areas where my learners were living, and a lot of these learners were unaware of the legalities and implications of drug use, addiction, and dependency. The social impact of drugs therefore became an important topic in our syllabus, as it gave the learners necessary knowledge to understand what was happening in their own neighbourhoods, to make informed decisions themselves, and to advise and give appropriate guidance to their children.

Another key problem with this controversy-averse approach to materials design is that the avoidance of issues such as politics, religion, racism, or gender discrimination guarantees that any discussion of these social issues, which could motivate engaged discussion, lead to effective language development, and provide opportunities to develop critical thinking skills, is effectively removed from the curriculum. This erasure is even more problematic when examined through the lens of critical pedagogy, which seeks to create ‘an informed, critical citizenry capable of participating and governing in a democratic society’ ( Giroux 2011 : 7). In order to become informed in this way, it is necessary for classroom discussions to incorporate these issues, challenge existing beliefs, and explore alternatives. This requires a focus on developing critical literacies: connecting with the learners’ own experiences, applying critical consciousness within the context of those experiences, and responding with follow-up action.

The relevance of critical literacies to language teaching is, or should be, self-evident, as it creates scope to teach useful language at the same time as focusing on content that is necessarily relevant to learners’ lives, and the area of critical language pedagogy has emerged as a field in its own right ( Crookes 2021 ). This book applies the principles of critical language pedagogy to explore how taboos and controversial issues can—and should—be incorporated into language education.

The book starts with an introductory chapter from the editors, followed by twenty chapters from contributing authors, which are divided into three parts. The introduction provides an overview of critical language pedagogy, a key tenet being ‘that beliefs and practices in society, which are related to power and the domination of certain groups over others, should be questioned and challenged’ (p. 5). It then discusses the PARSNIP policy in ELT materials design, before proposing the concept of ‘taboo literacy’ (p. 11), offering a pedagogic framework that can guide teachers towards effective incorporation of taboo topics into their teaching.

Part I (Chapters 2–6) is entitled ‘Theoretical Considerations and Insights’. It starts with a chapter by John Gray on the political economy of taboos, which explores how the ‘erasure’ or proscription of some topics, particularly sexual orientation, race, and class, allows the unfettered promotion of a more easily marketable, neoliberal world-view. This is followed by a chapter by Daniel Becker which presents a case for making taboos visible in the English language classroom. David Gerlach and Mareen Lüke present ideas for bringing critical approaches into language teacher education, and Grit Alter and Stefanie Fuchs provide a chapter discussing what constitutes a taboo topic in the first place, and analysing some ethical and practical issues that emerge in regard to their use in the foreign language classroom. The final chapter, by Aline Williams, focuses on the importance of resilience as a prerequisite for the application of critical language pedagogy.

Part II, ‘Empirical Enquiries’, consists of three chapters that present research related to taboo topics in language education. A chapter by Theresa Summer and Jeanine Steinbock presents findings from a study of adolescent learners’ perceptions of taboo topics in the English language classroom in Germany. In this study, participants broadly agreed that it is necessary for educators not only to include such topics in the curriculum, but also to pay attention to what the learners themselves have to say about them. Then Theresa Summer and Christian Ludwig, the book’s editors, provide a chapter on a survey of pre-service teachers’ attitudes to taboos in language teacher education which reveals that, while student teachers support the inclusion of taboo topics in the ELT curriculum, they also identify numerous challenges in doing this. These findings imply a need for a more systematic approach in teacher education that addresses the challenge of incorporating taboo and social justice issues in ELT. With this in mind, the following chapter by Christine Gardemann presents a study related to developing a pedagogic alliance in the ELT classroom between teachers and learners—‘a reciprocal relationship of mutual trust’ (p. 93).

Part III, entitled ‘Specific Taboos and Practical Examples’, is the longest section of the book, with twelve chapters exploring different taboo topics and their application in the language classroom. The range of topics discussed in these chapters is broad, and includes some that may not immediately spring to mind as taboo. Chapters on the issues of disability (by Katrin Thomson), mental health (Christian Ludwig and Veronika Martinez), swearing and taboo language (Valentin Werner), racism (Silke Braselmann), and human trafficking (Christian Ludwig) provide useful insight into how these issues can be incorporated into the language classroom. Other chapters are concerned with topics that are, in a sense, so taboo that they are unlikely to even be named as taboo topics, such as Maria Eisenmann’s interesting discussion of critical animal pedagogy and the incorporation of animal rights as a taboo topic in foreign language education, and Roman Bartosch’s chapter on death and extinction. Other chapters focus on taboo topics more generally and their application in different contexts or using certain media, such as Anchala Amarasinghe and Susanne Borgwaldt’s chapter using feature films to develop taboo literacy in the Sri Lankan context, a chapter from Sandra Stadler-Heer on taboo topics for South African students of European literature, and Janina Reinhardt’s chapter on using television series to legitimize discussion of taboo topics. Part III ends with a chapter from Eva Seidl that is concerned with translator and interpreter training and the role of taboo topics in developing agency.

A final chapter by Tyson Seburn, entitled ‘International Perspectives on Taboos in Foreign Language Education’, provides a conclusion to the book, drawing key themes together and discussing the relevance of critical pedagogy in foreign language teaching. This chapter ends with a call for both global and local teacher education programmes to incorporate these practices in order for ‘those in our profession, including our learners, to tackle injustice and build connections through communicative critical pedagogy’ (p. 259).

A key strength of this book is its logical layout and the way the chapters are grouped and sequenced. The theoretical overview of critical language pedagogy that the editors provide in the introduction, followed by the presentation of a framework for developing taboo literacy, is a helpful starting point that makes the book accessible to teachers who may be interested in incorporating a wider range of topics and issues in their teaching, but are unfamiliar with the background to the teaching of critical literacies. An awareness of the theoretical underpinnings is developed further throughout Part I, meaning that readers arrive at the research chapters with a clear understanding of the background informing the studies. The chapters that make up Part III could be dipped into individually by readers who are interested in specific taboos or learning contexts, but reading them all highlights the broad range of topics that currently tend to be omitted from the language curriculum, as well as providing useful insights into how it is not only possible, but highly beneficial, to bring discussion of these issues into our teaching. The diverse issues and contexts covered in Part III are supported by practical examples that are clearly and explicitly grounded in the principles of critical language pedagogy, demonstrating its universal relevance and applicability.

The closing chapter enhances the cohesion of the book further, stressing the need for critical pedagogy to be incorporated into teacher education. This message is prevalent throughout the book and is an important one. For the contributors to this book, the effective use of taboos and controversial issues is not simply a question of replacing the usual bland topics with racier ones. It involves adopting different pedagogies, and applying a new mindset from that which tends to be prescribed in most language teaching contexts. Keep things light, don’t cause offence, keep the students happy—mantras like these are drummed into language teachers from their initial training onwards, leading to the belief that any methodology designed to explore social problems, challenge existing beliefs, or encourage community action seems like a dangerous act of subversion. As this book convincingly argues, however, critical language pedagogy not only develops learners’ critical understanding of social issues, but also enriches and develops capacities for language development in areas that are highly relevant to their needs and, sadly, neglected in many language learning contexts.

Issues related to gender and sexuality in ELT have been a source of debate for quite some time. ELT textbooks in particular have been criticized for presenting only stereotypical gender roles and failing to represent people whose lifestyles and relationships do not conform to hetero- and cis-normative values. The narrow representation of gender and relationships in ELT materials has been highlighted as extremely unhelpful—to say the least—in most contexts, for a number of reasons. For John Gray, LGBTQ+ invisibility ‘means that lesbian and gay [and other] students are either silenced or forced into challenging the ways in which they are positioned’ ( Gray 2015 : 187). The failure to confront and criticize sexist attitudes and patriarchal structures has a similar impact with regard to gender identity, reinforcing beliefs and behaviours that perpetuate gender inequality while at the same time delegitimizing any challenge to the heteronormative, patriarchal status quo. Also, for migrant learners of English who have moved from socially repressive countries to new environments where LGBTQ+ and women’s rights are enshrined in law, the need to incorporate discussion of these issues and their social acceptability in the classroom is self-evident, and their failure to do this makes many popular ELT textbooks inappropriate for this context ( Brown and Nanguy 2021 ).

The lack of inclusion of a wider range of sexualities and gender roles in most popular ELT materials also affects teaching methodology, legitimizing the unhelpful narrative that discussing lifestyles which challenge some students’ beliefs, or have the potential to cause offence, is a bad idea and must be avoided at all costs. These materials encourage teachers to play safe through the use of innocuous topics, pandering to normative values that may be dominant but are also damaging. A new book that explores the issues of gender diversity and sexuality in ELT, which aims to develop queer critical literacies in teachers, and which offers ideas for their incorporation into the English language classroom, is therefore very welcome.

Although the contents page presents an introduction followed by twelve separate chapters, the editors tell us in their introduction that the book chapters are organized into three parts: ‘Teaching for Gender and Sexuality Diversity’, ‘Navigating Gender and Sexuality Diversity’, and ‘Interrogating Resources for Gender and Sexuality Diversity’. The first part includes chapters that cover initial teacher education in South Africa and Scotland (by Grant Andrews and Navan Govender), post-secondary education (Antonella Romiti and Jessie Smith), ELT in primary education (David Valente), and the teaching of younger learners in the private sector (Germán Canale). The chapters on navigating gender and sexual diversity present studies as wide-ranging as Gulsah Kutuk’s exploration of the effects of stereotyping on male Turkish learners, Lian Cao’s study of online dating as a source of informal language learning in Canada, and Shin-ying Huang’s analysis of Taiwanese university students’ reflections on, and reactions to, representations of gender and sexuality that seek to shape their identities. The final part, which is concerned with materials and resources, contains two chapters on gender and sexuality in ELT textbooks. The first, by Suha Alansari, focuses on gender representation in global textbooks and their localized versions, and finds that both versions are primarily concerned with promoting ‘the neoliberal conception of the global citizen’ (p. 156). Chris Richards then presents findings from a study that included a quantitative multimodal analysis of gender and sexuality representations in course materials, followed by the qualitative analysis of data collected from interviews with teachers. This study corroborates findings from previous research in identifying stereotypical gender representations and exclusively heteronormative presentations of relationships and families, although it does identify a progression towards more positive representation of women and a desire among teachers to adapt or subvert the negative representations they encounter. This chapter is followed by a critical analysis of language teacher education in Germany by David Gerlach, and a study of gender diversity in an online ELT programme in Bangladesh by Sayeedur Rahman and Mohammad Hamidul Haque. Joanna Pawelczyk’s final chapter is entitled ‘New Transnational Voices on Gender Diversity in English Language Education: Moving Forward’. This chapter refers back to some of the studies presented in the previous chapters, stressing the promotion of dominant ideologies surrounding gender roles and sexuality in ELT materials, and the concurrent erasure of non-heteronormative values and lifestyles. This creates a ‘transnational struggle to recognize a diversity of gender and sexual identities in English language education systems’ (p. 212), reflecting the global nature of some studies, the role of intercultural communication, and the fact that English language education needs to be ‘a truly diverse and inclusive space for all students’ (p. 216).

A small but important criticism of the book relates to the contents page, which does not separate the chapters into the three parts identified by the editors in the introduction. Doing this would allow readers to easily identify the book’s structure, making it more accessible. Nevertheless, there is a lot to commend this book for, not least its diversity of content. The editors state very clearly in the introduction their intention to ensure the book offers perspectives that go beyond those that are normally found in WEIRD (Western, educated, industrialized, rich, democratic) countries. This is certainly achieved, with writers from a wide range of backgrounds and the inclusion of studies conducted in countries as diverse as Taiwan, Bangladesh, and Argentina. The diverse contexts add to the richness of the content, allowing the book to convey the complexity and variety of the issues and concepts it seeks to address. The inclusion of diverse contexts also demonstrates how different issues have different degrees of relevance in specific locations; this all helps to make the book a very interesting read. However, it also presents a challenge for the book as a whole to maintain a consistent message. Highlighting the range of issues to consider in different teaching contexts is, of course, a useful message in itself, especially when a key underpinning point of queer critical literacy is that gender diversity and sexualities are far more complicated than the traditional binary identities (male/female, gay/straight, married/single, etc.) that the heteronormative patriarchy has socially conditioned us all to accept. But it does mean that the ideas being conveyed in some chapters could be seen as not particularly relevant to the book’s overall message. For example, Gulsah Kutuk’s study into the impact of gender stereotyping on the listening performance of male Turkish university students relates to the perception that women are better at learning languages than men, and is concerned with exploring whether this leads to negative self-perceptions and diminished performance among male learners of English. Although there may be evidence for this, concerns that male learners are victims of gender stereotyping seems—to this reader at least—to be rather low priority compared to the extent to which stereotyping affects women and people with queer and/or trans identities in society. Similarly, although Suha Alansari’s study of global, regional, and localized (Saudi) ELT textbooks throws up some interesting findings—in particular the prevalence of neoliberal values in all three—readers in societies that broadly accept the principles of gender equality may wonder if it might be more useful to explore the lack of representation of sexualities in these materials.

Of course, by forcing readers to ask themselves questions such as these, the book is achieving its aim of encouraging the development of critical literacies. From my own (WEIRD) perspective, some of the issues explored in the book may seem somewhat out of step with the liberal and inclusive values that are widely accepted in my context. In the United Kingdom, for example, debates about gender equality tend to centre around how to remove barriers to achieving it, rather than whether or not it is a good idea. By developing my understanding of important gender and sexuality issues in other parts of the world, the book has helped me to identify different locations of struggle. This in turn encourages me to be less intolerant of other perspectives. Readers with similar backgrounds to mine are likely to benefit from the book in similar ways, while others will appreciate having the issues that are relevant to their contexts incorporated into this important discussion. This can lead to Pawelczyk’s call for action in the final chapter to go beyond creating discourses about normalizing diversity, and instead to start ‘talking about doing normalizing diversity in English language education … [which] … needs teachers and educators who are equipped with critical skills … to include and respect the (intersectional) identities of all students’ (p. 216). These critical skills and inclusive approaches have not traditionally been the focus of TESOL education courses, and this call for action is a refreshing change.

While the first three books in this review offer some analysis of why certain issues or skills are absent or poorly dealt with in ELT, they are mostly concerned with filling a gap in the discourse by offering ideas and examples for incorporating new content that benefits learners in ways that go beyond the development of linguistic and communicative competence. I now turn to a book that is primarily focused on exploring how a socially constructed phenomenon has had (and continues to have) a damaging impact on ELT. By the far the most polemic of the five books being reviewed, J. P. B. Gerald’s Antisocial Language Teaching: English and the Pervasive Pathology of Whiteness explores the concept of whiteness and its role in shaping ELT as a field that supports discourses of white supremacy, while at the same time delegitimizing blackness. To do this, the author draws on critical race theory, critical whiteness studies, and disability critical race studies to offer a (re)presentation of ELT that many people within the profession’s establishment will be uncomfortable with, and some may be reluctant to accept.

As a concept, whiteness goes beyond the notion of belonging to a specific racial group. In the United States, in particular, it also includes various characteristics and behaviours that tend to be associated with being part of that racial group. Unlike the concept of ‘race’, which limits itself to categorizing people according to perceived physical and (ostensibly) inherited behavioural differences, whiteness focuses on specific characteristics and behaviours, commonly associated with dominant social groups, which are widely presented as neutral—the default or norm from which others deviate—meaning it can be a difficult concept for white people to grasp. Through the lens of whiteness, deviation from this ‘norm’ implies deficiency as well as difference. According to Olcoń, ‘Whiteness therefore implies a power structure, an ideology and a racial identity which confers dominance and privilege’ ( Olcoń 2023 : 4). The area of scholarship known as critical whiteness studies is primarily concerned with exploring the concept of whiteness and exposing its damaging social impact, both in history and in contemporary life. In defining the key concepts of Antisocial Language Teaching , the author equates whiteness and white supremacy, saying there is no functional difference between the two and that ‘whiteness was created to be supreme, as a protection from the oppression that others deserve because of the groups into which they have been placed’ (p. 6).

The main focus of the book is on pathologizing whiteness in ELT. Gerald does not just explore examples of whiteness and problematize their impact on the profession. He presents whiteness as an illness, a disorder that has had a chronically debilitating effect that ‘renders the industry callous, corrupt and cruel’ (back cover). To do this, Gerald uses the American Psychology Association (APA)’s seven criteria for diagnosing antisocial personality disorder to analyse the ideologies and institutions that shape the global ELT profession. According to the book these criteria are used as a rhetorical device, but in effect they form an analytical framework for identifying ‘symptoms’ of whiteness. The use of this framework allows Gerald to present ELT as imperialistic, elitist, racist, dishonest, irresponsible, immoral, and a number of other things besides.

There are three main parts to the book. Part 1 is entitled ‘Disorder’, with seven chapters that mostly provide a historical overview of the social construction of whiteness, its associations with capitalism, its contrast with the (also socially constructed) concept of blackness, and its role in the hegemony of white varieties of English as ‘standard’ and other varieties as ‘inferior’. Part 2 uses the APA’s seven criteria to ‘diagnose’ ELT as an antisocial practice. These criteria are:

Failure to conform to social norms concerning lawful behaviors, such as performing acts that are grounds for arrest.

Deceitfulness, repeated lying, use of aliases, or conning others for pleasure or personal profit.

Impulsivity or failure to plan.

Irritability and aggressiveness, often with physical fights or assaults.

Reckless disregard for the safety of others.

Consistent irresponsibility, failure to sustain consistent work behavior, or honor monetary obligations.

Lack of remorse, being indifferent to or rationalizing having hurt, mistreated or stolen from another person. (pp. 63–109, passim ).

In his analysis, Gerald draws on a mix of existing literature and anecdotal evidence to critically examine common practices in ELT such as the hiring of unqualified teachers based on their ‘nativeness’, the exploitation of teachers through discrimination and deprofessionalization, the ELT profession’s platitudinous response to the Black Lives Matter movement, its role in the spread of colonialism and linguistic imperialism, and the use of ‘standard’ English to pathologize all other varieties and their speakers. Part 3 is entitled ‘Treatment’, and contains just two chapters followed by a conclusion. The first of these chapters offers, as a specific example of how to ‘treat’ ELT, an account of a course that Gerald developed and delivered during the Covid-19 pandemic, entitled ‘Decoding and Decentering Whiteness’. This is followed by a chapter called ‘Prosocial Language Teaching’, which presents seven ideas for decentring whiteness in ELT.

Using the APA’s criteria for antisocial personality disorder to ‘diagnose’ ELT as suffering from a medical condition is, in my view, a very effective framework for presenting Gerald’s thesis. It allows Gerald to turn the tables on ELT, by presenting something that is usually regarded as healthy and innocuous as being, in fact, seriously unwell and very destructive. In ELT, native-speaker varieties of English are idealized, heterosexist lifestyles and cis-gendered identities are normalized, neoliberal ideology and myths of meritocracy are presented uncritically, and racist perceptions of teachers who ‘don’t look like native-speakers’ are pandered to by employers. In this antisocial reality, any alternative positions are seen as subversive and potentially damaging: the acceptance of ‘non-native’ models of English will lead to a loss of standards; the inclusion of LGBTQ+ issues in materials will upset cis-gendered heterosexual learners; questioning the belief that hard work leads to success devalues individualism and entrepreneurialism; and recruiting teachers who do not conform to the idealized (i.e. white ‘native-speaker’) English teacher may lead to financial loss. The use of antisocial personality disorder traits to deconstruct the ELT exposes it as a particularly damaging profession–one that not only fails to address what is wrong with the world, but is actively complicit in causing harm. The idea behind this rhetorical device is good and it is, to a large extent, successful. However, the use of whiteness as the main focus of the book’s critique means that the author is less able to explore areas of ELT that do not easily fit into the APA’s seven criteria, or are less directly related to whiteness. Some of the chapters in Part 2 do not focus on aspects of ELT that you might expect. For example, it is not particularly clear why ‘standard’ varieties of English are the main focus of the chapter relating to reckless disregard for the safety of others. Certainly, negative perceptions of ‘non-standard’ or racialized varieties of English create discriminatory environments that could cause harm to users of those varieties, and I do not intend to belittle this issue. I do feel, though, that other practices, such as publishing companies’ delegitimization of gay people in order to increase market share, is surely a more obvious example of powerful forces in ELT having a reckless disregard for the safety of others. In addition to the symptoms that Gerald attributes to the damaging influence of whiteness in ELT, there are other symptoms that could perhaps be more effectively examined if other factors were more central to the discussion.

One thing that—for me, at least—is missing from this book is an explicit and critical examination of the role of capitalism in ELT. Gerald does make some historical references to capitalism, identifying its links with colonialism and the slave trade. But the role of capitalism in twenty-first-century ELT is connected to many of the problems that Gerald is referring to. A universal and UK/US-centric approach to methodology and materials content has spread globally through accredited, commercial TESOL qualification providers and publishing companies who seek to maximize efficiencies and profits by globalizing their products. The application of (white) ‘native-speaker’ models of English as ‘standard’ is promoted by commercial examining bodies that also aim to have a global reach, and is being exploited by ‘accent reduction training’ entrepreneurs who peddle the myth that it is possible to have a ‘neutral accent’. The motivation to employ white, ‘native-speaker’ teachers is driven by school owners feeling the need to pander to the beliefs and demands of their learners, however misguided and racist they may be. Much of what is wrong with ELT today can quite easily be linked to capitalism and the profit motive. The dismantling of the status quo that the author is calling for, therefore, is unlikely to happen if ELT continues to function within a capitalist paradigm.

Perhaps Gerald would argue that capitalism is itself a symptom of whiteness: a concept devised by white people and designed to control, dominate, and exploit the more vulnerable groups in society. However, the obvious links between whiteness and capitalism are not made as clear as they could be in the book. In fact, Part 3 (entitled ‘Treatment’) seems, if anything, to be largely pro-capitalist. The first chapter in this part presents a course that the author designed and delivered—commercially—online. In this chapter and, seemingly, without any irony, Gerald refers to his students as ‘clients’ (e.g. p. 117). In the following chapter, he proposes that we buy materials from ‘more anti-oppressive companies’ (p. 148), implying an acceptance that we must continue to rely on publishing companies for teaching resources:

The only reason that the publishers continue to create materials that reify these harmful ideas is because they think it’s the most profitable way to operate. Even though we are currently stuck existing within racial capitalism, we do still have the power to band together to affect the almighty market. (p. 148)

Of course, using one’s limited power as a customer to buy less-damaging materials in the hope that this will influence the market is not a bad idea, but only if one believes that consumers can control the market, which entails a belief in neoliberal ideology. It is rather surprising that such a recommendation should be made in a book that, according to one of the testimonials on the back cover, ‘makes a passionate case for demolishing the status quo in English language teaching’. The idea of decentring whiteness while embracing the principles of capitalism at the same time could be seen as rather naive. And in any case, even if market forces did lead to an eradication of ideology associated with white supremacy in ELT materials, other damaging factors associated with the commodification of language teaching would continue to prevail, and the status quo would remain far from demolished.

Having said all of this, there is still relatively little published literature that is concerned with the concept of whiteness in ELT, which makes this an important book in terms of its contribution to the wider discourse problematizing deeply embedded structures and ideologies. Part 2 provides a compelling argument that the ELT profession is seriously unwell, and in need of treatment. What is more, although he does not provide an explicit guide for designing a course that addresses whiteness in the way that Hadley and Boon do for a course in critical thinking, the author’s account of his own course, ‘Decentering and Decoding Whiteness’, provides a useful template that teacher educators can follow to bring anti-whiteness education into their own practice. The syllabus and overall approach are very informative, and could perhaps be adapted and used in a wider range of contexts. This chapter also includes sections depicting how some of the students (or ‘clients’) on this course coped with and reacted to the experience. These sections offer some interesting insights into the application of Freirean principles of critical pedagogy, including some uncomfortable and revelatory critical incidents as students gained a deeper critical understanding of their own positions in society. These accounts suggest that developing their own understanding of whiteness also developed the students' capacities for using antiracist language pedagogies in their own teaching.

Indeed, it may be that my criticism of the book—that it focuses too narrowly on whiteness in ELT and fails to address other, wider issues—exposes my own limited understanding of whiteness as a concept. Coming as I do from a relatively privileged, white British background, it is perhaps difficult for me to appreciate the all-pervasive nature of whiteness, particularly from a North American perspective. I understand white supremacy to be a discriminatory ideology that creates structures which oppress, marginalize, and disadvantage certain people. But the same could be said for heteronormativity, or patriarchy, or ableism, or linguicism. These ideologies can intersect, compounding the extent to which some people are oppressed, but I find it difficult to see how one single ideology, whiteness, is the cause of all the others.

For the final book in this review, I turn to a publication that is concerned with the broad and somewhat nebulous concept of social justice. The issue of social justice and whether it should be a goal in education has proved to be a surprisingly divisive issue. Some educators and policymakers regard the social justice movement as an attempt to indoctrinate learners through the promotion of dangerous ideologies, while others regard it as a means of giving learners the skills to challenge existing, dominant ideologies, which are themselves dangerous in their promotion of structural inequality and oppression. As mentioned elsewhere in this review, ELT has tended to shy away from the inclusion of social justice issues on the basis that they could be interpreted as controversial or create divisiveness in the classroom. Deniz Ortaçtepe Hart’s Social Justice and the Language Classroom: Reflection, Action and Transformation challenges this prevailing attitude by presenting an overview of language teaching for social justice—not simply as a list of topics to focus on now and again, but as an approach that informs all aspects of language education.

The book is divided into three parts. Part I is entitled ‘Language Teaching for Social Justice’, and the four chapters included here introduce key concepts that inform the entire book, particularly hegemony and intersectionality, which are covered in some detail in Chapter 1. Hegemony, the idea that oppressive structures are normalized to ensure that power is retained by those who hold it, is central to the understanding of social injustice as a structural phenomenon, rather than as something that can be attributed to individual behaviours. The main lens through which Ortaçtepe Hart uses to analyse and discuss social justice is intersectionality, a concept used to explore how different features of people’s identities—such as race, gender or disability—combine and influence the types of social injustice they may be affected by. Chapter 2 provides a discussion of social justice education, and introduces an approach to teaching called ‘transformative liberatory education’ (p. 27), which is based heavily on the work of Freire and education for ‘critical consciousness’ ( Freire 2013 ). This approach is all about developing an understanding of the societal structures that locate learners in certain positions in society, and to identify ways of challenging any injustices that they identify. Chapter 3 focuses in on social justice language education, identifying solid justifications for taking a social justice approach to language teaching. These justifications may resonate particularly strongly with readers who are involved in ELT, which has clear associations with colonialism, imperialism, and linguicism. Chapter 4 is concerned with curriculum, exploring the politically charged nature of decisions that affect all aspects of curriculum design and delivery, and justifying the need for social justice to inform all aspects of curriculum rather than being included as a single topic within a syllabus, or tagged on at the end of each lesson.

The three chapters in Part II each focus on critical themes and frameworks: neoliberalism, social class, and anticlassism; race, ethnicity, and antiracist language pedagogy; and social justice pedagogies for all gender and sexual identities. Each chapter starts with a critical analysis of each theme, and then moves on to present some alternative approaches and models. This is perhaps the ‘meatiest’ part of the book; it deconstructs the themes and exposes them as profoundly unjust, as well as deeply embedded. In Chapter 5 the author argues that the pervasive ideology of neoliberalism and the classless society has erased class from any discussion of social justice. Chapter 6 explores how race and ethnicity are socially constructed, and how racialized ideologies are embedded within the curriculum, while racism itself is often presented as a problem with some individuals, rather than in a structural sense. In Chapter 7, standard representations of sex and sexuality are revealed to be overwhelmingly patriarchal and heteronormative, and therefore repressive and unjust. Once exposed in this way, the author explores ways in which a social-justice-oriented approach to language education can orient learners towards different perspectives of these themes. Ortaçtepe Hart proposes an anti-neoliberal framework, and presents this with a view to dismantling classism. According to the author, antiracist pedagogies can be incorporated through practices that focus on the denormalization of whiteness and that seek to dismantle discriminatory systems and policies. By queering language education, a more inclusive curriculum can give legitimacy to families, relationships, and lifestyles that do not conform to the narrow, heteronormative images that languages learners tend to be exposed to.

Part III consists of a single, concluding chapter, which addresses the challenges we face as language educators when it comes to social justice education, before offering some useful ideas for reflection and practical application. It starts with an acknowledgement of the challenges that language educators are likely to face if they wish to take a social justice approach. These include challenges related to changing the attitudes of other stakeholders, accepting that social justice education requires difficult and sometimes painful discussions to take place in the classroom, and issues related to this such as tolerance, and the creation of ‘safe’ spaces. The author then offers some practical considerations on areas such as assessment for learning and the potential role of technology in social justice language education.

Before I go any further in my evaluation of this book, I should disclose that I am a colleague of the author, and also that I have already written an overwhelmingly positive review of this book for another journal. Sceptical readers may feel that my professional relationship with the author makes a positive review inevitable. However, as someone who cares deeply about ELT and its impact on the wider world, I can honestly say this book really speaks to me. The issue of social justice has, for too long, been neglected in most ELT contexts, and it is pleasing to see Ortaçtepe Hart calling for it to be placed at the centre of the language curriculum. It is the ‘all or nothing’ message that appeals to me the most: social justice is not something you can ‘touch on’ now and again. That would be performative social justice education, turning the whole concept into a commodity that can be incorporated into a neoliberal curriculum. No, social justice language education should drive everything: curriculum, materials, methods, assessment—everything.

Of course, this makes social justice language education seem like an impossible task. Most of us (and I include myself) work in contexts that are affected by profit-orientation, the need for efficiency, assessment criteria, policy, market forces, competition, employer expectations, and many other factors which run counter to the principles of a social justice approach. With this in mind, some people might criticize Ortaçtepe Hart’s book for not offering enough practical guidance on how to turn a ‘regular’ language curriculum into a social justice language curriculum. However, as I mentioned in my previous review, this is precisely the point. One of the key problems with the current model of ELT is that it assumes that a single, commodified approach can be applied by all teachers, to all learners, and in all contexts. For Ortaçtepe Hart, the social justice curriculum needs to be participatory, which means it has to be driven by the issues that are most relevant to the learners in each specific context. While Hadley and Boon provide a series of activities and clear guidelines that teachers can apply directly to incorporate critical thinking into ELT, and Gerald presents his own course as an example that can inspire teachers to find their own ways to counter the damaging impact of whiteness, Ortaçtepe Hart deliberately avoids providing readers with any kind of practical model for teachers to apply, or misapply, in whatever context they happen to be in. While this may seem to some like a frustrating omission, it ensures that the book adheres to the theories and pedagogies that it advocates, and forces readers to critically reflect on their own contexts and learners in adopting a social justice approach to language education.

Another key strength of the book is the use of intersectionality to explore the complexities of social (in)justice. It is the way in which different features of people’s individual identities combine to locate them in certain social positions and to affect their capacities for social mobility that creates social injustice in the first place. Although Gerald views the problems of ELT through the lens of whiteness, an intersectional lens allows Ortaçtepe Hart to incorporate not only race and ethnicity but also class, gender and sexual identity, sexuality, and a range of other issues into the discussion.

If I was to offer one criticism of the book, it would be that it could go into more depth to explore not only why social justice has been so neglected in ELT, but also why there continues to be such resistance to social justice language education in a world that is full of injustice, corruption, and exploitation. The concept of hegemony is used throughout the book to present ideas such as colour-blind pedagogies (which effectively deny the existence of racism), heteronormativity, and language ideologies. Nevertheless, it is the deeply embedded nature of these hegemonies, and the way they are controlled by those (usually profit-oriented) forces that benefit from them, that is, to my mind, the root of the problem. Having said that, the book already covers such a wide range of issues, and it could lose its balance and cohesion if it tried to squeeze in even more critical analysis.

All the books in this review follow the premise that ELT is about more than simply teaching and practising language items, systems, and skills, and therefore challenge the long-held belief that neutrality is both possible and preferable. Whereas Hadley and Boon’s Critical Thinking is primarily concerned with developing skills to assess the veracity of content, both inside and outside the classroom, the other books are more focused on the nature of the ELT curriculum itself. Proposing alternatives to existing models, however imperialistic, colonizing, or racist they may be, could simply give way to a different kind of colonization, particularly if they propose alternative ideologies that are popular in the Global North and seek to impose them on the rest of the world ( Selvi 2024 ). However, these publications all take steps to incorporate a global perspective, either by including chapters from diverse authors working in a wide range of contexts, or by including sections that stress the relevance of the issues to all contexts where critical thinking and critical consciousness can address inequities, injustices, and misinformation.

Of course, reading these books individually could give the impression that the problem with ELT is narrower than it actually is. Hadley and Boon tell us that there is a lack of focus on critical thinking. Reading Ludwig and Summer allows you to identify deficiencies in the range of topics being explored. Banegas and Govender’s book leads us to conclude that we need to do more to address issues of sexuality and gender identity. For Gerald, the problems in ELT are grounded in racism and the hegemony of whiteness. The reality, however, is that all of these problems exist, and they are all interconnected. Only Ortaçtepe Hart attempts to bring these issues together under the umbrella of social justice; this allows her to write about race, gender and sexuality, class, and how hegemonic forces have led to the construction of a pedagogical approach that is devoid of criticality and perpetuates inequity. Perhaps what our profession needs is more literature that aims to develop a clearer understanding of the bigger picture: the wide-ranging nature of hegemony and the fact that rather than exposing and challenging social injustice, ELT has been complicit in its global spread.

In addition to the main themes of each book, a recurring issue is the role of globalization and neoliberal capitalism in our profession. The dominant forces in ELT are motivated more by commercial success than by effective teaching practice. A reluctance to challenge the status quo, therefore, is understandable when the current system yields so much profit for the corporations who are responsible for designing the teacher education courses, creating the materials, and providing the assessments that measure our students’ success.

Nevertheless, the long-prevalent myth of neutrality in ELT, and the concomitant avoidance of critical engagement with real social and environmental problems, is losing credibility as the need for a more critically conscious approach becomes increasingly urgent. These books highlight something that is not discussed often enough in the global ELT profession: the fact that English language teachers are more than facilitators of procedure—we are educators. Of course, the primary purpose is to teach the language. We need to help our students to communicate ideas more effectively in English. But what ideas? And to what end? Are we happy to simply give people the language skills that allow them to play the system to their advantage, when the system itself is failing? These books all encourage us to reflect on our individual and collective responsibility as language educators, to develop our students’ capacities to problematize local and global issues, and to effect positive change in the world. Surely that is the purpose of education, and it is becoming abundantly clear that this is what the world needs more than it ever has before.

Steve Brown has worked in English language teaching since 1993. He started his career as a volunteer before spending several years in private language schools in Central Europe and South Africa. He returned to Scotland in 2001 and had a long career in the further education sector, before moving to higher education in 2018. Steve has held various management, leadership and teacher development positions, and is currently a lecturer in TESOL at the University of Glasgow. His research interests include materials analysis, curriculum design, ESOL policy, and ELT as emancipatory practice.

Email : [email protected]

Block , D. , J.   Gray , and M.   Holborow .   2012 . Neoliberalism and Applied Linguistics . London : Routledge .

Google Scholar

Google Preview

Brown , S. , and C.   Nanguy .   2021 . ‘Global Coursebooks and Equalities Legislation: A Critical Study.’   New York State TESOL Journal   8 ( 2 ): 51 – 62 .

Byram , M.   1988 . ‘Foreign Language Education and Cultural Studies.’   Language, Culture and Curriculum   1 ( 1 ): 15 – 31 .

Crookes , G.   2021 . ‘Critical Language Pedagogy: An Introduction to Principles and Values.’   ELT Journal   75 ( 3 ): 247 – 55 .

De Sousa Santos , B.   2014 . Epistemologies of the South: Justice Against Epistemicide . London : Routledge .

Freire , P.   2013 . Education for Critical Consciousness.   London : Bloomsbury .

Freire , P.   1996 . Pedagogy of the Oppressed . revised edn. London : Penguin .

Giroux , H.   2011 . On Critical Pedagogy.   New York : Bloomsbury .

Gray , J.   2002 . ‘The Global Coursebook in English Language Teaching.’ In Globalization and Language Teaching , edited by D.   Block and D.   Cameron , 151 – 67 . London : Routledge .

Gray , J.   2015 . The Construction of English . Basingstoke : Palgrave MacMillan .

Littlejohn , A.   2012 . ‘Language Teaching Materials and the (very) Big Picture.’   Electronic Journal of Foreign Language Teaching   9 ( 1 ): 283 – 97 .

Morris , E.   2014 . ‘Teaching Needs Less Ideology and More Evidence.’ The Guardian Online , accessed 25 November 2015 . https://www.theguardian.com/education/2014/nov/25/teaching-needs-less-ideology-more-evidence

Olcoń , K.   2023 . ‘Key Concepts in Critical Whiteness Studies.’ In Handbook of Critical Whiteness , edited by J.   Ravulo , K.   Olcoń , T.   Dune , A.   Workman , and P.   Liamputtong . Singapore : Springer . https://doi.org/10.1007/978-981-19-1612-0_2-1

Pennycook , A.   1989 . ‘The Concept of Method, Interested Knowledge and the Politics of Language Teaching.’   TESOL Quarterly   23 ( 4 ): 589 – 618 .

Pennycook , A.   2021 . Critical Applied Linguistics: A (Critical Re)introduction.   Abingdon : Routledge .

Schiro , M. S.   2013 . Curriculum Theory: Conflicting Issues and Enduring Concerns . 2nd edn. London : Sage .

Selvi , A. F.   2024 . ‘The Myopic Focus on Decoloniality in Applied Linguistics and English Language Education:   Citations and Stolen Subjectivities.’   Applied Linguistics Review , pp. 1 – 25 .  Online ahead of print .  https://doi.org/10.1515/applirev-2024-0011

Email alerts

Citing articles via.

• Recommend to Your Library

Affiliations

• Online ISSN 1477-4526
• Print ISSN 0951-0893
• Copyright © 2024 Oxford University Press
• About Oxford Academic
• Publish journals with us
• University press partners
• What we publish
• New features  
• Open access
• Institutional account management
• Rights and permissions
• Get help with access
• Accessibility
• Advertising
• Media enquiries
• Oxford University Press
• Oxford Languages
• University of Oxford

Oxford University Press is a department of the University of Oxford. It furthers the University's objective of excellence in research, scholarship, and education by publishing worldwide

• Copyright © 2024 Oxford University Press
• Cookie settings
• Cookie policy
• Privacy policy
• Legal notice

This Feature Is Available To Subscribers Only

Sign In or Create an Account

This PDF is available to Subscribers Only

For full access to this pdf, sign in to an existing account, or purchase an annual subscription.