critical thinking not concerns in determining the quality of our beliefs

How it works

For Business

Join Mind Tools

Article • 8 min read

Critical Thinking

Developing the right mindset and skills.

By the Mind Tools Content Team

We make hundreds of decisions every day and, whether we realize it or not, we're all critical thinkers.

We use critical thinking each time we weigh up our options, prioritize our responsibilities, or think about the likely effects of our actions. It's a crucial skill that helps us to cut out misinformation and make wise decisions. The trouble is, we're not always very good at it!

In this article, we'll explore the key skills that you need to develop your critical thinking skills, and how to adopt a critical thinking mindset, so that you can make well-informed decisions.

What Is Critical Thinking?

Critical thinking is the discipline of rigorously and skillfully using information, experience, observation, and reasoning to guide your decisions, actions, and beliefs. You'll need to actively question every step of your thinking process to do it well.

Collecting, analyzing and evaluating information is an important skill in life, and a highly valued asset in the workplace. People who score highly in critical thinking assessments are also rated by their managers as having good problem-solving skills, creativity, strong decision-making skills, and good overall performance. [1]

Key Critical Thinking Skills

Critical thinkers possess a set of key characteristics which help them to question information and their own thinking. Focus on the following areas to develop your critical thinking skills:

Being willing and able to explore alternative approaches and experimental ideas is crucial. Can you think through "what if" scenarios, create plausible options, and test out your theories? If not, you'll tend to write off ideas and options too soon, so you may miss the best answer to your situation.

To nurture your curiosity, stay up to date with facts and trends. You'll overlook important information if you allow yourself to become "blinkered," so always be open to new information.

But don't stop there! Look for opposing views or evidence to challenge your information, and seek clarification when things are unclear. This will help you to reassess your beliefs and make a well-informed decision later. Read our article, Opening Closed Minds , for more ways to stay receptive.

Logical Thinking

You must be skilled at reasoning and extending logic to come up with plausible options or outcomes.

It's also important to emphasize logic over emotion. Emotion can be motivating but it can also lead you to take hasty and unwise action, so control your emotions and be cautious in your judgments. Know when a conclusion is "fact" and when it is not. "Could-be-true" conclusions are based on assumptions and must be tested further. Read our article, Logical Fallacies , for help with this.

Use creative problem solving to balance cold logic. By thinking outside of the box you can identify new possible outcomes by using pieces of information that you already have.

Self-Awareness

Many of the decisions we make in life are subtly informed by our values and beliefs. These influences are called cognitive biases and it can be difficult to identify them in ourselves because they're often subconscious.

Practicing self-awareness will allow you to reflect on the beliefs you have and the choices you make. You'll then be better equipped to challenge your own thinking and make improved, unbiased decisions.

One particularly useful tool for critical thinking is the Ladder of Inference . It allows you to test and validate your thinking process, rather than jumping to poorly supported conclusions.

Developing a Critical Thinking Mindset

Combine the above skills with the right mindset so that you can make better decisions and adopt more effective courses of action. You can develop your critical thinking mindset by following this process:

Gather Information

First, collect data, opinions and facts on the issue that you need to solve. Draw on what you already know, and turn to new sources of information to help inform your understanding. Consider what gaps there are in your knowledge and seek to fill them. And look for information that challenges your assumptions and beliefs.

Be sure to verify the authority and authenticity of your sources. Not everything you read is true! Use this checklist to ensure that your information is valid:

• Are your information sources trustworthy ? (For example, well-respected authors, trusted colleagues or peers, recognized industry publications, websites, blogs, etc.)
• Is the information you have gathered up to date ?
• Has the information received any direct criticism ?
• Does the information have any errors or inaccuracies ?
• Is there any evidence to support or corroborate the information you have gathered?
• Is the information you have gathered subjective or biased in any way? (For example, is it based on opinion, rather than fact? Is any of the information you have gathered designed to promote a particular service or organization?)

If any information appears to be irrelevant or invalid, don't include it in your decision making. But don't omit information just because you disagree with it, or your final decision will be flawed and bias.

Now observe the information you have gathered, and interpret it. What are the key findings and main takeaways? What does the evidence point to? Start to build one or two possible arguments based on what you have found.

You'll need to look for the details within the mass of information, so use your powers of observation to identify any patterns or similarities. You can then analyze and extend these trends to make sensible predictions about the future.

To help you to sift through the multiple ideas and theories, it can be useful to group and order items according to their characteristics. From here, you can compare and contrast the different items. And once you've determined how similar or different things are from one another, Paired Comparison Analysis can help you to analyze them.

The final step involves challenging the information and rationalizing its arguments.

Apply the laws of reason (induction, deduction, analogy) to judge an argument and determine its merits. To do this, it's essential that you can determine the significance and validity of an argument to put it in the correct perspective. Take a look at our article, Rational Thinking , for more information about how to do this.

Once you have considered all of the arguments and options rationally, you can finally make an informed decision.

Afterward, take time to reflect on what you have learned and what you found challenging. Step back from the detail of your decision or problem, and look at the bigger picture. Record what you've learned from your observations and experience.

Critical thinking involves rigorously and skilfully using information, experience, observation, and reasoning to guide your decisions, actions and beliefs. It's a useful skill in the workplace and in life.

You'll need to be curious and creative to explore alternative possibilities, but rational to apply logic, and self-aware to identify when your beliefs could affect your decisions or actions.

You can demonstrate a high level of critical thinking by validating your information, analyzing its meaning, and finally evaluating the argument.

Critical Thinking Infographic

See Critical Thinking represented in our infographic: An Elementary Guide to Critical Thinking .

You've accessed 1 of your 2 free resources.

Get unlimited access

Discover more content

How can i improve my memory.

Four Techniques That Can Help

What Is Leadership?

How to Lead Your Team Skilfully

Add comment

Comments (1)

priyanka ghogare

Gain essential management and leadership skills

Busy schedule? No problem. Learn anytime, anywhere. 

Subscribe to unlimited access to meticulously researched, evidence-based resources.

Join today and take advantage of our 30% offer, available until May 31st .

Sign-up to our newsletter

Subscribing to the Mind Tools newsletter will keep you up-to-date with our latest updates and newest resources.

Subscribe now

Business Skills

Personal Development

Leadership and Management

Member Extras

Most Popular

Latest Updates

Winning Body Language

Business Stripped Bare

Mind Tools Store

About Mind Tools Content

Discover something new today

Nine ways to get the best from x (twitter).

Growing Your Business Quickly and Safely on Social Media

Managing Your Emotions at Work

Controlling Your Feelings... Before They Control You

How Emotionally Intelligent Are You?

Boosting Your People Skills

Self-Assessment

What's Your Leadership Style?

Learn About the Strengths and Weaknesses of the Way You Like to Lead

Recommended for you

What is posdcorb.

Running Your Team in the Most Efficient Way

Business Operations and Process Management

Strategy Tools

Customer Service

Business Ethics and Values

Handling Information and Data

Project Management

Knowledge Management

Self-Development and Goal Setting

Time Management

Presentation Skills

Learning Skills

Career Skills

Communication Skills

Negotiation, Persuasion and Influence

Working With Others

Difficult Conversations

Creativity Tools

Self-Management

Work-Life Balance

Stress Management and Wellbeing

Coaching and Mentoring

Change Management

Team Management

Managing Conflict

Delegation and Empowerment

Performance Management

Leadership Skills

Developing Your Team

Talent Management

Problem Solving

Decision Making

Member Podcast

To read this content please select one of the options below:

Please note you do not have access to teaching notes, critical thinking of our beliefs, behaviors, and their impact systems.

A Primer on Critical Thinking and Business Ethics

ISBN : 978-1-83753-309-1 , eISBN : 978-1-83753-308-4

Publication date: 27 July 2023

Executive Summary

Being the most powerful creatures on the planet, we humans should carefully consider our beliefs for the simple reason that the way in which we think influences our behaviors; this in turn can either transform the world or negatively affect the world. Our mores, paradigms, and worldviews translate into behaviors (e.g., factory farming for meat production and consumption) that in turn modify the environment. In general, much of our thinking system is backed up by some concept, theory, paradigm, or ideology. Our thinking systems generate our belief systems of goals and mission statements; our belief systems, in turn, determine our behavior systems (e.g., our strategies, choices, commissions, omissions as implementation systems); our behavior systems determine our impact systems (e.g., impact on us, our families and neighborhoods, our cities and villages, our state and our country, our globe and sometimes our cosmos). Thus, our behavior systems eventually impact our thinking systems, which we started with, thus completing a circular or spiral loop. This chapter examines the thinking–beliefs–behaviors–impact loop, exploring its internal and external dynamics and validities. Specifically, in Part I, we examine the structure of our belief systems in business; in Part II, we explore the power of our structured belief systems in business; in Part III, we apply critical thinking that systematically questions and seeks to redesign our presumed thinking and belief systems.

Mascarenhas, O.A.J. , Thakur, M. and Kumar, P. (2023), "Critical Thinking of Our Beliefs, Behaviors, and Their Impact Systems", A Primer on Critical Thinking and Business Ethics , Emerald Publishing Limited, Leeds, pp. 117-146. https://doi.org/10.1108/978-1-83753-308-420231005

Emerald Publishing Limited

Copyright © 2023 Oswald A. J. Mascarenhas, Munish Thakur and Payal Kumar. Published under exclusive licence by Emerald Publishing Limited

We’re listening — tell us what you think

Something didn’t work….

Report bugs here

All feedback is valuable

Please share your general feedback

Join us on our journey

Platform update page.

Visit emeraldpublishing.com/platformupdate to discover the latest news and updates

Questions & More Information

Answers to the most commonly asked questions here

1 Introduction to Critical Thinking

I. what is c ritical t hinking [1].

Critical thinking is the ability to think clearly and rationally about what to do or what to believe.  It includes the ability to engage in reflective and independent thinking. Someone with critical thinking skills is able to do the following:

• Understand the logical connections between ideas.
• Identify, construct, and evaluate arguments.
• Detect inconsistencies and common mistakes in reasoning.
• Solve problems systematically.
• Identify the relevance and importance of ideas.
• Reflect on the justification of one’s own beliefs and values.

Critical thinking is not simply a matter of accumulating information. A person with a good memory and who knows a lot of facts is not necessarily good at critical thinking. Critical thinkers are able to deduce consequences from what they know, make use of information to solve problems, and to seek relevant sources of information to inform themselves.

Critical thinking should not be confused with being argumentative or being critical of other people. Although critical thinking skills can be used in exposing fallacies and bad reasoning, critical thinking can also play an important role in cooperative reasoning and constructive tasks. Critical thinking can help us acquire knowledge, improve our theories, and strengthen arguments. We can also use critical thinking to enhance work processes and improve social institutions.

Some people believe that critical thinking hinders creativity because critical thinking requires following the rules of logic and rationality, whereas creativity might require breaking those rules. This is a misconception. Critical thinking is quite compatible with thinking “out-of-the-box,” challenging consensus views, and pursuing less popular approaches. If anything, critical thinking is an essential part of creativity because we need critical thinking to evaluate and improve our creative ideas.

II. The I mportance of C ritical T hinking

Critical thinking is a domain-general thinking skill. The ability to think clearly and rationally is important whatever we choose to do. If you work in education, research, finance, management or the legal profession, then critical thinking is obviously important. But critical thinking skills are not restricted to a particular subject area. Being able to think well and solve problems systematically is an asset for any career.

Critical thinking is very important in the new knowledge economy.  The global knowledge economy is driven by information and technology. One has to be able to deal with changes quickly and effectively. The new economy places increasing demands on flexible intellectual skills, and the ability to analyze information and integrate diverse sources of knowledge in solving problems. Good critical thinking promotes such thinking skills, and is very important in the fast-changing workplace.

Critical thinking enhances language and presentation skills. Thinking clearly and systematically can improve the way we express our ideas. In learning how to analyze the logical structure of texts, critical thinking also improves comprehension abilities.

Critical thinking promotes creativity. To come up with a creative solution to a problem involves not just having new ideas. It must also be the case that the new ideas being generated are useful and relevant to the task at hand. Critical thinking plays a crucial role in evaluating new ideas, selecting the best ones and modifying them if necessary.

Critical thinking is crucial for self-reflection. In order to live a meaningful life and to structure our lives accordingly, we need to justify and reflect on our values and decisions. Critical thinking provides the tools for this process of self-evaluation.

Good critical thinking is the foundation of science and democracy. Science requires the critical use of reason in experimentation and theory confirmation. The proper functioning of a liberal democracy requires citizens who can think critically about social issues to inform their judgments about proper governance and to overcome biases and prejudice.

Critical thinking is a   metacognitive skill . What this means is that it is a higher-level cognitive skill that involves thinking about thinking. We have to be aware of the good principles of reasoning, and be reflective about our own reasoning. In addition, we often need to make a conscious effort to improve ourselves, avoid biases, and maintain objectivity. This is notoriously hard to do. We are all able to think but to think well often requires a long period of training. The mastery of critical thinking is similar to the mastery of many other skills. There are three important components: theory, practice, and attitude.

III. Improv ing O ur T hinking S kills

If we want to think correctly, we need to follow the correct rules of reasoning. Knowledge of theory includes knowledge of these rules. These are the basic principles of critical thinking, such as the laws of logic, and the methods of scientific reasoning, etc.

Also, it would be useful to know something about what not to do if we want to reason correctly. This means we should have some basic knowledge of the mistakes that people make. First, this requires some knowledge of typical fallacies. Second, psychologists have discovered persistent biases and limitations in human reasoning. An awareness of these empirical findings will alert us to potential problems.

However, merely knowing the principles that distinguish good and bad reasoning is not enough. We might study in the classroom about how to swim, and learn about the basic theory, such as the fact that one should not breathe underwater. But unless we can apply such theoretical knowledge through constant practice, we might not actually be able to swim.

Similarly, to be good at critical thinking skills it is necessary to internalize the theoretical principles so that we can actually apply them in daily life. There are at least two ways to do this. One is to perform lots of quality exercises. These exercises don’t just include practicing in the classroom or receiving tutorials; they also include engaging in discussions and debates with other people in our daily lives, where the principles of critical thinking can be applied. The second method is to think more deeply about the principles that we have acquired. In the human mind, memory and understanding are acquired through making connections between ideas.

Good critical thinking skills require more than just knowledge and practice. Persistent practice can bring about improvements only if one has the right kind of motivation and attitude. The following attitudes are not uncommon, but they are obstacles to critical thinking:

• I prefer being given the correct answers rather than figuring them out myself.
• I don’t like to think a lot about my decisions as I rely only on gut feelings.
• I don’t usually review the mistakes I have made.
• I don’t like to be criticized.

To improve our thinking we have to recognize the importance of reflecting on the reasons for belief and action. We should also be willing to engage in debate, break old habits, and deal with linguistic complexities and abstract concepts.

The  California Critical Thinking Disposition Inventory  is a psychological test that is used to measure whether people are disposed to think critically. It measures the seven different thinking habits listed below, and it is useful to ask ourselves to what extent they describe the way we think:

• Truth-Seeking—Do you try to understand how things really are? Are you interested in finding out the truth?
• Open-Mindedness—How receptive are you to new ideas, even when you do not intuitively agree with them? Do you give new concepts a fair hearing?
• Analyticity—Do you try to understand the reasons behind things? Do you act impulsively or do you evaluate the pros and cons of your decisions?
• Systematicity—Are you systematic in your thinking? Do you break down a complex problem into parts?
• Confidence in Reasoning—Do you always defer to other people? How confident are you in your own judgment? Do you have reasons for your confidence? Do you have a way to evaluate your own thinking?
• Inquisitiveness—Are you curious about unfamiliar topics and resolving complicated problems? Will you chase down an answer until you find it?
• Maturity of Judgment—Do you jump to conclusions? Do you try to see things from different perspectives? Do you take other people’s experiences into account?

Finally, as mentioned earlier, psychologists have discovered over the years that human reasoning can be easily affected by a variety of cognitive biases. For example, people tend to be over-confident of their abilities and focus too much on evidence that supports their pre-existing opinions. We should be alert to these biases in our attitudes towards our own thinking.

IV. Defining Critical Thinking

There are many different definitions of critical thinking. Here we list some of the well-known ones. You might notice that they all emphasize the importance of clarity and rationality. Here we will look at some well-known definitions in chronological order.

1) Many people trace the importance of critical thinking in education to the early twentieth-century American philosopher John Dewey. But Dewey did not make very extensive use of the term “critical thinking.” Instead, in his book  How We Think (1910), he argued for the importance of what he called “reflective thinking”:

…[when] the ground or basis for a belief is deliberately sought and its adequacy to support the belief examined. This process is called reflective thought; it alone is truly educative in value…

Active, persistent and careful consideration of any belief or supposed form of knowledge in light of the grounds that support it, and the further conclusions to which it tends, constitutes reflective thought.

There is however one passage from How We Think where Dewey explicitly uses the term “critical thinking”:

The essence of critical thinking is suspended judgment; and the essence of this suspense is inquiry to determine the nature of the problem before proceeding to attempts at its solution. This, more than any other thing, transforms mere inference into tested inference, suggested conclusions into proof.

2) The  Watson-Glaser Critical Thinking Appraisal  (1980) is a well-known psychological test of critical thinking ability. The authors of this test define critical thinking as:

…a composite of attitudes, knowledge and skills. This composite includes: (1) attitudes of inquiry that involve an ability to recognize the existence of problems and an acceptance of the general need for evidence in support of what is asserted to be true; (2) knowledge of the nature of valid inferences, abstractions, and generalizations in which the weight or accuracy of different kinds of evidence are logically determined; and (3) skills in employing and applying the above attitudes and knowledge.

3) A very well-known and influential definition of critical thinking comes from philosopher and professor Robert Ennis in his work “A Taxonomy of Critical Thinking Dispositions and Abilities” (1987):

Critical thinking is reasonable reflective thinking that is focused on deciding what to believe or do.

4) The following definition comes from a statement written in 1987 by the philosophers Michael Scriven and Richard Paul for the  National Council for Excellence in Critical Thinking (link), an organization promoting critical thinking in the US:

Critical thinking is the intellectually disciplined process of actively and skillfully conceptualizing, applying, analyzing, synthesizing, and/or evaluating information gathered from, or generated by, observation, experience, reflection, reasoning, or communication, as a guide to belief and action. In its exemplary form, it is based on universal intellectual values that transcend subject matter divisions: clarity, accuracy, precision, consistency, relevance, sound evidence, good reasons, depth, breadth, and fairness. It entails the examination of those structures or elements of thought implicit in all reasoning: purpose, problem, or question-at-issue, assumptions, concepts, empirical grounding; reasoning leading to conclusions, implications and consequences, objections from alternative viewpoints, and frame of reference.

The following excerpt from Peter A. Facione’s “Critical Thinking: A Statement of Expert Consensus for Purposes of Educational Assessment and Instruction” (1990) is quoted from a report written for the American Philosophical Association:

We understand critical thinking to be purposeful, self-regulatory judgment which results in interpretation, analysis, evaluation, and inference, as well as explanation of the evidential, conceptual, methodological, criteriological, or contextual considerations upon which that judgment is based. CT is essential as a tool of inquiry. As such, CT is a liberating force in education and a powerful resource in one’s personal and civic life. While not synonymous with good thinking, CT is a pervasive and self-rectifying human phenomenon. The ideal critical thinker is habitually inquisitive, well-informed, trustful of reason, open-minded, flexible, fairminded in evaluation, honest in facing personal biases, prudent in making judgments, willing to reconsider, clear about issues, orderly in complex matters, diligent in seeking relevant information, reasonable in the selection of criteria, focused in inquiry, and persistent in seeking results which are as precise as the subject and the circumstances of inquiry permit. Thus, educating good critical thinkers means working toward this ideal. It combines developing CT skills with nurturing those dispositions which consistently yield useful insights and which are the basis of a rational and democratic society.

V. Two F eatures of C ritical T hinking

A. how not what .

Critical thinking is concerned not with what you believe, but rather how or why you believe it. Most classes, such as those on biology or chemistry, teach you what to believe about a subject matter. In contrast, critical thinking is not particularly interested in what the world is, in fact, like. Rather, critical thinking will teach you how to form beliefs and how to think. It is interested in the type of reasoning you use when you form your beliefs, and concerns itself with whether you have good reasons to believe what you believe. Therefore, this class isn’t a class on the psychology of reasoning, which brings us to the second important feature of critical thinking.

B. Ought N ot Is ( or Normative N ot Descriptive )

There is a difference between normative and descriptive theories. Descriptive theories, such as those provided by physics, provide a picture of how the world factually behaves and operates. In contrast, normative theories, such as those provided by ethics or political philosophy, provide a picture of how the world should be. Rather than ask question such as why something is the way it is, normative theories ask how something should be. In this course, we will be interested in normative theories that govern our thinking and reasoning. Therefore, we will not be interested in how we actually reason, but rather focus on how we ought to reason.

In the introduction to this course we considered a selection task with cards that must be flipped in order to check the validity of a rule. We noted that many people fail to identify all the cards required to check the rule. This is how people do in fact reason (descriptive). We then noted that you must flip over two cards. This is how people ought to reason (normative).

• Section I-IV are taken from http://philosophy.hku.hk/think/ and are in use under the creative commons license. Some modifications have been made to the original content. ↵

Critical Thinking Copyright © 2019 by Brian Kim is licensed under a Creative Commons Attribution 4.0 International License , except where otherwise noted.

Share This Book

• Get started with computers
• Learn Microsoft Office
• Apply for a job
• Improve my work skills
• Design nice-looking docs
• Getting Started
• Smartphones & Tablets
• Typing Tutorial
• Online Learning
• Basic Internet Skills
• Online Safety
• Social Media
• Zoom Basics
• Google Docs
• Google Sheets
• Career Planning
• Resume Writing
• Cover Letters
• Job Search and Networking
• Business Communication
• Entrepreneurship 101
• Careers without College
• Job Hunt for Today
• 3D Printing
• Freelancing 101
• Personal Finance
• Sharing Economy
• Decision-Making
• Graphic Design
• Photography
• Image Editing
• Learning WordPress
• Language Learning
• Critical Thinking
• For Educators
• Translations
• Staff Picks
• English expand_more expand_less

Critical Thinking and Decision-Making  - What is Critical Thinking?

Critical thinking and decision-making  -, what is critical thinking, critical thinking and decision-making what is critical thinking.

Critical Thinking and Decision-Making: What is Critical Thinking?

Lesson 1: what is critical thinking, what is critical thinking.

Critical thinking is a term that gets thrown around a lot. You've probably heard it used often throughout the years whether it was in school, at work, or in everyday conversation. But when you stop to think about it, what exactly is critical thinking and how do you do it ?

Watch the video below to learn more about critical thinking.

Simply put, critical thinking is the act of deliberately analyzing information so that you can make better judgements and decisions . It involves using things like logic, reasoning, and creativity, to draw conclusions and generally understand things better.

This may sound like a pretty broad definition, and that's because critical thinking is a broad skill that can be applied to so many different situations. You can use it to prepare for a job interview, manage your time better, make decisions about purchasing things, and so much more.

The process

As humans, we are constantly thinking . It's something we can't turn off. But not all of it is critical thinking. No one thinks critically 100% of the time... that would be pretty exhausting! Instead, it's an intentional process , something that we consciously use when we're presented with difficult problems or important decisions.

Improving your critical thinking

In order to become a better critical thinker, it's important to ask questions when you're presented with a problem or decision, before jumping to any conclusions. You can start with simple ones like What do I currently know? and How do I know this? These can help to give you a better idea of what you're working with and, in some cases, simplify more complex issues.  

Real-world applications

Let's take a look at how we can use critical thinking to evaluate online information . Say a friend of yours posts a news article on social media and you're drawn to its headline. If you were to use your everyday automatic thinking, you might accept it as fact and move on. But if you were thinking critically, you would first analyze the available information and ask some questions :

• What's the source of this article?
• Is the headline potentially misleading?
• What are my friend's general beliefs?
• Do their beliefs inform why they might have shared this?

After analyzing all of this information, you can draw a conclusion about whether or not you think the article is trustworthy.

Critical thinking has a wide range of real-world applications . It can help you to make better decisions, become more hireable, and generally better understand the world around you.

/en/problem-solving-and-decision-making/why-is-it-so-hard-to-make-decisions/content/

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Front Psychol

The Role of Students’ Beliefs When Critically Reasoning From Multiple Contradictory Sources of Information in Performance Assessments

Olga zlatkin-troitschanskaia.

1 Department of Business and Economics Education, Johannes Gutenberg University Mainz, Mainz, Germany

Jennifer Fischer

Dominik braunheim, susanne schmidt, richard j. shavelson.

2 Graduate School of Education, Stanford University, Palo Alto, CA, United States

Associated Data

The datasets presented in this article will be made available by the authors, without undue reservation, to any qualified researcher. Requests to access the datasets should be directed to ed.zniam-inu@aiaksnahcstiort .

Critical reasoning (CR) when confronted with contradictory information from multiple sources is a crucial ability in a knowledge-based society and digital world. Using information without critically reflecting on the content and its quality may lead to the acceptance of information based on unwarranted claims. Previous personal beliefs are assumed to play a decisive role when it comes to critically differentiating between assertions and claims and warranted knowledge and facts. The role of generic epistemic beliefs on critical stance and attitude in reflectively dealing with information is well researched. Relatively few studies however, have been conducted on the influence of domain-specific beliefs , i.e., beliefs in relation to specific content encountered in a piece of information or task, on the reasoning process , and on how these beliefs may affect decision-making processes. This study focuses on students’ task- and topic-related beliefs that may influence their reasoning when dealing with multiple and partly contradictory sources of information. To validly assess CR among university students, we used a newly developed computer-based performance assessment in which the students were confronted with an authentic task which contains theoretically defined psychological stimuli for measuring CR. To investigate the particular role of task- and topic-related beliefs on CR, a purposeful sample of 30 university students took part in a performance assessment and then were interviewed immediately afterward. In the semi-structured cognitive interviews, the participants’ task-related beliefs were assessed. Based on qualitative analyses of the interview transcripts, three distinct profiles of decision-making among students have been identified. More specifically, the different types of students’ beliefs and attitudes derived from the cognitive interview data suggest their influence on information processing, reasoning approaches and decision-making. The results indicated that the students’ beliefs had an influence on their selection, critical evaluation and use of information as well as on their reasoning processes and final decisions.

Research Background and Study Objectives

Critical reasoning (CR) when confronted with contradictory information from multiple sources is a crucial ability in a knowledge-based society and digital world ( Brooks, 2016 ; Newman and Beetham, 2017 ; Wineburg and McGrew, 2017 ). The Internet presents a flood of complex, potentially conflicting, and competing information on one and the same issue. To build a dependable and coherent knowledge base and to develop sophisticated (domain-specific and generic) attitudes and an analytical, reflective stance, students must be able to select and critically evaluate, analyze, synthesize, and integrate incoherent, fragmented, and biased information.

Students’ mental CR strategies may likely be insufficient for what is demanded for understanding heterogeneous information and, what is more, for effective and productive participation in a complex information environment (for a meta-study, see Huber and Kuncel, 2016 , for university students, see McGrew et al., 2018 ; Wineburg et al., 2018 ; Hahnel et al., 2019 ; Münchow et al., 2019 ; Shavelson et al., 2019 ; Zlatkin-Troitschanskaia et al., 2019b ). As a coping strategy, they may choose to reduce complexity by various means, for instance, by using cognitive heuristics, preferring simplified forms of information presentation, or relying on sources without verification, which can be exploited for manipulation ( Walthen and Burkell, 2002 ; Metzger, 2007 ; Horstmann et al., 2009 ; Metzger et al., 2010 ).

In addition, certain information representations may be (sub)consciously preferred not for their informational but for their entertainment value, their elicitation of certain affects, or their engagement properties ( Maurer et al., 2018 , 2020 ). Based on students’ previous media experience, knowledge, and expectations, they may have learned to assume that certain types of media representations are more trustworthy than others ( McGrew et al., 2017 ). They may follow a heuristic that similar media representations offer similarly reliable evidence, without considering the communicative context, communicator’s intentions, and possibilities of becoming a victim of manipulation. This is particularly the case when it comes to online information channels ( Metzger et al., 2010 ; Ciampaglia, 2018 ).

As some current studies indicate, students who habitually avoid information that contradicts their beliefs may easily miss important content and fall prey to biased information [see Section “State of Research on Beliefs and Their Impact on (Online) Information Processing”]. Using information without critically reflecting on the content and its quality may lead to the acceptance of information based on unwarranted claims. Deficits in due critical evaluation arise most likely because of shallow processing or insufficient reasoning and may occur subconsciously ( Stanovich, 2003 , 2016 ).

Insufficient reasoning can be amplified when information on a topic is distorted or counterfactual and when students do not recognize biased or false information and use it to build knowledge. As a result, learners may neglect complex, academically warranted knowledge and rely more on lower-quality information that is consistent with their beliefs and biases and that is easier to comprehend ( Hahnel et al., 2019 ; Schoor et al., 2019 ). The internalization of this biased information may subsequently affect learning by acting to inhibit or distort more advanced information processing and knowledge acquisition ( List and Alexander, 2017 , 2018 ).

Theoretically, previous personal beliefs are assumed to play a very decisive role when it comes to critically differentiating between assertions and claims on the one hand and warranted knowledge and facts on the other hand. Rather, the role of generic epistemic beliefs on critical stance and attitude in reflectively dealing with information is well researched [see Section “State of Research on Beliefs and Their Impact on (Online) Information Processing”]. Relatively few studies have been conducted on the influence of domain-specific beliefs, i.e., beliefs in relation to specific content encountered in a piece of information or task , on the reasoning process. Beliefs of this kind are usually measured using psychological scales, but without insight into the reasoning process and how these beliefs may affect the information-processing and decision-making processes [for an overview of current research, see Section “State of Research on Beliefs and Their Impact on (Online) Information Processing”].

With our study, we aim to contribute to this research desideratum. This study focuses on students’ task- and topic-related beliefs that may influence their reasoning when dealing with multiple and partly contradictory sources of information. To validly assess CR among university students, we used a newly developed computer-based performance assessment of learning in which the students are confronted with an authentic task which contains theoretically defined psychological stimuli for measuring CR (for details, see Section “Assessment Frameworks for Measuring Critical Reasoning”) in accordance with our construct definition (see Section “Critically Reasoning from Multiple Sources of Information”; Shavelson et al., 2019 ; Zlatkin-Troitschanskaia et al.,2019b ).

To investigate the particular role of task- and topic-related beliefs on CR, a purposeful sample of 30 university students from the overall sample of the overarching German iPAL study took part in a performance assessment and then were interviewed immediately afterward (for details, see Sections “A Study on Performance Assessments of Higher Education Students’ CR” and “Materials and Methods”). In the semi-structured cognitive interviews, the task-related beliefs of the participants were elicited and then qualitatively analyzed (see Section “Cognitive Interviews and Qualitative Analyses”). The cognitive interview transcripts were examined in order to address the two overarching questions (i) how do students’ beliefs influence their selection, evaluation and use of information as well as their subsequent reasoning and decision-making? and (ii) how do students’ beliefs change as they progress through the task and encounter multiple new information sources along the way. Based on qualitative analyses ( Strauss and Corbin, 1998 ; for details, see “Materials and Methods”), different profiles of participants have been identified, which can be distinguished by different personal characteristics such as the level of prior knowledge.

In this paper, we present our theoretical and empirical analyses to address these two questions (see Section “Results”). The study results – despite the necessary limitations (see Section “Limitations and Implication for Future Research”) – lead to a valuable specification of theoretical assumptions for further empirical research in this highly relevant but under-researched field (see Section “Summary and Interpretation of Results”).

State of Research on Beliefs and Their Impact on (Online) Information Processing

For a systematic analysis of the state of research, we conducted a criteria-driven online search. Based on expert interviews, we determined a set of keywords and conducted the search on the ERIC database and Google Scholar for the period 2009–2020. The stepwise search using keywords related to online information processing and critical reasoning among university students resulted in 56 eligible studies. The review of the abstracts showed that students’ beliefs were assessed and analyzed in 15 studies. The essential results of these studies are summarized in the following overview (see Table 1 ).

Overview of recent studies on beliefs and their impact on information processing.

About half of these 15 studies focus explicitly on the relation between beliefs and (online) information processing (see Table 1 ), while critical reasoning was only implicitly addressed. Despite this narrow research focus, all studies indicate a clear connection between epistemic beliefs and the approach to (online) information processing, especially regarding judgment of information sources and their content. Well-developed and more advanced epistemic beliefs positively influenced the quality of students’ information processing.

Ulyshen et al. (2015) provided one of the few studies specifically investigating the relation between general epistemic beliefs and Internet search behavior . Using participants’ think-aloud protocols they investigated the impact of students’ task-related epistemic beliefs on their information processing. The results indicate a positive impact of well-developed epistemic beliefs on evaluating the quality and credibility of information.

Chiu et al. (2013) used a questionnaire to investigate the relation between university students’ Internet-specific epistemic beliefs and Internet search behavior. The authors identified four dimensions of beliefs: certainty, simplicity, source , and justification of Internet-based knowledge . The results indicate a positive association between Internet searching and justification , and a negative association with simplicity and source . In a follow-up study, Chiu et al. (2015) examined gender differences in students’ Internet-specific epistemic beliefs, indicating a gender gap in certainty and simplicity , and revealing more perceived uncertainty and complexity among females compared to males.

Mason et al. (2010) specifically focused on students’ Internet search when working on academic tasks dealing with a controversial topic , and in relation to epistemic metacognition , which was defined as students’ ability to spontaneously reflect on the accessed information. In addition, they examined the relationship between personal characteristics and prior topic-related knowledge. Test participants were asked to think aloud during their Internet search . Qualitative and quantitative analyses revealed diverse epistemic metacognitions among all study participants, but to different extents and levels. No correlation between epistemic beliefs and prior knowledge was identified. Overall, two patterns of epistemic metacognition were determined that significantly affected students’ Internet search. Students who spontaneously generated more sophisticated reflections about the sources and the information provided outperformed students who were at a lower epistemic level.

In an experimental study with an intervention-control group design (the intervention aiming at improving medicine-specific epistemic beliefs ), Kienhues et al. (2011) focused on the relationship between processing conflicting versus consistent (medical) information on the Internet and topic-related and medicine-specific epistemic beliefs . The intervention groups differed in both their topic-related and medicine-specific epistemic beliefs, and were more advanced compared to the control group.

van Strien et al. (2016) examined the influence of attitude strength on the processing and evaluation of sources of information on the Web. In an eye-tracking study, university students received information from pre-selected websites from different sources on a controversial topic. Participants who felt strongly about the topic did not consider websites with attitude-inconsistent content for as long and did not rate the credibility of this information as highly as students with less strongly established prior attitudes. The participants with strong prior attitudes also included more attitude-consistent information in an essay task than participants with weaker prior attitudes. Thus, differences in prior attitudes bias the evaluation and processing of information in different ways. Even though students were not fully biased during initial information processing, they were so when evaluating the information and presenting it in an essay task.

Similar biases were found in a study by Bråten et al. (2011) , who examined how undergraduates judged the trustworthiness of information sources on a controversial topic. Students judged information differently depending on the sources (e.g., textbooks, official documents, newspapers). In addition, students with limited topic-specific knowledge were inclined to trust less trustworthy sources. Lucassen and Schraagen (2011) show similar results in terms of relation with domain-specific knowledge and source expertise .

Following the assumption that students spontaneously engage in epistemic cognition when processing conflicting scientific information, van Strien et al. (2012b) examined how this epistemic cognition is related to students’ actual beliefs. In addition, the interplay of students’ epistemic beliefs and prior attitudes when encountering conflicting and partly attitude-inconsistent information on a controversial socio-scientific topic was studied using think-aloud methods. The results indicated that students’ difficulties in adequately evaluating diverse and conflicting information do not correlate with their prior epistemic beliefs . These beliefs might be developing from naïve to sophisticated , i.e., from absolutism to multiplism to evaluativism (which were measured using a test developed by van Strien et al., 2012a ).

van Strien et al. (2014) investigated the effects of prior attitudes on how students deal with conflicting information in multiple texts, indicating that students with strong prior attitudes were significantly more likely to write essays that were biased toward their prior attitudes. Moreover, students with strong attitudes took explicit stances and used large proportions of information not presented in the sources in their essays, while students with neutral attitudes wrote syntheses and used more information from the given documents.

To gain a deeper insight into the role of experience in the evaluation phase of the information search process and into the development of beliefs influencing the evaluation of information, Johnson et al. (2016) found significant differences between first-year vs. third-year undergraduates regarding the factors that influence their judgment of the trustworthiness of online information. The results indicate that the more advanced students were not only more sophisticated in evaluating information sources but also more aware in terms of making use of the evaluation criteria.

Likewise, Hsu et al. (2014) examined how students’ different levels of development of their scientific-epistemic beliefs impact their online information searching strategies and behaviors. They divided undergraduates and graduates into two groups depending on whether they employed a naïve or sophisticated strategy. They measured students’ self-perceived online searching strategies and video recorded their search behaviors. Students with higher-quality scientific epistemic beliefs showed more advanced online searching strategies and demonstrated a rather meta-cognitive search pattern.

Mason et al. (2014) studied whether topic-specific prior knowledge and epistemological beliefs influence visual behavior when reading verbally and graphically presented information on webpages. They found that readers made a presumably implicit evaluation of the sources they were confronted with. University students with more elaborated topic-specific epistemic beliefs spent more time on graphics in the context of more reliable sources and increased their knowledge of the topic.

The study of Kahne and Bowyer (2017) is of particular interest for our analysis, as they took policy positions into consideration, an aspect which plays an important role in the task scenario we administered to our test participants (see section “Research Questions”). In their survey of young adults, which is representative for the U.S., they asked participants to judge the veracity of simulated online postings. Controlling for political knowledge and media literacy, their main finding was that the alignment of statements with prior policy beliefs is more decisive for the evaluation of information quality than their accuracy.

Summing up, from the findings reported in recent literature, we register several commonalities in respect to the relation between beliefs and the evaluation of internet-based information . First, information as such and especially information encountered on the Internet was generally recognized and processed on the basis of beliefs and attitudes. Initially, students were always inclined to consider information trustworthy that corresponds with their own (prior) knowledge, whereas they tended to neglect conflicting information. Other biasing factors were prior beliefs (attitudes), which were of comparatively greater impact on the ascription of quality of information in terms of credibility, reliability, plausibility, or trustworthiness. Students appear to be liable to believe and to use information sources in line with their previous convictions, i.e., to avoid “cognitive dissonances” ( Festinger, 1962 ). In addition, the impact of these factors is moderated by their strength (i.e., attitude strength). All in all, well-developed and more advanced (domain-specific) prior knowledge and epistemic beliefs seem to positively influence the quality of students’ Internet searches and (online) information processing.

Research Questions

In the studies we referenced above, the question of whether (prior) beliefs and attitudes are personal traits or states and to what extent they may change remains open. We do not yet know whether (prior) beliefs and attitudes will change during the information acquisition process, and if so, under what circumstances. Our study aims to shed some light on the answers to these questions.

More specifically, based on the analyses of the current state of international research (see Section “State of Research on Beliefs and Their Impact on (Online) Information Processing”), we developed an analytical framework for our study as presented in Figure 1 , and specify the following research questions ( RQs ):

The analytical framework of the study including the research questions ( RQ ).

(I) The Relationship between Beliefs and Decision-Making

RQ1: Students’ beliefs at the beginning of task processing

• • Do the students indicate that they held certain beliefs before they began the performance assessment?
• • Is it possible to identify distinct types based on these beliefs?

RQ2: The relationship between students’ beliefs and their reasoning process as well as their final decision (written task response)

• • At which point in time during task processing did the students make their decision?
• • Do the students’ beliefs affect their decision-making process?
• • Is it possible to identify distinct profiles of decision-making?
• • Which reasoning approaches become evident that may influence the decision-making of the participants?

(II) Change of Beliefs While Solving the Task.

RQ3: Interaction between students’ beliefs and the processing of the given information (in the task)

• • Do the students’ beliefs change as they progress through the task and encounter multiple new information sources along the way (which could indicate that the processed information influences the students’ beliefs)? If so, to what extent is this reflected in their final decision (written task response)?

Conceptual and Methodological Background

Critically reasoning from multiple sources of information.

Students’ skills in judging (online) information are of central importance to avoid the acquisition of erroneous domain-specific and generic knowledge ( Murray and Pérez, 2014 ; Brooks, 2016 ). The abilities involved in finding, accessing, selecting, critically evaluating, and applying information from the Internet and from various media are crucial to learning in a globalized digital information society ( Pellegrino, 2017 ; List and Alexander, 2019 ). Students need critical reasoning (CR) skills to judge the quality of the information sources and content they access inside as well as outside of higher education ( Harrison and Luckett, 2019 ). Students need CR to recognize easily available biased and counterfactual information, withstand manipulation attempts ( Wineburg and McGrew, 2017 ; McGrew et al., 2018 ), and avoid generating erroneous domain-specific and generic knowledge or arguments. 1

In our study, we follow the definition of CR and its facets as described in Zlatkin-Troitschanskaia et al. (2019b) . CR is defined as students’ (I.) identification, evaluation, and integration of data sources; (II.) recognition and use of evidence; (III.) reasoning based on evidence, and synthesis; (IV) (causal and moral) recognition of consequences of decision-making, which ultimately lead to (V) the use of appropriate communicative action . The performance assessments used in this study to measure CR (see next section) are based on these five theoretically driven central facets of this definition of CR. Students’ ability to critically reason from multiple sources of information as a specific representation of CR was measured within this assessment framework.

Assessment Frameworks for Measuring Critical Reasoning

Valid measurement of CR skills is an important component of a program of research on how CR can be effectively promoted in higher education. Moreover, as part of a validity argument, CR’s relation to other related constructs needs to be examined. Based on existing psychological learning models ( Mislevy, 2018 ; Pellegrino, 2020 ), analyses of this kind can provide a significant contribution to developing appropriate explanatory approaches to CR. Despite the urgency of this topic for higher education ( Harrison and Luckett, 2019 ), theoretically sophisticated CR learning and performance assessment tools have so far been developed by only a few projects internationally (for an overview, see Zlatkin-Troitschanskaia et al., 2018a ).

Multidimensional and multifaceted (meta)cognitive higher-order (procedural) skills, such as CR, can be validly measured with closed-format tests to a limited extent, as selected-response items fall at the lower end of the ‘lifelike fidelity scale.’ Multiple-choice tests predominantly assess declarative and conceptual/factual knowledge (e.g., Braun, 2019 ). As Liu et al. (2014) and Oser and Biedermann (2020) documented, there are several closed-format tests for assessing CR (or related constructs). One main shortcoming of tests of this kind is their limited face validity, ecological validity, or content validity ( Davey et al., 2015 ). They usually demonstrate (extremely) strong correlations with tests focused on general intellectual ability [e.g., intelligence tests or the Scholastic Aptitude Test (SAT)], but they tend to fail to measure more specific procedural skills regarding the use and the evaluation of information sources used for learning in higher education. Well-established CR assessments have been based on standard-setting research ( Facione, 1990 ; Facione et al., 1998 ), but have used multiple-choice formats and brief situational contexts and have assessed generic minimal inferencing abilities. 2 Despite the broad use of this test type in educational assessment, it remains unclear to what extent these tasks are ecologically valid and whether students can transfer the measured abilities to more authentic and complex requirement situations.

At the other end of the assessment spectrum are traditional essay prompts with open responses and rubric scoring. Their suitability for assessing CR based on multiple sources of information in particular, may be limited by challenges in objective scoring and the brevity of the prompt ( Zlatkin-Troitschanskaia et al., 2019b ). While ecological validity in particular is especially limited in standardized tests ( Braun, 2019 ), CR can be more adequately measured through performance assessments that simulate the complex environment students find themselves in ‘in everyday life,’ and provide an addition to standardized measures, as they are better suited to reflect current contexts and learning conditions inside and outside of higher education ( Oliveri and Mislevy, 2019 ; Shavelson et al., 2019 ).

So far, to measure university students’ performance on concrete, real-world tasks and to tap their critical thinking skills, the Council for Aid to Education (CAE) has developed the Collegiate Learning Assessment (CLA) ( Klein et al., 2007 ), which was also used in the Assessment of Higher Education Learning Outcomes study, and has launched a refined performance test on CT, the CLA+. The assessment contains an hour-long performance task and a half-hour set of multiple-choice items so as to produce reliable individual student scores ( Zahner, 2013 ). The CLA+ is available internationally for various countries ( Wolf et al., 2015 ). It has been used in the United States and was also adapted and used in Finland 3 , Italy, and the United Kingdom ( Zahner and Ciolfi, 2018 ), and has undergone preliminary validation for Germany ( Zlatkin-Troitschanskaia et al., 2018b ). This computer-delivered assessment consists of a performance task where students are confronted with a complex scenario. Additionally, they are presented with a collection of documents with further information and data that should be used to properly evaluate the case and decide on a course of action. The test has an open-ended response format and is complemented by 25 selected-response questions on separate item stems. According to Wolf et al. (2015) , the assessment measures the following student abilities: Problem-solving and analysis, writing effectiveness, writing mechanics, reasoning scientifically and quantitatively, reading critically and evaluatively, and critiquing an argument.

Other assessments that were recently developed for higher education, such as HEIghten by ETS 4 or The Cap Critical Reasoning test, can be considered knowledge-based analytical-thinking, multiple-choice tests 5 and do not encompass any performance tasks (for an overview, see Zlatkin-Troitschanskaia et al., 2018a ).

The iPAL Study on Performance Assessments of Higher Education Students’ CR

In iPAL (international Performance Assessment of Learning), an international consortium focuses on the development and testing of performance assessments as the next generation of student learning outcome measurements ( Shavelson et al., 2019 ). The researchers address the question how performance assessments can enhance targeted student learning beyond rote memorization of facts and actively foster students’ acquisition of 21st century skills (including CR). The subproject presented here is designed to measure higher education students’ CR by simulating real-life decision-making and judgment situations ( Shavelson et al., 2019 ).

The German iPAL subproject follows a criterion-sampling measurement approach to assessing students’ CR. Criterion-sampled performance assessment tasks present real-world decision-making and judgment situations that students may face in academic and professional domains as well as in public and private life. Test takers are assigned a role in an authentic holistic scenario and are given additional documents and links to Internet sources related to the topic of the task (presented in different print and online formats) to be judged in respect to their varying degrees of trustworthiness and relevance. The skillset tapped by these tasks comprises skills necessary to critically reason from multiple sources of information, i.e., to critically select and evaluate (online) sources and information, and to use them to make and justify an evidence-based conclusive decision.

In the German iPAL study, we developed a performance assessment with a case scenario (renewable energy) [comprising 22 (ir)relevant, (un)reliable and partly conflicting pieces of information]. This newly developed computer-based performance assessment was comprehensively validated according to the Standards for Educational and Psychological Testing ( American Educational Research Association [AERA], 2014 ; see Zlatkin-Troitschanskaia et al., 2019b ; Nagel et al., 2020 ). Validity evidence was gathered (i) by evaluating the test-takers’ responses to the performance assessment (for details, see Shavelson et al., 2019 ; Zlatkin-Troitschanskaia et al.,2019b ), (ii) a semi-structured cognitive interview, and ( iii ) an additional questionnaire on the students’ personal characteristics such as prior knowledge, general intellectual ability, and media use behavior (for details, see Nagel et al., 2020 ).

In the following, we focus on the validation work conducted in (ii) cognitive interviews and present the analyses of transcripts of these cognitive interviews and corresponding results. To strengthen our validity argument (in the sense of Messick, 1994 ; Kane, 2013 ; Mislevy, 2018 ), we additionally refer to the particular findings from (i) to demonstrate how students’ beliefs and reasoning processes as identified in the cognitive interviews are related to their task performance (written final response on the case presented in the task).

Materials and Methods

In this section, we first describe the entire study, including the performance task and the other assessments applied, before presenting the sub-study of the cognitive interviews and its results.

Instruments

The performance task.

To assess students’ CR and their ability to critically reason from multiple sources of information, the German iPAL study developed and tested the “Wind Turbine” performance task. This computer-based assessment consists of a realistic short-frame scenario that describes a particular situation and requests a recommendation for a decision based on information provided in an accompanying document library (including 22 snippets and sources of information of different types; e.g., Wikipedia articles, videos, public reports, official statistics). These information sources, on which the students are to base their decision recommendation, vary in their relevance to the task topic and in the trustworthiness of their contents (for detailed descriptions of the performance task, see Shavelson et al., 2019 ; Zlatkin-Troitschanskaia et al.,2019b ).

In this case scenario, test takers were assigned the role of a member of the municipal council of a small town confronted with the opportunity to build a wind farm on its grounds. They were asked to review the information sources provided in the task and, based on the evidence, to write a policy recommendation for a course of action, i.e., to recommend to the city council whether or not to permit the construction of the wind turbines in its agricultural countryside (for more details, see Shavelson et al., 2019 ; Zlatkin-Troitschanskaia et al., 2019b ).

Accompanying Assessments

To control for task- and topic-related prior knowledge, we used a short version of the WiWiKom test, which was comprehensively validated in the representative nation-wide WiWiKom study as an indicator of knowledge in economics and social sciences ( Zlatkin-Troitschanskaia et al., 2019a ). As two indicators of general cognitive ability, the scale “Choosing figures” of the Intelligence Structure Test (IST-2000 R, Liepmann et al., 2007 ) as well as the grade of university entrance qualification were used in the present study (for details, see Nagel et al., 2020 ). The participants’ levels of interest in the task topic and case scenario (renewable energy) as well as their test motivation were also assessed in this study using two five-point-Likert-type scales (validated in the previous studies cited).

Furthermore, socio-demographic information and personal characteristics (e.g., scales on ‘media use,’ ‘need of evaluation,’ ‘information overload’; for details, see Nagel et al., 2020 ) expected to affect test performance were collected. Indicators of relevant expertise in the context of solving the performance task, such as completed commercial or vocational training, were also surveyed, as they might also influence task performance.

Study Design and Validation

To test and validate the “Wind Turbine” task in accordance with the Standards of Educational and Psychological Testing ( American Educational Research Association [AERA], 2014 ), assessments were conducted with a total of 95 undergraduate and graduate students from different study domains (e.g., business and economics, teacher education, psychology) at a German university ( Shavelson et al., 2019 ; Zlatkin-Troitschanskaia et al., 2019b ).

The students worked on the task in a controlled laboratory on computers configured specifically for this assessment and had no access to other resources to solve the task. The study was carried out in small groups on several dates under the supervision of experienced test administrators.

The total test time for the performance task was 60 min. The time limit put the participants under pressure, which led to them not having enough time to study all given sources intensively. Instead, it required them to decide which sources and contents to select and review more thoroughly, considering their relevance, validity, and trustworthiness. After the performance task (and a short break), the participants were asked to work on the accompanying assessments. The participants received an incentive of €20, and were offered an individual feedback on their test results.

Test performance was scored using a 6-point Likert-type anchored rating scheme based on the CR definition with 5 dimensions and 23 subdimensions (Section “Sample and Data”; for details on scoring, see Zlatkin-Troitschanskaia et al., 2019b ). The individual task responses, i.e., short essays, were randomly assigned to two of four trained raters, and the written responses were evaluated according to this newly developed and validated scoring scheme with behavioral anchors for each sub-category. Two raters independently evaluated the participants’ task responses, and a sufficient interrater agreement was determined (Cohen’s κ > 0.80; p = 0.000, for the overall test score).

In terms of psychometric diagnostics, the student response when solving the performance task (well-founded written final decision) was interpreted as a manifestation of their latent (meta)cognitions. The students’ task performance, i.e., their written responses, were regarded as valid indicators of the students’ ability to critically reason from multiple and contradictory sources of information (in the sense of the construct definition, see Section “Critically Reasoning from Multiple Sources of Information”). The theoretically hypothesized multidimensional internal structure of this construct was supported empirically using confirmatory factor analysis (CFA) ( Zlatkin-Troitschanskaia et al., 2019b ).

As theoretically expected, analyses of the task performance did not identify any significant domain-specific effects among students from different study domains ( Nagel et al., 2020 ). This also holds true for prior knowledge from previous vocational training, which showed no significant effect on the test results. As the performance task was developed to measure generic CR skills, this finding indicates that as expected, the assessment is not domain-specific. However, with regard to theories in learning and expertise research, it could be assumed that domain-specific expertise, though acquired within a certain domain, can be transferred to generic problems or tasks ( Alexander, 2004 ). In this respect, these results indicate that students may have deficits in their (meta)cognitive abilities that would enable them to transfer their prior knowledge and skills to the new context encountered in the performance task. Overall, the results from these validation studies provide evidence of the technical quality of the developed performance task and provided evidence as to the test’s construct validity and reliability ( Zlatkin-Troitschanskaia et al., 2019b ; Nagel et al., 2020 ).

Cognitive Interviews and Qualitative Analyses

The analyses of test performance per se do not permit us to draw valid conclusions on students’ underlying response processes when performing this task. In accordance with our construct and test definition, we expect that while working on the performance task, the test participants selected and evaluated the given information with the goal of finding relevant and reliable/trustworthy information for their evidence-based reasoning, decision-making, and final conclusion in the written response. To investigate how the test participants dealt with multiple contradictory sources, to what extent they integrated and evaluated given information during their reasoning and decision-making process, and which individual factors influenced their response processes, a semi-structured cognitive interview with stimulated recalls was conducted immediately after the performance assessment with a subsample of participants (see next section). The participants were first shown a screen displaying all 22 documents included in the document library one after the other. The students reflected and commented on, for instance, whether they considered the source in question and the information given therein to be relevant and/or credible (and why), and whether they used or ignored this source and information (and why). Particular attention was paid to controlling whether the students were aware of their task topic-related beliefs and attitudes and if so, whether they were aware to which extent these influenced their critical reasoning while processing the task, for example resulting in selective inclusion of the given information. The interviews took approximately 80 minutes and were recorded for later transcription.

The interview questions included, for instance, whether a test participant held task topic-related beliefs about wind power, the environment, etc. prior to the performance assessment, and to what extent previous experience, individual knowledge, attitudes and beliefs relevant to the task topic influenced the students’ information selection, evaluation and decision-making. In particular, the students reported at which point in time during the task processing they made their decision whether or not to suggest to the municipal council to allow the building of the wind farm (for instance, indicating that many students made their decision before they had even read the given information; see Section “Results”). The cognitive interviews also indicated that the performance task with its task prompt is clear and suitable for the objectives of the presented study.

The evaluation of the data from the cognitive interviews in the German iPAL project was carried out using the software MaxQDA. Based on the cognitive interview protocols, a differentiated category system was developed and validated. More specifically, the qualitative analysis of the cognitive interviews was guided by grounded- and data-driven theory for developing a coding scheme ( Strauss and Corbin, 1998 ).

The iterative process of coding consists of (1) open, (2) axial ( Strauss and Corbin, 1990 ) and (3) selective coding. At first, an open coding was used to access the data that did not yet follow any schematics. In the subsequent step, first categories were identified, such as the students’ beliefs at the beginning of the task or at which time point during the task processing the decision was made. Then, all interviews were analyzed and coded based on the defined categories. The coding scheme highlighted the points of reference regarding different information sources used by the students in their interviews. It linked the use of different sources to the students’ reasoning processes while reaching their decision, making it possible to derive and generalize response patterns. Coding development was complemented by an analytical approach of constantly comparing phenomena within the dataset (minimum and maximum contrast between the phenomena). Selected codes with a focus on student beliefs are presented in this paper (see Section “Results”).

The classification of the participants into the three profiles described in Section “Results” was based on a combination of criteria from the category system. These were primarily: (1) at what point during the assessment they made their decision (reported in the cognitive interviews), (2) their decision-making process (pros and cons; intuitively, based on original opinion; based on a specific source, etc.) and (3) the strength of their beliefs (strong personal beliefs primarily about nature conservation/animal welfare, etc. and general personal identification with the task topic). As the participants were classified into profiles based on a combination of these three categories, participants classified into different profiles may share certain attributes (e.g., listing pros and cons).

Sample and Data

The semi-structured interviews were conducted with a purposefully selected subsample 6 (which is part of the overall sample used in the German iPAL study) of 30 university students from one German university and from different courses of study. With this subsample, we aimed to include students from all study domains represented in the main sample in the cognitive interviews as well. Accordingly, the subsample consists of about 50% students of economics education, while the other half comprises students from other study domains (economics, psychology, and geoscience). Another important criterion for purposeful sampling was to include as wide a range as possible in terms of participants’ prior study experience and other personal characteristics that may influence students’ task topic-related beliefs, attitudes and knowledge, and their task performance. Accordingly, the subsample consists of students from both undergraduate and graduate programs and in different study semesters. To gain first indications of the possible impact of knowledge and skills achieved during academic studies, we focused on more experienced students. The average duration of studies to date among subsample participants was therefore 5.1 semesters, indicating that the students were fairly advanced in their respective study programs. Additionally, the university entrance qualification (with an average grade of 2.1; range from 1.0 = best to 6.0 = worst; n = 25 ∗7 . To control for the possible impact of pre-university education and practical experience, we included students with completed vocational education and training (11 students had completed an apprenticeship before beginning their academic studies; n = 29 ∗ 7 ). The average interviewee’s age was 24 years; 21 students were female – these proportions were similar to those in the overall sample.

Despite this purposeful sampling procedure, as participation in this study was voluntary, our sample cannot be considered representative. However, no significant deviations from the entire student population described in Nagel et al. (2020) were found with regard to the socio-biographical characteristics (e.g., gender and age).

Prior Findings on Test Performance and Additional Assessments

The students achieved an average intelligence-test (IST) score of 17.18 points (out of a maximum score of 40 points; n = 28 ∗ 7 ) and an average economics knowledge test score of 10.46 points (out of a maximum score of 15 points; n = 23 ∗ 7 ). Only four students stated that they had previous practical experience with Wind Turbines. Most students reported a low to medium level of task topic-related previous knowledge while a high level of knowledge on wind turbines was very rare ( n = 1).

The mean performance on the task was 3.52 points with 6 points being the highest possible score (for the scoring, see Zlatkin-Troitschanskaia et al., 2019b ). The median number of information sources students used in their written statements was 7 (out of 22 information sources given in the task). The written argument-based statements within the scope of the task processing differed in length, which was on average 426 words, with a maximum of 866 words and a minimum of 68 words, indicating a high deviation ( SD = 196 words) within the sample.

In the following, these results were used as external criteria to demonstrate how the following results from the analyses of the cognitive interviews correspond to these results of the quantitative analyses of the test scores.

The Relationship Between Beliefs and Decision-Making

Rq1: students’ beliefs at the beginning of task processing.

In the cognitive interviews, the students were asked whether they had been aware of their task topic-related beliefs prior to working on the task and if so, whether they were aware that their personal beliefs may have influenced their decision in the performance task and how they believed this influence may have shaped their response. Most participants ( n = 23) stated that they had already held certain beliefs on the task topic before beginning the task . For instance, one participant stated: “[…] I think I would have recommended this from the beginning because this is also a topic I hear about in the media from time to time, so that I already have a personal opinion about wind power and energy “(ID15). In response to the question whether his personal beliefs had influenced his response, interviewee ID7 stated: “Sure, because then I did not even look at the controversial sources at all and, that is… for example, if I believed that the bats from source 21 were extremely important, then of course I would have looked at the source.” Seven participants ( n = 7) reported that they did not have any prior beliefs about the topic of the performance assessment.

In terms of distinct types based on the reported beliefs, both groups of students – those who indicated prior beliefs and those who did not – can be further distinguished into two subgroups each (i) depending on the students’ positive or negative stance toward wind turbines , which vary considerably in stance strength and (ii) which can also be linked to a more economics-focused or a more ecologically oriented reasoning perspective (see Section “The Relationship between Beliefs and Decision-Making”).

R2: The Relationship Between Students’ Beliefs, Their Decision-Making Process and Their Final Decision

Time of the decision-making and types of decision (intuition-based vs. evidence-based).

In the cognitive interviews, the students reported at which point in time while working on the performance task and processing the information they made their decision as to whether renewable energies should be promoted or not in the given case (see Table 2 ). About one third of the students ( n = 8) made their decision at the beginning of the task, after having read the scenario, even though they had not yet read or considered the given information at all or only very briefly.

Time of decision-making and type of decision.

Another group of students ( n = 8) used the given sources and made their decision mostly after (more or less thoroughly) looking through the information provided. For instance, when asked when he had decided in favor of or against the construction of wind turbines, interviewee (ID 1) stated “ yes […], I actually knew from the beginning when I went through this [task] what direction my statement would go in. ” Interviewee (ID 23) made his decision while working on the audiovisual information: “ So after I watched the videos […] I changed my opinion .”

In contrast, other students made their recommendations after having reviewed the information material and after weighing up the pros and cons ( n = 9), as it is the case with, for example, interviewee (ID 7): “Interviewer: So that means that you first read all the sources and all the arguments? Participant: First the pros and cons, and only then I had a feeling .” This finding indicates that for some students the creation of pro and contra lists was an important step in their decision-making process. However, not all of those who made their decision comparably late in the task-solving process stated that they had done so based on weighing up the pros and cons: five students indicated that they made a late but still intuitive decision .

Overall, with regard to the time of decision-making , four types can be distinguished among the participants ( Table 2 ), which differ in terms of intuition-based vs. evidence-based decision-making as well as the extent to which the given information and pros/cons were considered or ignored.

Students who made a late intuition-based decision ( n = 5) performed worse, with an average test score of 3.25. Students who made their decision at the end of the task based on weighing pros and cons ( n = 9) performed better compared to all other participants, with an average test score of 3.83. Noticeably, there were hardly any differences in task performance between the students who decided intuitively at the beginning of the task and the students who weighed up pros and cons and decided at the end of the task.

The Relationship Between Students’ Beliefs and the Decision-Making Process: Profiles of Decision-Making

Regarding the question to what extent students’ were aware that their beliefs impact their decision-making process and whether distinct profiles of decision-making can be determined, among the 30 study participants, we identified students who indicated that their previous beliefs played a major , minor or no role in their decision-making process. Combined with the time at which they made their decision, we distinguished three profiles of decision-making:

Profile 1 “determined” : Participants who ignored the given information and made their decision solely based on their individual beliefs, almost immediately after having read the task ( n = 7). For example, (ID5) stated: “I wouldn’t have made a recommendation that goes against my gut instinct. For example, I think that even if the sources had been chosen in such a way that they would have given me a negative impression, I am not sure whether that would have caused me to change my initial positive stance. I simply couldn’t just ignore my background knowledge and my personal attitude when giving my recommendation at the end.”

Profile 2 “deliberative” : Participants who decided contrary to their task topic-related beliefs, and changed their decision after having read the information provided in the task ( n = 11), as well as participants who stated that they held certain beliefs at the beginning of the task but weighed up pros and cons while processing the task and made their decision based on these considerations ( n = 5). The two cases were merged into one profile as students in both cases stated that they held certain beliefs but made their decision based on the pros and cons of the evidence rather than on those beliefs.

There were some differences within this decision-making profile. For instance, some students switched between being in favor of or against the construction of wind turbines while working on the task: “ So basically I’m for it and then while I was writing this I just started to waver, you have to list the negative things and then I doubted it for a moment but then I finally decided in favor at the end. ” (ID 8); other students changed their prior opinions by reflecting on their own beliefs in the context of the given information: “ At the beginning I would have said yes [impact of belief on decision-making]. But then I tried to be as unbiased as possible, or rather to be subjective in my role as a member of the council. And then I kind of abandoned my [initial] decision and my personal belief. ” (ID 17).

Profile 3 “open minded” : Participants who did not state any prior beliefs, took note of the provided information, and made their decision after considering pros and cons ( n = 7). Interviewee (ID7) stated that he had had no prior beliefs before starting the task, and that he made his decision after considering the given information and making a pro and con list: “ No, I couldn’t decide at the beginning, it just happened toward the end of the argumentation. Well, I was not for or against it from the beginning. I just did not know how to decide. ”

Since the participants were classified into profiles based on a combination of the scoring categories (see Section “Cognitive Interviews and Qualitative Analyses”), participants classified into different profiles may share some attributes (e.g., listing pros and cons) and there may be some overlaps between the profiles.

The Relationship Between the Decision-Making Profiles and Task Performance (Test Scores) as Well as the Results of Additional Assessments

Noticeably, the participants in profile 3 on average achieved higher test scores than the other two profiles ( Table 3 ). Students who based their decision on their beliefs (profile 1) performed worse compared to other participants (profile 3). In terms of the average test score, the deviation between these two profiles (1 and 3) was more than 1 point.

Means of task performance of different profiles.

Upon further characterizing the three profiles, we found additional differences between the groups of students in terms of the number of information sources used and the number of words in the final recommendation statements, which differ greatly ( Table 4 ). Compared to students who made a decision based on their beliefs (profile 1), the average number of information sources used was 3.25 points higher for students who changed their beliefs (profile 2) and 2.86 points higher for students of profile 3. The mean number of words in the written final recommendation statements also varied heavily. Remarkably, the responses of “deliberative” students (profile 2) were the shortest with an average of 370 words. “Determined” students (profile 1) who did not change their beliefs wrote on average 33 more words than “open minded” students (profile 3) with a mean of 473 words.

Characteristics of the three decision-making profiles.

With regard to personal characteristics, there were no significant differences in the intelligence test scores for the three profiles ( Table 4 ). The same was true for performance in the economics knowledge test, with results ranging from 10.20 to 10.65 points (on a 15-point scale).

Students who made their decision based on evidence and pros/cons, despite their beliefs or without considering previous task-related beliefs, tended to be older (profile 2: 3.58 years older on average; profile 3: 3.26 years older on average) than “determined” (profile 1) students. There were no significant differences in terms of gender, pre-university education (vocational training or university entry qualification grade) or degree course, which does not indicate any substantial influence of prior education on the response processes.

Task-Topic Related Attitudes and Their Relationship to Reasoning Processes and Decision-Making

Another approach to identifying certain beliefs and their possible relationship to information processing and critical reasoning was to analyze students’ task-topic related attitudes and their impact on reasoning approaches when solving the performance task. In this respect, the reasoning lines identified in the cognitive interviews (as well as in the written task responses) can be categorized as follows:

(1) The first category differentiates between primarily economics-focused or ecologically oriented reasoning lines. Twelve students’ recommendations had a primary economical focus in their reasoning, while 18 students relied more on ecological aspects and sources presenting an ecological perspective.

Remarkably, students who were in favor of building wind turbines tended to choose an economics-focused reasoning line, while students against the construction chose an ecologically focused perspective ( Table 5 ). An example for an economical reasoning line can be seen in the following statements: “ The trade tax to be paid by the operator could be sensibly invested in the modernization of facilities, the infrastructure of the place and the marketing of the local recreation area. This source of income seems to be important for the community, especially in the future, against the background of an increasingly dwindling agriculture” (ID 13); “In my opinion, the offer should be accepted, as the positive aspects outweigh the negative ones and, in general, the construction of wind turbines would mean a macroeconomic, long-term benefit for the community. In addition, it is an investment in infrastructure.” (ID 25). In contrast, an example for an ecological line of reasoning and their relationship to information processing and decision-making can be seen in the following statement (ID 26): “ That caused me to have fewer choices, and I had already had the notion in mind that wind turbines are good and nuclear power plants are bad, which is why I said from the very outset that yes, no matter in which form, more renewable energy should be produced and, well, that’s why I said all along that that would be the most sensible result in my opinion, without any of those arguments.”

Economic- and ecological-focused reasoning and decision against or in favor of wind turbines at the end of the tasks.

(2) The students’ decision-making processes and (final) recommendations can also be categorized in terms of the extent to which the specific situation described in the task was considered . While half of the participants took the task-specific perspective of the local council and the current situation of the city into account ( n = 15), other students choose a more general approach in making a recommendation for or against wind turbines ( n = 15).

One example for considering of local conditions can be found in the statement of participant (ID 7): “ I consider the construction of the wind turbines in the north of the municipality to be an incalculable risk, as the tertiary sector and especially the tourism that goes along with it represent an important source of income for the town. I think it makes much more sense to locate the wind turbines in the west. Farmers who live there, such as Mr. Anders and Mr. Bender, should welcome an additional source of income besides agriculture, so that they should agree to the construction of the wind turbines.“ A more general approach is expressed in the statement of participant (ID 16): “The fact that wind energy is initially a clean and environmentally friendly way of generating energy speaks for the installation of wind turbines. In addition, there are also economic reasons for this, as good money can be made from the rent that incurs when a wind turbine is installed. […].”

While the majority of students who took the task-specific current situation of the city into account tended to express a negative attitude about wind turbines, students who took a more general reasoning approach were rather in favor of building wind turbines ( Table 6 ).

Perspective of reasoning (local council included or not) and decision against or in favor of wind turbines at the end of the tasks.

The Relationship Between the Reasoning Approaches and Task Performance (Test Scores)

In terms of task performance, no significant differences were found between the students with different reasoning approaches, although students who chose economics-focused reasoning achieved slightly higher performances than the other students. When taking into account the positive vs. negative stance toward wind turbines at the end of the task, however, the difference in task performance of students with ecological-focused reasoning is about 0.8 points, whereas the difference in the group with economic-focused approach is only 0.1 point.

Change of Beliefs While Solving the Task

Rq3: interaction between students’ beliefs and processing of the given information.

Looking at the time of decision-making, we found that some students changed their opinion about the construction of wind turbines (once or several times) while processing and working on the task, while others did not. While 14 interviewees reported that they did not change their opinion about the wind turbines over the course of their task solving, 12 interviewees changed their opinion after processing of information given in the task ( Table 7 ). Four participants claimed that they had not been initially disposed either way. Both groups of students–those who changed their opinion and those who did not–can each be further distinguished into two subgroups depending on their positive or negative stance toward wind turbines, which vary considerably in size. Within the group with no change of opinion, participants who had voted against the construction on wind power plants at the beginning of the tasks and remained negative ( n = 3) can be distinguished from participants who had a positive stance toward wind turbines before and after completing the task ( n = 12). We can also differentiate between students who have changed their opinion during working on the task. Some students initially had negative attitudes toward wind turbines, but changed their opinion during the task processing and in the end voted in favor wind turbines ( n = 2). The same applies to participants who were in favor of constructing wind turbines at the beginning, but ultimately spoke out against wind turbines ( n = 9).

Change of opinion.

The Relationship Between a Change of Students’ Beliefs and Task Performance (Test Scores)

There was hardly any significant difference in the test score of the two groups, although students who did not change their opinion performed slightly better than students who changed their opinion: the difference in task performance was about 0.7 points.

Discussion and Conclusion

Summary and interpretation of results.

The data from the cognitive interviews on the students’ beliefs, information processing and reasoning processes make a valuable contribution to explaining the students’ CR abilities and the complex interplay between their underlying thought processes and task topic-related beliefs. In the interviews, most participants expressed that they were aware of holding certain beliefs at the beginning of task processing (RQ1). The results of the qualitative analysis of the cognitive interview protocols indicated that the students’ task topic-related beliefs had an influence on their selection, critical evaluation and use of information as well as on their reasoning process and final decision (RQ2). As an additional decisive contribution to existing research [see Section “State of Research on Beliefs and Their Impact on (Online) Information Processing”], we provide initial evidence that some students’ task topic-related beliefs changed over the course of task processing, indicating that the processed information (recognized and reflected evidence and pros/cons) influenced the students’ beliefs to varying degrees (RQ3).

Overall, the evidence from this qualitative analysis suggests a complex reciprocal and changeable relationship between students’ task topic-related beliefs, their processing of new (confirm or deviant) information and their decision-making based on both beliefs and evidence.

More specifically, the types of beliefs and attitudes derived from the cognitive interview data suggest their influence on information processing, reasoning approaches and decision-making. In particular, the students who already had strong task topic-related beliefs at the beginning regarded these as decisive while solving the task. For instance, students who had already made a decision based on their beliefs at the beginning of the task cited fewer sources in their written response (final decision).

Overall, the selection, evaluation, and use of information while working on the task were influenced, in particular, by the participants’ task topic-related beliefs (RQ2) . By contrast, hardly any differences became evident in terms of students’ relevant knowledge. However, the majority of the participants had only little prior knowledge of the subject, i.e., a large amount of the information in the task was new to them. Though most students had a positive or negative stance toward renewable energy in general, their personal beliefs concerning wind energy in particular did not appear to be very firm and well-founded. The few test participants who had already dealt with the subject area in more detail appeared to have more solid personal beliefs about wind energy (RQ1). Furthermore, there were no differences in terms of students’ general interest in the topic. However, two reasoning lines – more ecologically oriented vs. economics-focused approaches – became evident, which appear to influence students’ decision-making processes and final decision.

Remarkably, the students who had more elaborated beliefs prior to processing the task were more likely to come to a decision that contradicted their personal beliefs. For instance, the information on the negative effects of wind turbines on the health of people and animals living in the vicinity of a wind farm (noise emission, bird strike, infrasound) was particularly relevant for these participants when making their decision; they were more astonished by this information than the students who had hardly any prior knowledge about the subject and no well-developed beliefs (RQ2).

Most students started selecting information right away after obtaining an initial overview of the sources presented in the task. The participants’ subsequent evaluation of the given information with regard to the reliability, validity, objectivity, and trustworthiness of the respective sources (as stated in the interviews) does not appear to have had much of an influence on their selection and use of information. In contrast, the participants evaluated the relevance of the sources differently, whereby a large number of the sources that were evaluated as relevant were used to inform their decisions and help them formulate their written recommendations. For instance, in the interviews, the majority of students rated Wikipedia as a less reliable source (of course the exact details vary, but in general, it received rather negative ratings), as Wikipedia pages can potentially be edited by any Internet user. However, the choice as to whether or not to use information from Wikipedia sources was primarily made on the basis of the content of these sources (“ do I want to address bird mortality or not? ”). In contrast, when it came to the evaluation of the public-service broadcaster videos, a large number of participants assessed these videos as trustworthy despite not having watched them, as they considered this source to be particularly reliable.

Overall, in the cognitive interviews it became evident that the students mostly selected and evaluated (or ignored) new information depending on media or source type (i.e., whether they believed that certain types of media and presented sources are relevant and reliable) but not on the particular content/evidence. This finding is in line with previous research reported in the Section “State of Research on Beliefs and Their Impact on (Online) Information Processing” and stresses the importance of epistemic beliefs regarding information sources, which was not a focus of this study and requires further investigations in the particular context of online reasoning (for limitations, see the next section). In addition, this result points to a demand for more observational studies that capture in detail what documents, what parts of these documents, and which content the participants read and comprehend while solving the task.

Although participants used different sources in their statements, most of the students did not compile the information provided to them and weigh the evidence (pros/cons), but rather selected information related to their own beliefs, indicating biased selection, evaluation and use of information (for the confirmation bias, see Mercier and Sperber, 2009 , 2011 ; Metzger et al., 2010 ; Metzger and Flanagin, 2015 ). A (repeated) critical examination of the information and evidence provided did not take place.

Linking the results from the qualitative analyses of the cognitive interviews with task performance further suggests a confirmation bias in reasoning, showing that students who only made their decision based on their beliefs (profile 1) had the worst test scores on average. This was also reflected in the number of sources used. They wrote the longest statements but based on the lowest number of used sources , without sufficiently reflecting on the available information and evidence. This finding is also supported by the lower performance of students who tended to overemphasize a single source while neglecting all contradicting source information (for the authority bias, see Metzger et al., 2010 ; Metzger and Flanagin, 2015 ). Overall, the finding from the qualitative analyses that often no sufficient critical reasoning took place in the decision-making process and that the decision was based on beliefs (and bias) was also reflected in the students’ statements.

In contrast, the students with no early inclination (profile 3) approached more source material neutrally and decided on the incorporation of the information and evidence individually, outperforming the other students in terms of task score. Their statements were less belief-driven since they addressed the specific task scenario and prioritized the town’s needs and restrictions over their personal stance on renewable energy.

As the students only had limited time (60 min) to respond, time pressure also played an important role and forced them to gather relevant information as quickly as possible. If the participants selected the information they intended to read more precisely, worked with it and then used it in their decision-making at an early stage ( right at the beginning ), quickly ( without deliberative thinking ), and consistently ( without changing their minds ), the issue of time pressure apparently did not have much of an effect on their task-solving efforts. The cognitive interviews indicate that for some students, however, selecting suitable information was a major challenge while working on the task (indicating the higher cognitive load; Sweller, 1988 ). These participants often opted to use internal sources as opposed to external sources, indicating that they mostly focused on the information that was available within the task document itself and disregarded the hyperlinks. The majority of participants did not watch the two videos (completely) due to time issues. This aspect also points to some limitations of our study (see next section).

Limitations and Implication for Future Research

Though the study provides some important insights into the complex reciprocal relationship between students’ beliefs and their reasoning and decision-making process, some limitations (besides those related to the sample, see Section “Sample and Data”) must be critically noted, which indicate some perspectives for further research.

While the results of the qualitative analyses pertaining to RQs 1&2 allow for some clear statements about students’ beliefs and their influence on critical reasoning, the findings pertaining to RQ 3 regarding changes in beliefs are still limited. First, in our study, we can only derive conclusions about task topic-related beliefs. These need to be distinguished from general personal (e.g., epistemic) beliefs, which were not analyzed in our study. In prior research, general beliefs usually were seen as a trait that does not change during the course of solving a task. However, measuring epistemic beliefs is considered challenging from a conceptual as well as a methodological perspective, and requires further research ( van Strien et al., 2012a ).

Second, based on the cognitive interview protocols, a clear distinction between a change in task topic-related beliefs and a change of overall opinion could also only be made to a limited extent. Although some students clearly stated that they had beliefs prior to processing the task that influenced their information processing and decision-making, and they had changed their opinion, we cannot conclude, on the basis of the interviews, whether this change of opinion was due to a change in their underlying beliefs . It is also questionable whether students were able to clearly distinguish between their belief, their attitude toward the task topic, and their opinion, and to express this difference in the interviews. This limitation results in an important follow-up for further research: Is a change of opinion accompanied by a change of task topic-related beliefs?

Though the results of both assessed scales on students’ interest in the task topic and students’ test motivation showed (very) high levels among all participants in this study, we noticed some differences in the way students approached the cognitive interviews. While some students were very communicative and talked a lot about their beliefs and task processing, other students gave short answers. Consequently, the cognitive interview protocols vary substantially in length and detail. The results of the qualitative analyses must therefore be viewed critically in terms of this data limitation. For instance, it could not be ruled out that students who did not express that they had topic-related beliefs prior to processing the task may not have deliberatively reflected on this interview question or simply not have wanted to share this information (e.g., due to a bias of social desirability). Despite the use of a standardized guide in the semi-structured interviews, the comparability of the cognitive interview protocols may be limited in this regard.

The task topic may also be not without bias in this respect, since renewable energy can be generally framed in a positive light. For this reason, it can also not be ruled out that students’ responses to the task and their answers to the interview questions were biased in terms of social desirability. However, the fact that some students in our sample were both initially and ultimately against the construction of the wind turbines ( n = 3) may contradict this assumption.

In addition, though (i) the task prompt to write an evidence-based statement regarding the decision for the community should have been clear and strong enough to indicate that a discussion of the evidence (pros-cons) made available in the task was required, and (ii) (very) high levels of assessed interest in the task topic and overall test motivation among participants were determined, a difference among participants in terms of (metacognitively) engaging their critical reasoning skills when solving the performance task can still not be ruled out. Based on prior research, however, it can be assumed that the activation of critical reasoning abilities requires metacognitive skills (e.g., Brand-Gruvel and Stadtler, 2011 ). Therefore, further understanding of students’ (metacognitive) engaging (and other influences) during the decision-making process is required to help identify certain patterns in task processing strategies for this type of performance assessment and to further improve computer-based simulations in terms of their ecological validity and reliability to ensure more authentic assessment (for a critical discussion, see also Mercier and Sperber, 2011 ).

In this context, it is remarkable that the group of students who were aware of the influence of their beliefs – despite the task prompt asking them to include the given information and evidence in their decision-making process-decided to use only information that supported their beliefs (profile 1). These students had already recognized at the beginning of the task processing that their beliefs would have a decisive influence on their decision. If we transfer this finding to other real-life situations, in particular the everyday use of online sources in Internet searching, further research is required as to whether students, when searching for sources and in the context of their university education, also specifically focus on sources and information that confirm their beliefs. In this respect, the identified reasoning profile 1 may lead to an acquisition of biased (domain-specific) knowledge. In contrast, the “open minded” profile 3 approached more information neutrally, outperforming the other students in terms of the scored quality of written statements.

In this context, it is also important to focus on those students who claimed to have certain beliefs on the topic before starting the task but still reviewed all the information given and even partly decided against their beliefs after having regarded all information (“deliberative” profile 2). This profile should be analyzed more in-depth, especially taking into consideration both additional underlying cognitive and non-cognitive student characteristics as well as specific learning opportunities that this group might have had to develop this deliberative reasoning approach. Here, further questions arise: Why did students choose this approach and decide against their beliefs? What personal or contextual factors may have played a decisive role?

The complex relationship between prior knowledge and beliefs also requires further in-depth investigation. Ho et al. (2008) found that task topic-related beliefs interact with the amount and quality of topic-relevant knowledge, whereby the topic-related beliefs may have a stronger impact on decision-making than knowledge. Analogously, the results of our study suggest that in general, no matter how experienced a student is in a topic or how much previous knowledge they had, certain beliefs seemed to be influential and predominant. However, to what extent the beliefs influence students in their approach to a task topic and which aspects were particularly crucial for students to be influenced by their beliefs (e.g., strength of beliefs or additional personal characteristics) must also be analyzed in further research (for an overview, see Brand-Gruvel and Stadtler, 2011 ).

In addition, looking at the differences in the students’ reported reasoning processes, we can conclude that diverse students’ beliefs and attitudes, which were related to the task context and topic to a very different extent (e.g., in the area of sustainability), had an influence on the students’ decision-making and final decision. Based on the data from the cognitive interview protocols, however, we were not able to analyze the complex relationship between beliefs, reasoning approaches and lines of argumentation . Though critical reasoning is indeed related to aspects of argumentative skills, this latter aspect was not the focus of our study (as described in Section “Conceptual and Methodological Background”) and requires further investigation in several regards. Particular investigation of argumentative skills would require a substantial change and further development of the experimental and assessment setting. For instance, there are several performance tests available that specifically focus on measuring argumentative skills (e.g., Argument Structure Test, Münchow et al., 2020 ; Agrument Judgment Test, Münchow et al., 2019 ) and are suitable for discriminant validation of CR assessments, which should be investigated in a follow-up research. In addition, comprehensive qualitative analyses of both the argumentative importance of the material on the one hand as well as (i) arguments (more or less reflective or intuitive, Mercier and Sperber, 2011 ) used by students in their responses and (ii) (new) arguments created by the students themselves based on given arguments in the provided information on the other hand need to be conducted in further studies, and explicitly linked to students’ critical reasoning ability and performance.

Finally, the method of cognitive interviews also has certain limitations in terms of understanding and explaining students’ reasoning processes during task-solving, for instance due to a bias of social desirability (e.g., Kahne and Bowyer, 2017 ) as mentioned above or limited mental recall capacities. However, one central focus of the presented study lies on the investigation of self-awareness of one’s owns beliefs, i.e., whether the students were aware of their beliefs and whether they were aware if their beliefs influencing their perception, evaluation, selection and use of the given information. Hence, cognitive interviews were necessary to gain indications regarding the students (critical) reflection on their thought processes involved in solving the task, i.e., writing a statement. Especially any conclusions about self-awareness regarding one’s beliefs and their relation to decision-making can best be reached by means of stimulated recalls in cognitive interviews, which has been shown to approximate think-aloud methods in the study settings where participants cannot think aloud while processing the task (as in this computer-based test environment).

Follow-up research observing these limitations and implications would provide a better understanding of successful CR and a more significant basis for developing targeted instructional interventions in order to promote students’ CR skills in dealing with new more or less trustworthy or contradictory information.

Data Availability Statement

Ethics statement.

Ethical review and approval was not required for the study on human participants in accordance with the local legislation and institutional requirements. The participants provided their written informed consent to participate in this study.

Author Contributions

OZ-T provided the idea for the study, co-developed the assessment, supervised the analyses, and co-wrote the manuscript. KB co-developed the assessment, supervised the analyses, and was involved in preparing and reviewing the manuscript. JF and DB conducted the analyses, and were involved in preparing the manuscript. SS was involved in the data collection and in the analyses. RS was involved in the development of the performance assessment and in preparing the manuscript. All the authors contributed to the article and approved the submitted version.

Conflict of Interest

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

Acknowledgments

We would like to thank the two reviewers and the editor who provided constructive feedback and helpful guidance in the revision of this manuscript. We would also like to thank all students from the Johannes Gutenberg University Mainz who participated in this study as well as the raters who evaluated the written responses.

Funding. This study is part of the PLATO project, which is funded by the German federal state of Rhineland-Palatinate.

1 In contrast to other concepts related to critical thinking , critical online reasoning (COR) is explicitly limited to the online information environment and includes the specific ability of “online information acquisition”. While there is currently no unified definition of COR, there are numerous definitions of its related construct critical thinking (CT) that include and describe different dimensions or levels. For instance, Oser and Biedermann (2020) distinguish between CT as alertness, CT as immediate reflection, and CT as analysis. Facione(2004 , p. 9) describes CT as “inference, explanation, interpretation, evaluation, analysis, self-regulation” (for further definitions of CT, see Moore, 2013 ; Beck, 2020 ). As Brookfield (1987) emphasizes, “Being a critical thinker involves more than cognitive activities such as logical reasoning or scrutinizing arguments for assertions unsupported by empirical evidence. Thinking critically involves us recognizing the assumptions underlying our beliefs and behaviors ”.

2 One well-known test of this kind is the Watson-Glaser Critical Thinking Appraisal (2002), which comprises tasks on inferences, recognition of assumptions, deduction, interpretation, and evaluation of arguments ( Watson and Glaser, 2002 ).

3 https://ktl.jyu.fi/fi/hankkeet/kappas/copy_of_lyhyesti

4 www.ets.org/heighten/about/critical_thinking/

5 http://practice.cappassessments.com

6 The criteria for selection from the overall sample and inclusion were the participants’ socio-biographical and educational characteristics to ensure: (1) gender balance, (2) age distribution, (3) course of study representation (all), (4) study year/progress (advanced students), and (5) prior education (e.g., completed vocational training).

7 The deviation from the total sample size (30 participants) is due to missing values.

• Alexander P. A. (2004). “ A model of domain learning: Reinterpreting expertise as a multidimensional, multistage process ,” in Motivation, emotion, and cognition: Integrative perspectives on intellectual functioning and development , eds Dai D. Y., Sternberg R. J. (New Jersey: Lawrence Erlbaum Associates; ), 273–298. [ Google Scholar ]
• American Educational Research Association [AERA], American Psychological Association, and National Council on Measurement in Education (2014). Standards for Educational and Psychological Testing. Washington, DC: American Educational Research Association. [ Google Scholar ]
• Beck K. (2020). “ On the relationship between “Education” and “Critical Thinking” ,” in Frontiers and Advances in Positive Learning in the Age of Information (PLATO) , ed. Zlatkin-Troitschanskaia O. (New York, NY: Springer; ), 73–87. [ Google Scholar ]
• Brand-Gruvel S., Stadtler M. (2011). Solving information-based problems: evaluating sources and information. Learn. Instr. 21 175–179. 10.1016/j.learninstruc.2010.02.008 [ CrossRef ] [ Google Scholar ]
• Bråten I., Strømsø I. H., Salmerón L. (2011). Trust and mistrust when students read multiple information sources about climate change. Learn. Instr. 21 180–192. 10.1016/j.learninstruc.2010.02.002 [ CrossRef ] [ Google Scholar ]
• Braun H. (2019). Performance assessment and standardization in higher education: a problematic conjunction? Br. J. Educ. Psychol. 89 429–440. 10.1111/bjep.12274 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Brookfield S. D. (1987). Developing Critical Thinkers: Challenging Adults to Explore Alternative Ways of Thinking and Acting. San Francisco, CA: Jossey-Bass. [ Google Scholar ]
• Brooks C. (2016). ECAR Study of Students and Information Technology. Louisville KY: ECAR. [ Google Scholar ]
• Chiu Y.-L., Liang Y.-C., Tsai C.-C. (2013). Internet-specific epistemic beliefs and self-regulated learning in online academic information searching. Metacogn. Learn. 8 235–260. 10.1007/s11409-013-9103-x [ CrossRef ] [ Google Scholar ]
• Chiu Y.-L., Tsai C.-C., Liang J.-C. (2015). Testing measurement invariance and latent mean differences across gender groups in college students’ internet-specific epistemic beliefs. Austr. J. Educ. Technol. 31 486–499. 10.14742/ajet.1437 [ CrossRef ] [ Google Scholar ]
• Ciampaglia G. L. (2018). “ The digital misinformation pipeline ,” in Positive Learning in the Age of Information , eds Zlatkin-Troitschanskaia O., Wittum G., Dengel A. (Wiesbaden: Springer; ), 413–421. 10.1007/978-3-658-19567-0_25 [ CrossRef ] [ Google Scholar ]
• Davey T., Ferrara S., Shavelson R., Holland P., Webb N., Wise L. (2015). Psychometric Considerations for the Next Generation of Performance Assessment. Washington, DC: Center for K-12 Assessment & Performance Management, Educational Testing Service. [ Google Scholar ]
• Facione P. A. (1990). Critical Thinking: A Statement of Expert Consensus for Purposes of Educational Assessment and Instruction—The Delphi Report. Millbrae, CA: Academic Press. [ Google Scholar ]
• Facione P. A. (2004). Critical Thinking: What It Is and Why It Counts. Milbrae, CA: Academic Press. [ Google Scholar ]
• Facione P. A., Facione N. C., Giancarlo C. A. F. (1998). The California Critical Thinking Disposition Inventory (CA). Cambridge, MA: Academic Press. [ Google Scholar ]
• Festinger L. (1962). Cognitive dissonance. Sci. Ame. 207 93–107. 10.1038/scientificamerican1062-93 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Hahnel C., Kroehne U., Goldhammer F., Schoor C., Mahlow N., Artelt C. (2019). Validating process variables of sourcing in an assessment of multiple document comprehension. Br. J. Educ. Psychol. 89 524–537. 10.1111/bjep.12278 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Harrison N., Luckett K. (2019). Experts, knowledge and criticality in the age of ‘alternative facts’: re-examining the contribution of higher education. Teach. Higher Educ. 24 259–271. 10.1080/13562517.2019.1578577 [ CrossRef ] [ Google Scholar ]
• Ho S., Bossard D., Scheufele D. A. (2008). Effects of value predispositions, mass media and knowledge on public attitudes toward embryonic stem cell research. Int. J. Public Opin. Res. 20 171–192. 10.1093/ijpor/edn017 [ CrossRef ] [ Google Scholar ]
• Horstmann N., Ahlgrimm A., Glockner A. (2009). How distinct are intuition and deliberation? An eye tracking analysis of instruction-induced decision modes. Judgm. Decis. Mak. 4 335–354. [ Google Scholar ]
• Hsu C.-H., Tsai M.-J., Hou H.-T., Tsai C.-C. (2014). Epistemic beliefs, online search strategies, and behavioral patterns while exploring socioscientific issues. J. Sci. Educ. Technol. 23 471–480. 10.1007/s10956-013-9477-1 [ CrossRef ] [ Google Scholar ]
• Huber C. R., Kuncel N. R. (2016). Does college teach critical thinking? A meta-analysis. Rev. Educ. Res. 86 431–468. 10.3102/0034654315605917 [ CrossRef ] [ Google Scholar ]
• Johnson F., Sbaffi L., Rowley J. (2016). Students’ approaches to the evaluation of digital information: insights from their trust judgments. Br. J. Educ. Technol. 47 1243–1258. 10.1111/bjet.12306 [ CrossRef ] [ Google Scholar ]
• Kahne J., Bowyer B. (2017). Educating for democracy in a partisan age: confronting the challenges of motivated reasoning and misinformation. Am. Educ. Res. J. 54 3–34. 10.3102/0002831216679817 [ CrossRef ] [ Google Scholar ]
• Kammerer Y., Gerjets P. (2012). Effects of search interface and Internet-specific epistemic beliefs on source evaluations during Web search for medical information: an eye-tracking study. Behav. Inform. Technol. 31 83–97. 10.1080/0144929X.2011.599040 [ CrossRef ] [ Google Scholar ]
• Kane M. T. (2013). Validating the interpretations and uses of test scores. J. Educ. Meas. 50 1–73. 10.1111/jedm.12000 [ CrossRef ] [ Google Scholar ]
• Kienhues D., Stadtler M., Bromme R. (2011). Dealing with conflicting or consistent medical information on the web: when expert information breeds laypersons’ doubts about experts. Learn. Instr. 21 193–204. 10.1016/j.learninstruc.2010.02.004 [ CrossRef ] [ Google Scholar ]
• Klein S., Benjamin R., Shavelson R., Bolus R. (2007). The collegiate learning assessment: facts and fantasies. Eval. Rev. 31 415–439. 10.1177/0193841X07303318 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Liepmann D., Beauducel A., Brocke B., Amthauer R. (2007). I-S-T 2000 R. Intelligenz-Struktur-Test 2000 R , 2nd Edn Göttingen: Hogrefe. [ Google Scholar ]
• List A., Alexander P. A. (2017). Cognitive affective engagement model of multiple source use. Educ. Psychol. 52 182–199. 10.1080/00461520.2017.1329014 [ CrossRef ] [ Google Scholar ]
• List A., Alexander P. A. (2018). “ Cold and warm perspectives on the cognitive affective engagement model of multiple source use ,” in Handbook of Multiple Source Use , eds Braasch J. L. G., Bråten I., McCrudden M. T. (New York, NY: Routledge; ), 34–54. 10.4324/9781315627496-3 [ CrossRef ] [ Google Scholar ]
• List A., Alexander P. A. (2019). Toward an integrated framework of multiple text use. Educ. Psychol. 54 20–39. 10.1080/00461520.2018.1505514 [ CrossRef ] [ Google Scholar ]
• Liu O. L., Frankel L., Roohr K. C. (2014). Assessing critical thinking in higher education: current state and directions for next-generation assessments. ETS Res. Rep. 2014 1–23. 10.1002/ets2.12009 [ CrossRef ] [ Google Scholar ]
• Lucassen T., Schraagen J. M. (2011). Factual accuracy and trust in information: the role of expertise. J. Am. Soc. Inform. Sci. Technol. 62 1232–1242. 10.1002/asi.21545 [ CrossRef ] [ Google Scholar ]
• Mason L., Boldrin A., Ariasi N. (2010). Searching the Web to learn about a controversial topic: are students epistemically active? Instr. Sci. 38 607–633. 10.1007/s11251-008-9089-y [ CrossRef ] [ Google Scholar ]
• Mason L., Pluchino P., Ariasi N. (2014). Reading information about a scientific phenomenon on webpages varying for reliability: an eye-movement analysis. Educ. Technolo. Res. Dev. 62 663–685. 10.1007/s11423-014-9356-3 [ CrossRef ] [ Google Scholar ]
• Maurer M., Quiring O., Schemer C. (2018). “ Media effects on positive and negative learning ,” in Positive Learning in the Age of Information (PLATO) – A Blessing or a Curse? , eds Zlatkin-Troitschanskaia O., Wittum G., Dengel A. (Wiesbaden: Springer; ), 197–208. 10.1007/978-3-030-26578-6 [ CrossRef ] [ Google Scholar ]
• Maurer M., Schemer C., Zlatkin-Troitschanskaia O., Jitomirski J. (2020). “ Positive and negative media effects on university students’ learning: preliminary findings and a research program ,” in Frontiers and Advances in Positive Learning in the Age of Information (PLATO) , ed. Zlatkin-Troitschanskaia O. (New York, NY: Springer; ), 109–119. 10.1007/978-3-030-26578-6_8 [ CrossRef ] [ Google Scholar ]
• McGrew S., Breakstone J., Ortega T., Smith M., Wineburg S. (2018). Can students evaluate online sources? Learning from assessments of civic online reasoning. Theory Res. Soc. Educ. 46 165–193. 10.1080/00933104.2017.1416320 [ CrossRef ] [ Google Scholar ]
• McGrew S., Ortega T., Breakstone J., Wineburg S. (2017). The challenge that’s bigger than fake news: civic reasoning in a social media environment. Am. Edu. 41 4–9. [ Google Scholar ]
• Mercier H., Sperber D. (2009). “ Intuitive and reflective inferences ,” in In Two Minds: Dual Processes and Beyond , eds Evans J., Frankish K. (Oxford: Oxford University Press; ), 149–170. 10.1093/acprof:oso/9780199230167.003.0007 [ CrossRef ] [ Google Scholar ]
• Mercier H., Sperber D. (2011). Why do humans reason? Arguments for an argumentative theory. Behav. Brain Sci. 34 57–74. 10.1017/S0140525X100009 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Messick S. (1994). The interplay of evidence and consequences in the validation of performance assessments. Edu. Res. 23 13–23. 10.3102/0013189X023002013 [ CrossRef ] [ Google Scholar ]
• Metzger M. J. (2007). Making sense of credibility on the web: models for evaluating online information and recommendations for future research. J. Am. Soc. Inform. Sci. Technol. 58 2078–2091. 10.1002/asi.20672 [ CrossRef ] [ Google Scholar ]
• Metzger M. J., Flanagin A. J. (2015). Credibility and trust of information in online environments: the use of cognitive heuristics. J. Prag. 59 210–220. 10.1016/j.pragma.2013.07.012 [ CrossRef ] [ Google Scholar ]
• Metzger M. J., Flanagin A. J., Medders R. B. (2010). Social and heuristic approaches to credibility evaluation online. J. Commun. 60 413–439. 10.1111/j.1460-2466.2010.01488.x [ CrossRef ] [ Google Scholar ]
• Mislevy R. J. (2018). Socio-Cognitive Foundations of Educational Measurement. New York, NY: Routledge. [ Google Scholar ]
• Moore T. (2013). Critical thinking: seven definitions in search of a concept. Stud. Higher Educ. 38 506–522. 10.1080/03075079.2011.586995 [ CrossRef ] [ Google Scholar ]
• Münchow H., Richter T., Schmid S. (2020). “ What does it take to deal with academic literature? ,” in Student Learning in German Higher Education: Innovative Measurement Approaches and Research Results, Vol. 2 , eds Zlatkin-Troitschanskaia O., Pant H. A., Toepper M., Lautenbach C. (Wiesbaden: Springer; ), 241–260. 10.1007/978-3-658-27886-1_12 [ CrossRef ] [ Google Scholar ]
• Münchow H., Richter T., von der Mühlen S., Schmid S. (2019). The ability to evaluate arguments in scientific texts: measurement, cognitive processes, nomological network, and relevance for academic success at the university. Br. J. Educ. Psychol. 89 501–523. 10.1111/bjep.12298 [ PMC free article ] [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Murray M. C., Pérez J. (2014). Unraveling the digital literacy paradox: how higher education fails at the fourth literacy. Issues in Inform. Sci. Inform. Technol. 11 85–100. 10.28945/1982 [ CrossRef ] [ Google Scholar ]
• Nagel M.-T., Zlatkin-Troitschanskaia O., Schmidt S., Beck K. (2020). “ Performance assessment of generic and domain-specific skills in higher education economics ,” in Student Learning in German Higher Education: Innovative Measurement Approaches and Research Results , Zlatkin-Troitschanskaia O., Pant H. A., Toepper M., Lautenbach C. (Wiesbaden: Springer; ), 281–299. 10.1007/978-3-658-27886-1_14 [ CrossRef ] [ Google Scholar ]
• Newman T., Beetham H. (2017). Student Digital Experience Tracker 2017: the Voice of 22,000 UK Learners. Bristol: JISC. [ Google Scholar ]
• Oliveri M. E., Mislevy R. J. (2019). Introduction to “Challenges and Opportunities in the Design of ‘Next-Generation Assessments of 21st Century Skills”’ Special Issue. Int. J. Test. 19 97–102. 10.1080/15305058.2019.1608551 [ CrossRef ] [ Google Scholar ]
• Oser F., Biedermann H. (2020). “ A three-level model for critical thinking: critical alertness, critical reflection, and critical analysis ,” in Frontiers and Advances in Positive Learning in the Age of Information (PLATO) , ed. Zlatkin-Troitschanskaia O. (New York, NY: Springer; ), 89–106. 10.1007/978-3-030-26578-6_7 [ CrossRef ] [ Google Scholar ]
• Pellegrino J. (2017). “ Teaching, learning and assessing 21st century skills ,” in Pedagogical Knowledge and the Changing Nature of the Teaching Profession , ed. Guerriero S. (Paris: OECD Publishing; ), 10.1787/9789264270695-12-en [ CrossRef ] [ Google Scholar ]
• Pellegrino J. W. (2020). Sciences of learning and development: some thoughts from the learning sciences. Appl. Dev. Sci. 24 48–56. 10.1080/10888691.2017.1421427 [ CrossRef ] [ Google Scholar ]
• Schoor C., Melzner N., Artelt C. (2019). The effect of the wording of multiple documents on learning. Zeitschrift für Pädagogische Psychologie 33 223–240. 10.1024/1010-0652/a000246 [ CrossRef ] [ Google Scholar ]
• Shavelson R. J., Zlatkin-Troitschanskaia O., Beck K., Schmidt S., Marino J. (2019). Assessment of university students’ critical thinking: next generation performance assessment. Int. J. Test. 19 337–362. 10.1080/15305058.2018.1543309 [ CrossRef ] [ Google Scholar ]
• Stanovich K. E. (2003). “ The fundamental computational biases of human cognition: heuristics that (sometimes) impair decision making and problem solving ,” in The Psychology of Problem Solving , eds Davidson J. E., Sternberg R. J. (New York, NY: Cambridge University Press; ), 291–342. 10.1017/cbo9780511615771.011 [ CrossRef ] [ Google Scholar ]
• Stanovich K. E. (2016). The Rationality Quotient: Toward a Test of Rational Thinking , 1st Edn Cambridge, MA: MIT Press. [ Google Scholar ]
• Strauss A., Corbin J. (1990). Basic of Grounded Theory Methods. Beverly Hills, CA: Sage. [ Google Scholar ]
• Strauss A., Corbin J. (1998). Basics of Qualitative Research. Techniques and Procedures for Developing Grounded Theory , 2nd Edn Thousand Oaks, CA: Sage Publications. [ Google Scholar ]
• Sweller J. (1988). Cognitive load during problem solving: effects on learning. Cogn. Sci. 12 257–285. 10.1207/s15516709cog1202_4 [ CrossRef ] [ Google Scholar ]
• Ulyshen T. Z., Koehler M. J., Gao F. (2015). Understanding the connection between epistemic beliefs and internet searching. J. Educ. Comput. Res. 53 345–383. 10.1177/0735633115599604 [ CrossRef ] [ Google Scholar ]
• van Strien J. L. H., Brand-Gruvel S., Boshuizen H. P. A. (2014). Dealing with conflicting information from multiple nonlinear texts: effects of prior attitudes. Comput. Hum. Behav. 32 101–111. 10.1016/j.chb.2013.11.021 [ CrossRef ] [ Google Scholar ]
• van Strien J. L. H., Bijker M., Brand-Gruwel S., Boshuizen H. P. A. (2012a). “ Measuring sophistication of epistemic beliefs using rasch analysis ,” in The Future of Learning: Proceedings of the 10th International Conference of the Learning Sciences (ICLS 2012) Volume 2. Short Papers, Symposia, and Abstracts , eds van Aalst J., Thompson K., Jacobson M. J., Reimann P. (Sydney: International Society of the Learning Sciences; ), 197–201. [ Google Scholar ]
• van Strien J. L. H., Brand-Gruwel S., Boshuizen H. P. A. (2012b). Do Prior Attitudes Influence Epistemic Cognition While Reading Conflicting Information? Poster Presented at the Biannual Meeting of the EARLI Special Interest Group Comprehension of Text and Graphics in August 2016. Grenoble (France). Avaliable at: https://www.researchgate.net/publication/254848942_Do_prior_attitudes_influence_epistemic_cognition_while_reading_conflicting_information (accessed May 16, 2020). [ Google Scholar ]
• van Strien J. L. H., Kammerer Y., Brand-Gruvel S., Boshuizen H. P. A. (2016). How attitude strength biases information processing and evaluation on the web. Comput. Hum. Behav. 60 245–252. 10.1016/j.chb.2016.02.057 [ CrossRef ] [ Google Scholar ]
• Walthen C. N., Burkell J. (2002). Believe it or not: factors influencing credibility on the web. J. Am. Soc. Infor. Sci. Technol. 53 134–144. 10.1002/asi.10016 [ CrossRef ] [ Google Scholar ]
• Watson G., Glaser E. (2002). Watson-Glaser Critical Thinking Appraisal – UK Edition. London: Pearson Assessment. [ Google Scholar ]
• Wineburg S., McGrew S. (2017). “ Lateral reading: Reading less and learning more when evaluating digital information ,” in Stanford History Education Group Working Paper No. 2017-A1. Avaliable at: https://ssrn.com/ abstract = 3048994 (accessed May 16, 2020). [ Google Scholar ]
• Wineburg S., Smith M., Breakstone J. (2018). What is learned in college history classes? J. Am. History 104 983–993. 10.1093/jahist/jax434 [ CrossRef ] [ Google Scholar ]
• Wolf R., Zahner D., Benjamin R. (2015). Methodological challenges in international comparative post-secondary assessment programs: lessons learned and the road ahead. Stud. Higher Educ. 40 471–481. 10.1080/03075079.2015.1004239 [ CrossRef ] [ Google Scholar ]
• Zahner D. (2013). Reliability and Validity–CLA+. New York, NY: CAE. [ Google Scholar ]
• Zahner D., Ciolfi A. (2018). “ International comparison of a performance-based assessment in higher education ,” in Assessment of Learning Outcomes in Higher Education. Methodology of Educational Measurement and Assessment , eds Zlatkin-Troitschanskaia O., Toepper M., Pant H., Lautenbach C., Kuhn C. (Wiebaden: Springer; ), 10.1007/978-3-319-74338-7 [ CrossRef ] [ Google Scholar ]
• Zlatkin-Troitschanskaia O., Jitomirski J., Happ R., Molerov D., Schlax J., Kühling-Thees C., et al. (2019a). Validating a test for measuring knowledge and understanding of economics among university students. Zeitschrift für Pädagogische Psychologie , 33 119–133. [ Google Scholar ]
• Zlatkin-Troitschanskaia O., Shavelson R. J., Schmidt S., Beck K. (2019b). On the complementarity of holistic and analytic approaches to performance assessment scoring. Br. J. Educ. Psychol. 89 468–484. 10.1111/bjep.12286 [ PubMed ] [ CrossRef ] [ Google Scholar ]
• Zlatkin-Troitschanskaia O., Shavelson R. J., Pant H. A. (2018a). “ Assessment of Learning outcomes in higher education: international comparisons and perspectives ,” in Handbook on Measurement, Assessment and Evaluation in Higher Education , Secolsky C., Denison B. (New York, NY: Routledge; ). [ Google Scholar ]
• Zlatkin-Troitschanskaia O., Toepper M., Molerov D., Buske R., Brückner S., Pant H. A. (2018b). “ Adapting and validating the collegiate learning assessment to measure generic academic skills of students in germany – implications for international assessment studies in higher education ,” in Assessment of Learning Outcomes in Higher Education , eds Zlatkin-Troitschanskaia O., Pant H. A., Toepper M., Lautenbach C., Kuhn C. (Wiesbaden: Springer; ), 245–266. [ Google Scholar ]

Academia.edu no longer supports Internet Explorer.

To browse Academia.edu and the wider internet faster and more securely, please take a few seconds to  upgrade your browser .

Enter the email address you signed up with and we'll email you a reset link.

• We're Hiring!
• Help Center

What are the roles of beliefs and claims in critical thinking

Related Papers

Charlene Tan

This chapter introduces key concepts in critical thinking using films and music videos. It focuses on the critical thinking skills needed for the identification, analysis and evaluation of arguments. Based on 12 key questions, readers are introduced to core features of an argument such as “premise”, “conclusion” and “assumption”. The main types of arguments and the criteria for evaluating these arguments are also discussed. Throughout the chapter, films such as A Beautiful Mind, Bowling for Columbine and CSI: Miami, and music videos of John Lennon’s “Imagine”, Britney Spears’ “Toxic”, Michael Jackson’s “Billie Jean” and others are used to illustrate the concepts.

Octavian Repolschi

The paper will present the relation between students' beliefs and their behaviours observed in the process of learning critical thinking skills. In the first place some consideration concerning the fundamental epistemological concepts used in the research and about the particular critical thinking skills are to be sketched. Then the testing-learning procedure will be shortly summarized. Thirdly the evaluation of beliefs, their relations with knowledge and the associated behaviors are presented. The results of the periodic testing procedures that were taking place according to the established methodology are to be discussed. Finally, some general considerations concerning the relations between beliefs, behaviors and knowledge that have emerged in the process of learning are going to be presented.

Tracy Bowell

There are some beliefs that are difficult to think critically about, even for those who have critical thinking skills and are committed to applying them to their own beliefs. These resistant beliefs are not all of a kind, and so a range of different strategies may be needed to get ourselves and others (in particular our students) to think critically about them. In this paper we suggest some such strategies.

Critical thinking deserves both imaginative teaching and serious theoretical attention. Studies in Critical Thinking assembles an all-star cast to serve both.EDITOR: J. Anthony Blair (Windsor) INTRO: On What Critical Thinking Is (Alec Fisher, East Anglia) PART II On Teaching CT (Blair & Scriven) 5 Exercises: Validity (Derek Allen, Toronto), Teaching Argument Construction (Kingsbury, Waikato), C.T About Students’ Own Beliefs (Tracy Bowell, Waikato & Justine Kingsbury), Settling Conflict by Compromise (Jan Albert van Laar, Groningen), Using Arguments to Inquire (Sharon Bailin, Simon Fraser & Mark Battersby, Capilano) PART III 7 Chapters on Argument: Arguments and CT (J. Anthony Blair), The Concept of an Argument (David Hitchcock, McMaster), Using Computer Aided Argument Mapping to Teach CT (Martin Davies, Ashley Barnett, Tim van Gelder, Melbourne), Argument Schemes and Argument Mining (Douglas Walton, Windsor), Constructing Effective Arguments (Beth Innocenti, Kansas), Judging Argum...

Inquiry: Critical Thinking Across the Disciplines

Amanda Hiner

Critical thinking, metacognition, and epistemological beliefs

Educational Theory

Sophie Haroutunian-Gordon

Mark Sainsbury

I Gusti Ayu Gde sosiowati

Richards (2006) states that the purpose of learning language is to master the communicative competence, meaning that by the end of the leaming process, the students should be able to produce proper language in any genre and in any situation. However, that competence alone, without accompanied by the ability to perform critical thinking will end in the conversation talking about explicit information only. It can not be denied that understanding the implicit infbrmation r'vill be challenging and making the conversation interesting. Halpern (cited on l5 March 2015) states that critical thinking refers to the use of cognitive skills or strategies that increase the probability of a desirable outcome. It is the kind of thinking which is involved in solving problems- formulating inferences, calculating likelihoods, and making decisions.The purpose of this article is to show that literary work can be used to develop critical thinking and at the same time is able to improve the students&...

Aidyn Aldaberdikyzy

RELATED PAPERS

Laura alvear

IRJET Journal

Pediatric Dentistry

Heba Sabbagh

Cardiologia Croatica

Jasna Habek

Human immunology

barbara guedes

Polish Journal of Sports Medicine

Kamila Musiał

BIOEDUSAINS: Jurnal Pendidikan Biologi dan Sains

merti triyanti

Wortfolge: Jahrbuch des Instituts für Germanische Philologie der Schlesischen Universität = Szyk Słów: Rocznik Instytutu Filologii Germańskiej Uniwersytetu Ślas̨kiego

Grażyna Krupińska

Felix F Schäfer , Christin Keller , Frank Duehrkohp , Benjamin Höke

Mette Byriel-Thygesen

Journal of Neuroradiology

René Chapot

Turkish Journal Of Neurology

Sukru Torun

Barbara Seebacher

Ecotrophic: Journal of Environmental Science

AGUNG SRI SULISTYAWATI

Research Square (Research Square)

Covadonga Huidobro

International Journal of Reproduction, Contraception, Obstetrics and Gynecology

Ashish Kale

International Journal of Innovative Science and Research Technology

Undang Rosidin

arXiv (Cornell University)

Gauri Arora

Neurocritical Care

Ashok Panigrahy

Journal of Hepatology

Benedicte Lescrauwaet

Journal of Medicine and Life

Jorge Oscar Errecalde

•   We're Hiring!
•   Help Center
• Find new research papers in:
• Health Sciences
• Earth Sciences
• Cognitive Science
• Mathematics
• Computer Science
• Academia ©2024

• Register or Log In
• 0) { document.location='/search/'+document.getElementById('quicksearch').value.trim().toLowerCase(); }">

Chapter 1 Self-Quiz

Quiz content, are you sure, select your country.

• Table of Contents
• New in this Archive
• Chronological
• Editorial Information
• About the SEP
• Editorial Board
• How to Cite the SEP
• Special Characters
• Support the SEP
• PDFs for SEP Friends
• Make a Donation
• SEPIA for Libraries
• Entry Contents

Bibliography

Academic tools.

• Friends PDF Preview
• Author and Citation Info
• Back to Top

Critical Thinking

Critical thinking is a widely accepted educational goal. Its definition is contested, but the competing definitions can be understood as differing conceptions of the same basic concept: careful thinking directed to a goal. Conceptions differ with respect to the scope of such thinking, the type of goal, the criteria and norms for thinking carefully, and the thinking components on which they focus. Its adoption as an educational goal has been recommended on the basis of respect for students’ autonomy and preparing students for success in life and for democratic citizenship. “Critical thinkers” have the dispositions and abilities that lead them to think critically when appropriate. The abilities can be identified directly; the dispositions indirectly, by considering what factors contribute to or impede exercise of the abilities. Standardized tests have been developed to assess the degree to which a person possesses such dispositions and abilities. Educational intervention has been shown experimentally to improve them, particularly when it includes dialogue, anchored instruction, and mentoring. Controversies have arisen over the generalizability of critical thinking across domains, over alleged bias in critical thinking theories and instruction, and over the relationship of critical thinking to other types of thinking.

2.1 Dewey’s Three Main Examples

2.2 dewey’s other examples, 2.3 further examples, 2.4 non-examples, 3. the definition of critical thinking, 4. its value, 5. the process of thinking critically, 6. components of the process, 7. contributory dispositions and abilities, 8.1 initiating dispositions, 8.2 internal dispositions, 9. critical thinking abilities, 10. required knowledge, 11. educational methods, 12.1 the generalizability of critical thinking, 12.2 bias in critical thinking theory and pedagogy, 12.3 relationship of critical thinking to other types of thinking, other internet resources, related entries.

Use of the term ‘critical thinking’ to describe an educational goal goes back to the American philosopher John Dewey (1910), who more commonly called it ‘reflective thinking’. He defined it as

active, persistent and careful consideration of any belief or supposed form of knowledge in the light of the grounds that support it, and the further conclusions to which it tends. (Dewey 1910: 6; 1933: 9)

and identified a habit of such consideration with a scientific attitude of mind. His lengthy quotations of Francis Bacon, John Locke, and John Stuart Mill indicate that he was not the first person to propose development of a scientific attitude of mind as an educational goal.

In the 1930s, many of the schools that participated in the Eight-Year Study of the Progressive Education Association (Aikin 1942) adopted critical thinking as an educational goal, for whose achievement the study’s Evaluation Staff developed tests (Smith, Tyler, & Evaluation Staff 1942). Glaser (1941) showed experimentally that it was possible to improve the critical thinking of high school students. Bloom’s influential taxonomy of cognitive educational objectives (Bloom et al. 1956) incorporated critical thinking abilities. Ennis (1962) proposed 12 aspects of critical thinking as a basis for research on the teaching and evaluation of critical thinking ability.

Since 1980, an annual international conference in California on critical thinking and educational reform has attracted tens of thousands of educators from all levels of education and from many parts of the world. Also since 1980, the state university system in California has required all undergraduate students to take a critical thinking course. Since 1983, the Association for Informal Logic and Critical Thinking has sponsored sessions in conjunction with the divisional meetings of the American Philosophical Association (APA). In 1987, the APA’s Committee on Pre-College Philosophy commissioned a consensus statement on critical thinking for purposes of educational assessment and instruction (Facione 1990a). Researchers have developed standardized tests of critical thinking abilities and dispositions; for details, see the Supplement on Assessment . Educational jurisdictions around the world now include critical thinking in guidelines for curriculum and assessment. Political and business leaders endorse its importance.

For details on this history, see the Supplement on History .

2. Examples and Non-Examples

Before considering the definition of critical thinking, it will be helpful to have in mind some examples of critical thinking, as well as some examples of kinds of thinking that would apparently not count as critical thinking.

Dewey (1910: 68–71; 1933: 91–94) takes as paradigms of reflective thinking three class papers of students in which they describe their thinking. The examples range from the everyday to the scientific.

Transit : “The other day, when I was down town on 16th Street, a clock caught my eye. I saw that the hands pointed to 12:20. This suggested that I had an engagement at 124th Street, at one o'clock. I reasoned that as it had taken me an hour to come down on a surface car, I should probably be twenty minutes late if I returned the same way. I might save twenty minutes by a subway express. But was there a station near? If not, I might lose more than twenty minutes in looking for one. Then I thought of the elevated, and I saw there was such a line within two blocks. But where was the station? If it were several blocks above or below the street I was on, I should lose time instead of gaining it. My mind went back to the subway express as quicker than the elevated; furthermore, I remembered that it went nearer than the elevated to the part of 124th Street I wished to reach, so that time would be saved at the end of the journey. I concluded in favor of the subway, and reached my destination by one o’clock.” (Dewey 1910: 68-69; 1933: 91-92)

Ferryboat : “Projecting nearly horizontally from the upper deck of the ferryboat on which I daily cross the river is a long white pole, having a gilded ball at its tip. It suggested a flagpole when I first saw it; its color, shape, and gilded ball agreed with this idea, and these reasons seemed to justify me in this belief. But soon difficulties presented themselves. The pole was nearly horizontal, an unusual position for a flagpole; in the next place, there was no pulley, ring, or cord by which to attach a flag; finally, there were elsewhere on the boat two vertical staffs from which flags were occasionally flown. It seemed probable that the pole was not there for flag-flying.

“I then tried to imagine all possible purposes of the pole, and to consider for which of these it was best suited: (a) Possibly it was an ornament. But as all the ferryboats and even the tugboats carried poles, this hypothesis was rejected. (b) Possibly it was the terminal of a wireless telegraph. But the same considerations made this improbable. Besides, the more natural place for such a terminal would be the highest part of the boat, on top of the pilot house. (c) Its purpose might be to point out the direction in which the boat is moving.

“In support of this conclusion, I discovered that the pole was lower than the pilot house, so that the steersman could easily see it. Moreover, the tip was enough higher than the base, so that, from the pilot's position, it must appear to project far out in front of the boat. Morevoer, the pilot being near the front of the boat, he would need some such guide as to its direction. Tugboats would also need poles for such a purpose. This hypothesis was so much more probable than the others that I accepted it. I formed the conclusion that the pole was set up for the purpose of showing the pilot the direction in which the boat pointed, to enable him to steer correctly.” (Dewey 1910: 69-70; 1933: 92-93)

Bubbles : “In washing tumblers in hot soapsuds and placing them mouth downward on a plate, bubbles appeared on the outside of the mouth of the tumblers and then went inside. Why? The presence of bubbles suggests air, which I note must come from inside the tumbler. I see that the soapy water on the plate prevents escape of the air save as it may be caught in bubbles. But why should air leave the tumbler? There was no substance entering to force it out. It must have expanded. It expands by increase of heat, or by decrease of pressure, or both. Could the air have become heated after the tumbler was taken from the hot suds? Clearly not the air that was already entangled in the water. If heated air was the cause, cold air must have entered in transferring the tumblers from the suds to the plate. I test to see if this supposition is true by taking several more tumblers out. Some I shake so as to make sure of entrapping cold air in them. Some I take out holding mouth downward in order to prevent cold air from entering. Bubbles appear on the outside of every one of the former and on none of the latter. I must be right in my inference. Air from the outside must have been expanded by the heat of the tumbler, which explains the appearance of the bubbles on the outside. But why do they then go inside? Cold contracts. The tumbler cooled and also the air inside it. Tension was removed, and hence bubbles appeared inside. To be sure of this, I test by placing a cup of ice on the tumbler while the bubbles are still forming outside. They soon reverse” (Dewey 1910: 70–71; 1933: 93–94).

Dewey (1910, 1933) sprinkles his book with other examples of critical thinking. We will refer to the following.

Weather : A man on a walk notices that it has suddenly become cool, thinks that it is probably going to rain, looks up and sees a dark cloud obscuring the sun, and quickens his steps (1910: 6–10; 1933: 9–13).

Disorder : A man finds his rooms on his return to them in disorder with his belongings thrown about, thinks at first of burglary as an explanation, then thinks of mischievous children as being an alternative explanation, then looks to see whether valuables are missing, and discovers that they are (1910: 82–83; 1933: 166–168).

Typhoid : A physician diagnosing a patient whose conspicuous symptoms suggest typhoid avoids drawing a conclusion until more data are gathered by questioning the patient and by making tests (1910: 85–86; 1933: 170).

Blur : A moving blur catches our eye in the distance, we ask ourselves whether it is a cloud of whirling dust or a tree moving its branches or a man signaling to us, we think of other traits that should be found on each of those possibilities, and we look and see if those traits are found (1910: 102, 108; 1933: 121, 133).

Suction pump : In thinking about the suction pump, the scientist first notes that it will draw water only to a maximum height of 33 feet at sea level and to a lesser maximum height at higher elevations, selects for attention the differing atmospheric pressure at these elevations, sets up experiments in which the air is removed from a vessel containing water (when suction no longer works) and in which the weight of air at various levels is calculated, compares the results of reasoning about the height to which a given weight of air will allow a suction pump to raise water with the observed maximum height at different elevations, and finally assimilates the suction pump to such apparently different phenomena as the siphon and the rising of a balloon (1910: 150–153; 1933: 195–198).

Diamond : A passenger in a car driving in a diamond lane reserved for vehicles with at least one passenger notices that the diamond marks on the pavement are far apart in some places and close together in others. Why? The driver suggests that the reason may be that the diamond marks are not needed where there is a solid double line separating the diamond line from the adjoining lane, but are needed when there is a dotted single line permitting crossing into the diamond lane. Further observation confirms that the diamonds are close together when a dotted line separates the diamond lane from its neighbour, but otherwise far apart.

Rash : A woman suddenly develops a very itchy red rash on her throat and upper chest. She recently noticed a mark on the back of her right hand, but was not sure whether the mark was a rash or a scrape. She lies down in bed and thinks about what might be causing the rash and what to do about it. About two weeks before, she began taking blood pressure medication that contained a sulfa drug, and the pharmacist had warned her, in view of a previous allergic reaction to a medication containing a sulfa drug, to be on the alert for an allergic reaction; however, she had been taking the medication for two weeks with no such effect. The day before, she began using a new cream on her neck and upper chest; against the new cream as the cause was mark on the back of her hand, which had not been exposed to the cream. She began taking probiotics about a month before. She also recently started new eye drops, but she supposed that manufacturers of eye drops would be careful not to include allergy-causing components in the medication. The rash might be a heat rash, since she recently was sweating profusely from her upper body. Since she is about to go away on a short vacation, where she would not have access to her usual physician, she decides to keep taking the probiotics and using the new eye drops but to discontinue the blood pressure medication and to switch back to the old cream for her neck and upper chest. She forms a plan to consult her regular physician on her return about the blood pressure medication.

Candidate : Although Dewey included no examples of thinking directed at appraising the arguments of others, such thinking has come to be considered a kind of critical thinking. We find an example of such thinking in the performance task on the Collegiate Learning Assessment (CLA+), which its sponsoring organization describes as

a performance-based assessment that provides a measure of an institution’s contribution to the development of critical-thinking and written communication skills of its students. (Council for Aid to Education 2017)

A sample task posted on its website requires the test-taker to write a report for public distribution evaluating a fictional candidate’s policy proposals and their supporting arguments, using supplied background documents, with a recommendation on whether to endorse the candidate.

Immediate acceptance of an idea that suggests itself as a solution to a problem (e.g., a possible explanation of an event or phenomenon, an action that seems likely to produce a desired result) is “uncritical thinking, the minimum of reflection” (Dewey 1910: 13). On-going suspension of judgment in the light of doubt about a possible solution is not critical thinking (Dewey 1910: 108). Critique driven by a dogmatically held political or religious ideology is not critical thinking; thus Paulo Freire (1968 [1970]) is using the term (e.g., at 1970: 71, 81, 100, 146) in a more politically freighted sense that includes not only reflection but also revolutionary action against oppression. Derivation of a conclusion from given data using an algorithm is not critical thinking.

What is critical thinking? There are many definitions. Ennis (2016) lists 14 philosophically oriented scholarly definitions and three dictionary definitions. Following Rawls (1971), who distinguished his conception of justice from a utilitarian conception but regarded them as rival conceptions of the same concept, Ennis maintains that the 17 definitions are different conceptions of the same concept. Rawls articulated the shared concept of justice as

a characteristic set of principles for assigning basic rights and duties and for determining… the proper distribution of the benefits and burdens of social cooperation. (Rawls 1971: 5)

Bailin et al. (1999b) claim that, if one considers what sorts of thinking an educator would take not to be critical thinking and what sorts to be critical thinking, one can conclude that educators typically understand critical thinking to have at least three features.

• It is done for the purpose of making up one’s mind about what to believe or do.
• The person engaging in the thinking is trying to fulfill standards of adequacy and accuracy appropriate to the thinking.
• The thinking fulfills the relevant standards to some threshold level.

One could sum up the core concept that involves these three features by saying that critical thinking is careful goal-directed thinking. This core concept seems to apply to all the examples of critical thinking described in the previous section. As for the non-examples, their exclusion depends on construing careful thinking as excluding jumping immediately to conclusions, suspending judgment no matter how strong the evidence, reasoning from an unquestioned ideological or religious perspective, and routinely using an algorithm to answer a question.

If the core of critical thinking is careful goal-directed thinking, conceptions of it can vary according to its presumed scope, its presumed goal, one’s criteria and threshold for being careful, and the thinking component on which one focuses As to its scope, some conceptions (e.g., Dewey 1910, 1933) restrict it to constructive thinking on the basis of one’s own observations and experiments, others (e.g., Ennis 1962; Fisher & Scriven 1997; Johnson 1992) to appraisal of the products of such thinking. Ennis (1991) and Bailin et al. (1999b) take it to cover both construction and appraisal. As to its goal, some conceptions restrict it to forming a judgment (Dewey 1910, 1933; Lipman 1987; Facione 1990a). Others allow for actions as well as beliefs as the end point of a process of critical thinking (Ennis 1991; Bailin et al. 1999b). As to the criteria and threshold for being careful, definitions vary in the term used to indicate that critical thinking satisfies certain norms: “intellectually disciplined” (Scriven & Paul 1987), “reasonable” (Ennis 1991), “skillful” (Lipman 1987), “skilled” (Fisher & Scriven 1997), “careful” (Bailin & Battersby 2009). Some definitions specify these norms, referring variously to “consideration of any belief or supposed form of knowledge in the light of the grounds that support it and the further conclusions to which it tends” (Dewey 1910, 1933); “the methods of logical inquiry and reasoning” (Glaser 1941); “conceptualizing, applying, analyzing, synthesizing, and/or evaluating information gathered from, or generated by, observation, experience, reflection, reasoning, or communication” (Scriven & Paul 1987); the requirement that “it is sensitive to context, relies on criteria, and is self-correcting” (Lipman 1987); “evidential, conceptual, methodological, criteriological, or contextual considerations” (Facione 1990a); and “plus-minus considerations of the product in terms of appropriate standards (or criteria)” (Johnson 1992). Stanovich and Stanovich (2010) propose to ground the concept of critical thinking in the concept of rationality, which they understand as combining epistemic rationality (fitting one’s beliefs to the world) and instrumental rationality (optimizing goal fulfillment); a critical thinker, in their view, is someone with “a propensity to override suboptimal responses from the autonomous mind” (2010: 227). These variant specifications of norms for critical thinking are not necessarily incompatible with one another, and in any case presuppose the core notion of thinking carefully. As to the thinking component singled out, some definitions focus on suspension of judgment during the thinking (Dewey 1910; McPeck 1981), others on inquiry while judgment is suspended (Bailin & Battersby 2009), others on the resulting judgment (Facione 1990a), and still others on the subsequent emotive response (Siegel 1988).

In educational contexts, a definition of critical thinking is a “programmatic definition” (Scheffler 1960: 19). It expresses a practical program for achieving an educational goal. For this purpose, a one-sentence formulaic definition is much less useful than articulation of a critical thinking process, with criteria and standards for the kinds of thinking that the process may involve. The real educational goal is recognition, adoption and implementation by students of those criteria and standards. That adoption and implementation in turn consists in acquiring the knowledge, abilities and dispositions of a critical thinker.

Conceptions of critical thinking generally do not include moral integrity as part of the concept. Dewey, for example, took critical thinking to be the ultimate intellectual goal of education, but distinguished it from the development of social cooperation among school children, which he took to be the central moral goal. Ennis (1996, 2011) added to his previous list of critical thinking dispositions a group of dispositions to care about the dignity and worth of every person, which he described as a “correlative” (1996) disposition without which critical thinking would be less valuable and perhaps harmful. An educational program that aimed at developing critical thinking but not the correlative disposition to care about the dignity and worth of every person, he asserted, “would be deficient and perhaps dangerous” (Ennis 1996: 172).

Dewey thought that education for reflective thinking would be of value to both the individual and society; recognition in educational practice of the kinship to the scientific attitude of children’s native curiosity, fertile imagination and love of experimental inquiry “would make for individual happiness and the reduction of social waste” (Dewey 1910: iii). Schools participating in the Eight-Year Study took development of the habit of reflective thinking and skill in solving problems as a means to leading young people to understand, appreciate and live the democratic way of life characteristic of the United States (Aikin 1942: 17–18, 81). Harvey Siegel (1988: 55–61) has offered four considerations in support of adopting critical thinking as an educational ideal. (1) Respect for persons requires that schools and teachers honour students’ demands for reasons and explanations, deal with students honestly, and recognize the need to confront students’ independent judgment; these requirements concern the manner in which teachers treat students. (2) Education has the task of preparing children to be successful adults, a task that requires development of their self-sufficiency. (3) Education should initiate children into the rational traditions in such fields as history, science and mathematics. (4) Education should prepare children to become democratic citizens, which requires reasoned procedures and critical talents and attitudes. To supplement these considerations, Siegel (1988: 62–90) responds to two objections: the ideology objection that adoption of any educational ideal requires a prior ideological commitment and the indoctrination objection that cultivation of critical thinking cannot escape being a form of indoctrination.

Despite the diversity of our 11 examples, one can recognize a common pattern. Dewey analyzed it as consisting of five phases:

• suggestions , in which the mind leaps forward to a possible solution;
• an intellectualization of the difficulty or perplexity into a problem to be solved, a question for which the answer must be sought;
• the use of one suggestion after another as a leading idea, or hypothesis , to initiate and guide observation and other operations in collection of factual material;
• the mental elaboration of the idea or supposition as an idea or supposition ( reasoning , in the sense on which reasoning is a part, not the whole, of inference); and
• testing the hypothesis by overt or imaginative action. (Dewey 1933: 106–107; italics in original)

The process of reflective thinking consisting of these phases would be preceded by a perplexed, troubled or confused situation and followed by a cleared-up, unified, resolved situation (Dewey 1933: 106). The term ‘phases’ replaced the term ‘steps’ (Dewey 1910: 72), thus removing the earlier suggestion of an invariant sequence. Variants of the above analysis appeared in (Dewey 1916: 177) and (Dewey 1938: 101–119).

The variant formulations indicate the difficulty of giving a single logical analysis of such a varied process. The process of critical thinking may have a spiral pattern, with the problem being redefined in the light of obstacles to solving it as originally formulated. For example, the person in Transit might have concluded that getting to the appointment at the scheduled time was impossible and have reformulated the problem as that of rescheduling the appointment for a mutually convenient time. Further, defining a problem does not always follow after or lead immediately to an idea of a suggested solution. Nor should it do so, as Dewey himself recognized in describing the physician in Typhoid as avoiding any strong preference for this or that conclusion before getting further information (Dewey 1910: 85; 1933: 170). People with a hypothesis in mind, even one to which they have a very weak commitment, have a so-called “confirmation bias” (Nickerson 1998): they are likely to pay attention to evidence that confirms the hypothesis and to ignore evidence that counts against it or for some competing hypothesis. Detectives, intelligence agencies, and investigators of airplane accidents are well advised to gather relevant evidence systematically and to postpone even tentative adoption of an explanatory hypothesis until the collected evidence rules out with the appropriate degree of certainty all but one explanation. Dewey’s analysis of the critical thinking process can be faulted as well for requiring acceptance or rejection of a possible solution to a defined problem, with no allowance for deciding in the light of the available evidence to suspend judgment. Further, given the great variety of kinds of problems for which reflection is appropriate, there is likely to be variation in its component events. Perhaps the best way to conceptualize the critical thinking process is as a checklist whose component events can occur in a variety of orders, selectively, and more than once. These component events might include (1) noticing a difficulty, (2) defining the problem, (3) dividing the problem into manageable sub-problems, (4) formulating a variety of possible solutions to the problem or sub-problem, (5) determining what evidence is relevant to deciding among possible solutions to the problem or sub-problem, (6) devising a plan of systematic observation or experiment that will uncover the relevant evidence, (7) carrying out the plan of systematic observation or experimentation, (8) noting the results of the systematic observation or experiment, (9) gathering relevant testimony and information from others, (10) judging the credibility of testimony and information gathered from others, (11) drawing conclusions from gathered evidence and accepted testimony, and (12) accepting a solution that the evidence adequately supports (cf. Hitchcock 2017: 485).

Checklist conceptions of the process of critical thinking are open to the objection that they are too mechanical and procedural to fit the multi-dimensional and emotionally charged issues for which critical thinking is urgently needed (Paul 1984). For such issues, a more dialectical process is advocated, in which competing relevant world views are identified, their implications explored, and some sort of creative synthesis attempted.

If one considers the critical thinking process illustrated by the 11 examples, one can identify distinct kinds of mental acts and mental states that form part of it. To distinguish, label and briefly characterize these components is a useful preliminary to identifying abilities, skills, dispositions, attitudes, habits and the like that contribute causally to thinking critically. Identifying such abilities and habits is in turn a useful preliminary to setting educational goals. Setting the goals is in its turn a useful preliminary to designing strategies for helping learners to achieve the goals and to designing ways of measuring the extent to which learners have done so. Such measures provide both feedback to learners on their achievement and a basis for experimental research on the effectiveness of various strategies for educating people to think critically. Let us begin, then, by distinguishing the kinds of mental acts and mental events that can occur in a critical thinking process.

• Observing : One notices something in one’s immediate environment (sudden cooling of temperature in Weather , bubbles forming outside a glass and then going inside in Bubbles , a moving blur in the distance in Blur , a rash in Rash ). Or one notes the results of an experiment or systematic observation (valuables missing in Disorder , no suction without air pressure in Suction pump )
• Feeling : One feels puzzled or uncertain about something (how to get to an appointment on time in Transit , why the diamonds vary in frequency in Diamond ). One wants to resolve this perplexity. One feels satisfaction once one has worked out an answer (to take the subway express in Transit , diamonds closer when needed as a warning in Diamond ).
• Wondering : One formulates a question to be addressed (why bubbles form outside a tumbler taken from hot water in Bubbles , how suction pumps work in Suction pump , what caused the rash in Rash ).
• Imagining : One thinks of possible answers (bus or subway or elevated in Transit , flagpole or ornament or wireless communication aid or direction indicator in Ferryboat , allergic reaction or heat rash in Rash ).
• Inferring : One works out what would be the case if a possible answer were assumed (valuables missing if there has been a burglary in Disorder , earlier start to the rash if it is an allergic reaction to a sulfa drug in Rash ). Or one draws a conclusion once sufficient relevant evidence is gathered (take the subway in Transit , burglary in Disorder , discontinue blood pressure medication and new cream in Rash ).
• Knowledge : One uses stored knowledge of the subject-matter to generate possible answers or to infer what would be expected on the assumption of a particular answer (knowledge of a city’s public transit system in Transit , of the requirements for a flagpole in Ferryboat , of Boyle’s law in Bubbles , of allergic reactions in Rash ).
• Experimenting : One designs and carries out an experiment or a systematic observation to find out whether the results deduced from a possible answer will occur (looking at the location of the flagpole in relation to the pilot’s position in Ferryboat , putting an ice cube on top of a tumbler taken from hot water in Bubbles , measuring the height to which a suction pump will draw water at different elevations in Suction pump , noticing the frequency of diamonds when movement to or from a diamond lane is allowed in Diamond ).
• Consulting : One finds a source of information, gets the information from the source, and makes a judgment on whether to accept it. None of our 11 examples include searching for sources of information. In this respect they are unrepresentative, since most people nowadays have almost instant access to information relevant to answering any question, including many of those illustrated by the examples. However, Candidate includes the activities of extracting information from sources and evaluating its credibility.
• Identifying and analyzing arguments : One notices an argument and works out its structure and content as a preliminary to evaluating its strength. This activity is central to Candidate . It is an important part of a critical thinking process in which one surveys arguments for various positions on an issue.
• Judging : One makes a judgment on the basis of accumulated evidence and reasoning, such as the judgment in Ferryboat that the purpose of the pole is to provide direction to the pilot.
• Deciding : One makes a decision on what to do or on what policy to adopt, as in the decision in Transit to take the subway.

By definition, a person who does something voluntarily is both willing and able to do that thing at that time. Both the willingness and the ability contribute causally to the person’s action, in the sense that the voluntary action would not occur if either (or both) of these were lacking. For example, suppose that one is standing with one’s arms at one’s sides and one voluntarily lifts one’s right arm to an extended horizontal position. One would not do so if one were unable to lift one’s arm, if for example one’s right side was paralyzed as the result of a stroke. Nor would one do so if one were unwilling to lift one’s arm, if for example one were participating in a street demonstration at which a white supremacist was urging the crowd to lift their right arm in a Nazi salute and one were unwilling to express support in this way for the racist Nazi ideology. The same analysis applies to a voluntary mental process of thinking critically. It requires both willingness and ability to think critically, including willingness and ability to perform each of the mental acts that compose the process and to coordinate those acts in a sequence that is directed at resolving the initiating perplexity.

Consider willingness first. We can identify causal contributors to willingness to think critically by considering factors that would cause a person who was able to think critically about an issue nevertheless not to do so (Hamby 2014). For each factor, the opposite condition thus contributes causally to willingness to think critically on a particular occasion. For example, people who habitually jump to conclusions without considering alternatives will not think critically about issues that arise, even if they have the required abilities. The contrary condition of willingness to suspend judgment is thus a causal contributor to thinking critically.

Now consider ability. In contrast to the ability to move one’s arm, which can be completely absent because a stroke has left the arm paralyzed, the ability to think critically is a developed ability, whose absence is not a complete absence of ability to think but absence of ability to think well. We can identify the ability to think well directly, in terms of the norms and standards for good thinking. In general, to be able do well the thinking activities that can be components of a critical thinking process, one needs to know the concepts and principles that characterize their good performance, to recognize in particular cases that the concepts and principles apply, and to apply them. The knowledge, recognition and application may be procedural rather than declarative. It may be domain-specific rather than widely applicable, and in either case may need subject-matter knowledge, sometimes of a deep kind.

Reflections of the sort illustrated by the previous two paragraphs have led scholars to identify the knowledge, abilities and dispositions of a “critical thinker”, i.e., someone who thinks critically whenever it is appropriate to do so. We turn now to these three types of causal contributors to thinking critically. We start with dispositions, since arguably these are the most powerful contributors to being a critical thinker, can be fostered at an early stage of a child’s development, and are susceptible to general improvement (Glaser 1941: 175)

8. Critical Thinking Dispositions

Educational researchers use the term ‘dispositions’ broadly for the habits of mind and attitudes that contribute causally to being a critical thinker. Some writers (e.g., Paul & Elder 2006; Hamby 2014; Bailin & Battersby 2016) propose to use the term ‘virtues’ for this dimension of a critical thinker. The virtues in question, although they are virtues of character, concern the person’s ways of thinking rather than the person’s ways of behaving towards others. They are not moral virtues but intellectual virtues, of the sort articulated by Zagzebski (1996) and discussed by Turri, Alfano, and Greco (2017).

On a realistic conception, thinking dispositions or intellectual virtues are real properties of thinkers. They are general tendencies, propensities, or inclinations to think in particular ways in particular circumstances, and can be genuinely explanatory (Siegel 1999). Sceptics argue that there is no evidence for a specific mental basis for the habits of mind that contribute to thinking critically, and that it is pedagogically misleading to posit such a basis (Bailin et al. 1999a). Whatever their status, critical thinking dispositions need motivation for their initial formation in a child—motivation that may be external or internal. As children develop, the force of habit will gradually become important in sustaining the disposition (Nieto & Valenzuela 2012). Mere force of habit, however, is unlikely to sustain critical thinking dispositions. Critical thinkers must value and enjoy using their knowledge and abilities to think things through for themselves. They must be committed to, and lovers of, inquiry.

A person may have a critical thinking disposition with respect to only some kinds of issues. For example, one could be open-minded about scientific issues but not about religious issues. Similarly, one could be confident in one’s ability to reason about the theological implications of the existence of evil in the world but not in one’s ability to reason about the best design for a guided ballistic missile.

Critical thinking dispositions can usefully be divided into initiating dispositions (those that contribute causally to starting to think critically about an issue) and internal dispositions (those that contribute causally to doing a good job of thinking critically once one has started) (Facione 1990a: 25). The two categories are not mutually exclusive. For example, open-mindedness, in the sense of willingness to consider alternative points of view to one’s own, is both an initiating and an internal disposition.

Using the strategy of considering factors that would block people with the ability to think critically from doing so, we can identify as initiating dispositions for thinking critically attentiveness, a habit of inquiry, self-confidence, courage, open-mindedness, willingness to suspend judgment, trust in reason, wanting evidence for one’s beliefs, and seeking the truth. We consider briefly what each of these dispositions amounts to, in each case citing sources that acknowledge them.

• Attentiveness : One will not think critically if one fails to recognize an issue that needs to be thought through. For example, the pedestrian in Weather would not have looked up if he had not noticed that the air was suddenly cooler. To be a critical thinker, then, one needs to be habitually attentive to one’s surroundings, noticing not only what one senses but also sources of perplexity in messages received and in one’s own beliefs and attitudes (Facione 1990a: 25; Facione, Facione, & Giancarlo 2001).
• Habit of inquiry : Inquiry is effortful, and one needs an internal push to engage in it. For example, the student in Bubbles could easily have stopped at idle wondering about the cause of the bubbles rather than reasoning to a hypothesis, then designing and executing an experiment to test it. Thus willingness to think critically needs mental energy and initiative. What can supply that energy? Love of inquiry, or perhaps just a habit of inquiry. Hamby (2015) has argued that willingness to inquire is the central critical thinking virtue, one that encompasses all the others. It is recognized as a critical thinking disposition by Dewey (1910: 29; 1933: 35), Glaser (1941: 5), Ennis (1987: 12; 1991: 8), Facione (1990a: 25), Bailin et al. (1999b: 294), Halpern (1998: 452), and Facione, Facione, & Giancarlo (2001).
• Self-confidence : Lack of confidence in one’s abilities can block critical thinking. For example, if the woman in Rash lacked confidence in her ability to figure things out for herself, she might just have assumed that the rash on her chest was the allergic reaction to her medication against which the pharmacist had warned her. Thus willingness to think critically requires confidence in one’s ability to inquire (Facione 1990a: 25; Facione, Facione, & Giancarlo 2001).
• Courage : Fear of thinking for oneself can stop one from doing it. Thus willingness to think critically requires intellectual courage (Paul & Elder 2006: 16).
• Open-mindedness : A dogmatic attitude will impede thinking critically. For example, a person who adheres rigidly to a “pro-choice” position on the issue of the legal status of induced abortion is likely to be unwilling to consider seriously the issue of when in its development an unborn child acquires a moral right to life. Thus willingness to think critically requires open-mindedness, in the sense of a willingness to examine questions to which one already accepts an answer but which further evidence or reasoning might cause one to answer differently (Dewey 1933; Facione 1990a; Ennis 1991; Bailin et al. 1999b; Halpern 1998, Facione, Facione, & Giancarlo 2001). Paul (1981) emphasizes open-mindedness about alternative world-views, and recommends a dialectical approach to integrating such views as central to what he calls “strong sense” critical thinking.
• Willingness to suspend judgment : Premature closure on an initial solution will block critical thinking. Thus willingness to think critically requires a willingness to suspend judgment while alternatives are explored (Facione 1990a; Ennis 1991; Halpern 1998).
• Trust in reason : Since distrust in the processes of reasoned inquiry will dissuade one from engaging in it, trust in them is an initiating critical thinking disposition (Facione 1990a, 25; Bailin et al. 1999b: 294; Facione, Facione, & Giancarlo 2001; Paul & Elder 2006). In reaction to an allegedly exclusive emphasis on reason in critical thinking theory and pedagogy, Thayer-Bacon (2000) argues that intuition, imagination, and emotion have important roles to play in an adequate conception of critical thinking that she calls “constructive thinking”. From her point of view, critical thinking requires trust not only in reason but also in intuition, imagination, and emotion.
• Seeking the truth : If one does not care about the truth but is content to stick with one’s initial bias on an issue, then one will not think critically about it. Seeking the truth is thus an initiating critical thinking disposition (Bailin et al. 1999b: 294; Facione, Facione, & Giancarlo 2001). A disposition to seek the truth is implicit in more specific critical thinking dispositions, such as trying to be well-informed, considering seriously points of view other than one’s own, looking for alternatives, suspending judgment when the evidence is insufficient, and adopting a position when the evidence supporting it is sufficient.

Some of the initiating dispositions, such as open-mindedness and willingness to suspend judgment, are also internal critical thinking dispositions, in the sense of mental habits or attitudes that contribute causally to doing a good job of critical thinking once one starts the process. But there are many other internal critical thinking dispositions. Some of them are parasitic on one’s conception of good thinking. For example, it is constitutive of good thinking about an issue to formulate the issue clearly and to maintain focus on it. For this purpose, one needs not only the corresponding ability but also the corresponding disposition. Ennis (1991: 8) describes it as the disposition “to determine and maintain focus on the conclusion or question”, Facione (1990a: 25) as “clarity in stating the question or concern”. Other internal dispositions are motivators to continue or adjust the critical thinking process, such as willingness to persist in a complex task and willingness to abandon nonproductive strategies in an attempt to self-correct (Halpern 1998: 452). For a list of identified internal critical thinking dispositions, see the Supplement on Internal Critical Thinking Dispositions .

Some theorists postulate skills, i.e., acquired abilities, as operative in critical thinking. It is not obvious, however, that a good mental act is the exercise of a generic acquired skill. Inferring an expected time of arrival, as in Transit , has some generic components but also uses non-generic subject-matter knowledge. Bailin et al. (1999a) argue against viewing critical thinking skills as generic and discrete, on the ground that skilled performance at a critical thinking task cannot be separated from knowledge of concepts and from domain-specific principles of good thinking. Talk of skills, they concede, is unproblematic if it means merely that a person with critical thinking skills is capable of intelligent performance.

Despite such scepticism, theorists of critical thinking have listed as general contributors to critical thinking what they variously call abilities (Glaser 1941; Ennis 1962, 1991), skills (Facione 1990a; Halpern 1998) or competencies (Fisher & Scriven 1997). Amalgamating these lists would produce a confusing and chaotic cornucopia of more than 50 possible educational objectives, with only partial overlap among them. It makes sense instead to try to understand the reasons for the multiplicity and diversity, and to make a selection according to one’s own reasons for singling out abilities to be developed in a critical thinking curriculum. Two reasons for diversity among lists of critical thinking abilities are the underlying conception of critical thinking and the envisaged educational level. Appraisal-only conceptions, for example, involve a different suite of abilities than constructive-only conceptions. Some lists, such as those in (Glaser 1941), are put forward as educational objectives for secondary school students, whereas others are proposed as objectives for college students (e.g., Facione 1990a).

The abilities described in the remaining paragraphs of this section emerge from reflection on the general abilities needed to do well the thinking activities identified in section 6 as components of the critical thinking process described in section 5 . The derivation of each collection of abilities is accompanied by citation of sources that list such abilities and of standardized tests that claim to test them.

Observational abilities : Careful and accurate observation sometimes requires specialist expertise and practice, as in the case of observing birds and observing accident scenes. However, there are general abilities of noticing what one’s senses are picking up from one’s environment and of being able to articulate clearly and accurately to oneself and others what one has observed. It helps in exercising them to be able to recognize and take into account factors that make one’s observation less trustworthy, such as prior framing of the situation, inadequate time, deficient senses, poor observation conditions, and the like. It helps as well to be skilled at taking steps to make one’s observation more trustworthy, such as moving closer to get a better look, measuring something three times and taking the average, and checking what one thinks one is observing with someone else who is in a good position to observe it. It also helps to be skilled at recognizing respects in which one’s report of one’s observation involves inference rather than direct observation, so that one can then consider whether the inference is justified. These abilities come into play as well when one thinks about whether and with what degree of confidence to accept an observation report, for example in the study of history or in a criminal investigation or in assessing news reports. Observational abilities show up in some lists of critical thinking abilities (Ennis 1962: 90; Facione 1990a: 16; Ennis 1991: 9). There are items testing a person’s ability to judge the credibility of observation reports in the Cornell Critical Thinking Tests, Levels X and Z (Ennis & Millman 1971; Ennis, Millman, & Tomko 1985, 2005). Norris and King (1983, 1985, 1990a, 1990b) is a test of ability to appraise observation reports.

Emotional abilities : The emotions that drive a critical thinking process are perplexity or puzzlement, a wish to resolve it, and satisfaction at achieving the desired resolution. Children experience these emotions at an early age, without being trained to do so. Education that takes critical thinking as a goal needs only to channel these emotions and to make sure not to stifle them. Collaborative critical thinking benefits from ability to recognize one’s own and others’ emotional commitments and reactions.

Questioning abilities : A critical thinking process needs transformation of an inchoate sense of perplexity into a clear question. Formulating a question well requires not building in questionable assumptions, not prejudging the issue, and using language that in context is unambiguous and precise enough (Ennis 1962: 97; 1991: 9).

Imaginative abilities : Thinking directed at finding the correct causal explanation of a general phenomenon or particular event requires an ability to imagine possible explanations. Thinking about what policy or plan of action to adopt requires generation of options and consideration of possible consequences of each option. Domain knowledge is required for such creative activity, but a general ability to imagine alternatives is helpful and can be nurtured so as to become easier, quicker, more extensive, and deeper (Dewey 1910: 34–39; 1933: 40–47). Facione (1990a) and Halpern (1998) include the ability to imagine alternatives as a critical thinking ability.

Inferential abilities : The ability to draw conclusions from given information, and to recognize with what degree of certainty one’s own or others’ conclusions follow, is universally recognized as a general critical thinking ability. All 11 examples in section 2 of this article include inferences, some from hypotheses or options (as in Transit , Ferryboat and Disorder ), others from something observed (as in Weather and Rash ). None of these inferences is formally valid. Rather, they are licensed by general, sometimes qualified substantive rules of inference (Toulmin 1958) that rest on domain knowledge—that a bus trip takes about the same time in each direction, that the terminal of a wireless telegraph would be located on the highest possible place, that sudden cooling is often followed by rain, that an allergic reaction to a sulfa drug generally shows up soon after one starts taking it. It is a matter of controversy to what extent the specialized ability to deduce conclusions from premisses using formal rules of inference is needed for critical thinking. Dewey (1933) locates logical forms in setting out the products of reflection rather than in the process of reflection. Ennis (1981a), on the other hand, maintains that a liberally-educated person should have the following abilities: to translate natural-language statements into statements using the standard logical operators, to use appropriately the language of necessary and sufficient conditions, to deal with argument forms and arguments containing symbols, to determine whether in virtue of an argument’s form its conclusion follows necessarily from its premisses, to reason with logically complex propositions, and to apply the rules and procedures of deductive logic. Inferential abilities are recognized as critical thinking abilities by Glaser (1941: 6), Facione (1990a: 9), Ennis (1991: 9), Fisher & Scriven (1997: 99, 111), and Halpern (1998: 452). Items testing inferential abilities constitute two of the five subtests of the Watson Glaser Critical Thinking Appraisal (Watson & Glaser 1980a, 1980b, 1994), two of the four sections in the Cornell Critical Thinking Test Level X (Ennis & Millman 1971; Ennis, Millman, & Tomko 1985, 2005), three of the seven sections in the Cornell Critical Thinking Test Level Z (Ennis & Millman 1971; Ennis, Millman, & Tomko 1985, 2005), 11 of the 34 items on Forms A and B of the California Critical Thinking Skills Test (Facione 1990b, 1992), and a high but variable proportion of the 25 selected-response questions in the Collegiate Learning Assessment (Council for Aid to Education 2017).

Experimenting abilities : Knowing how to design and execute an experiment is important not just in scientific research but also in everyday life, as in Rash . Dewey devoted a whole chapter of his How We Think (1910: 145–156; 1933: 190–202) to the superiority of experimentation over observation in advancing knowledge. Experimenting abilities come into play at one remove in appraising reports of scientific studies. Skill in designing and executing experiments includes the acknowledged abilities to appraise evidence (Glaser 1941: 6), to carry out experiments and to apply appropriate statistical inference techniques (Facione 1990a: 9), to judge inductions to an explanatory hypothesis (Ennis 1991: 9), and to recognize the need for an adequately large sample size (Halpern 1998). The Cornell Critical Thinking Test Level Z (Ennis & Millman 1971; Ennis, Millman, & Tomko 1985, 2005) includes four items (out of 52) on experimental design. The Collegiate Learning Assessment (Council for Aid to Education 2017) makes room for appraisal of study design in both its performance task and its selected-response questions.

Consulting abilities : Skill at consulting sources of information comes into play when one seeks information to help resolve a problem, as in Candidate . Ability to find and appraise information includes ability to gather and marshal pertinent information (Glaser 1941: 6), to judge whether a statement made by an alleged authority is acceptable (Ennis 1962: 84), to plan a search for desired information (Facione 1990a: 9), and to judge the credibility of a source (Ennis 1991: 9). Ability to judge the credibility of statements is tested by 24 items (out of 76) in the Cornell Critical Thinking Test Level X (Ennis & Millman 1971; Ennis, Millman, & Tomko 1985, 2005) and by four items (out of 52) in the Cornell Critical Thinking Test Level Z (Ennis & Millman 1971; Ennis, Millman, & Tomko 1985, 2005). The College Learning Assessment’s performance task requires evaluation of whether information in documents is credible or unreliable (Council for Aid to Education 2017).

Argument analysis abilities : The ability to identify and analyze arguments contributes to the process of surveying arguments on an issue in order to form one’s own reasoned judgment, as in Candidate . The ability to detect and analyze arguments is recognized as a critical thinking skill by Facione (1990a: 7–8), Ennis (1991: 9) and Halpern (1998). Five items (out of 34) on the California Critical Thinking Skills Test (Facione 1990b, 1992) test skill at argument analysis. The College Learning Assessment (Council for Aid to Education 2017) incorporates argument analysis in its selected-response tests of critical reading and evaluation and of critiquing an argument.

Judging skills and deciding skills : Skill at judging and deciding is skill at recognizing what judgment or decision the available evidence and argument supports, and with what degree of confidence. It is thus a component of the inferential skills already discussed.

Lists and tests of critical thinking abilities often include two more abilities: identifying assumptions and constructing and evaluating definitions.

In addition to dispositions and abilities, critical thinking needs knowledge: of critical thinking concepts, of critical thinking principles, and of the subject-matter of the thinking.

We can derive a short list of concepts whose understanding contributes to critical thinking from the critical thinking abilities described in the preceding section. Observational abilities require an understanding of the difference between observation and inference. Questioning abilities require an understanding of the concepts of ambiguity and vagueness. Inferential abilities require an understanding of the difference between conclusive and defeasible inference (traditionally, between deduction and induction), as well as of the difference between necessary and sufficient conditions. Experimenting abilities require an understanding of the concepts of hypothesis, null hypothesis, assumption and prediction, as well as of the concept of statistical significance and of its difference from importance. They also require an understanding of the difference between an experiment and an observational study, and in particular of the difference between a randomized controlled trial, a prospective correlational study and a retrospective (case-control) study. Argument analysis abilities require an understanding of the concepts of argument, premiss, assumption, conclusion and counter-consideration. Additional critical thinking concepts are proposed by Bailin et al. (1999b: 293), Fisher & Scriven (1997: 105–106), and Black (2012).

According to Glaser (1941: 25), ability to think critically requires knowledge of the methods of logical inquiry and reasoning. If we review the list of abilities in the preceding section, however, we can see that some of them can be acquired and exercised merely through practice, possibly guided in an educational setting, followed by feedback. Searching intelligently for a causal explanation of some phenomenon or event requires that one consider a full range of possible causal contributors, but it seems more important that one implements this principle in one’s practice than that one is able to articulate it. What is important is “operational knowledge” of the standards and principles of good thinking (Bailin et al. 1999b: 291–293). But the development of such critical thinking abilities as designing an experiment or constructing an operational definition can benefit from learning their underlying theory. Further, explicit knowledge of quirks of human thinking seems useful as a cautionary guide. Human memory is not just fallible about details, as people learn from their own experiences of misremembering, but is so malleable that a detailed, clear and vivid recollection of an event can be a total fabrication (Loftus 2017). People seek or interpret evidence in ways that are partial to their existing beliefs and expectations, often unconscious of their “confirmation bias” (Nickerson 1998). Not only are people subject to this and other cognitive biases (Kahneman 2011), of which they are typically unaware, but it may be counter-productive for one to make oneself aware of them and try consciously to counteract them or to counteract social biases such as racial or sexual stereotypes (Kenyon & Beaulac 2014). It is helpful to be aware of these facts and of the superior effectiveness of blocking the operation of biases—for example, by making an immediate record of one’s observations, refraining from forming a preliminary explanatory hypothesis, blind refereeing, double-blind randomized trials, and blind grading of students’ work.

Critical thinking about an issue requires substantive knowledge of the domain to which the issue belongs. Critical thinking abilities are not a magic elixir that can be applied to any issue whatever by somebody who has no knowledge of the facts relevant to exploring that issue. For example, the student in Bubbles needed to know that gases do not penetrate solid objects like a glass, that air expands when heated, that the volume of an enclosed gas varies directly with its temperature and inversely with its pressure, and that hot objects will spontaneously cool down to the ambient temperature of their surroundings unless kept hot by insulation or a source of heat. Critical thinkers thus need a rich fund of subject-matter knowledge relevant to the variety of situations they encounter. This fact is recognized in the inclusion among critical thinking dispositions of a concern to become and remain generally well informed.

Experimental educational interventions, with control groups, have shown that education can improve critical thinking skills and dispositions, as measured by standardized tests. For information about these tests, see the Supplement on Assessment .

What educational methods are most effective at developing the dispositions, abilities and knowledge of a critical thinker? Abrami et al. (2015) found that in the experimental and quasi-experimental studies that they analyzed dialogue, anchored instruction, and mentoring each increased the effectiveness of the educational intervention, and that they were most effective when combined. They also found that in these studies a combination of separate instruction in critical thinking with subject-matter instruction in which students are encouraged to think critically was more effective than either by itself. However, the difference was not statistically significant; that is, it might have arisen by chance.

Most of these studies lack the longitudinal follow-up required to determine whether the observed differential improvements in critical thinking abilities or dispositions continue over time, for example until high school or college graduation. For details on studies of methods of developing critical thinking skills and dispositions, see the Supplement on Educational Methods .

12. Controversies

Scholars have denied the generalizability of critical thinking abilities across subject domains, have alleged bias in critical thinking theory and pedagogy, and have investigated the relationship of critical thinking to other kinds of thinking.

McPeck (1981) attacked the thinking skills movement of the 1970s, including the critical thinking movement. He argued that there are no general thinking skills, since thinking is always thinking about some subject-matter. It is futile, he claimed, for schools and colleges to teach thinking as if it were a separate subject. Rather, teachers should lead their pupils to become autonomous thinkers by teaching school subjects in a way that brings out their cognitive structure and that encourages and rewards discussion and argument. As some of his critics (e.g., Paul 1985; Siegel 1985) pointed out, McPeck’s central argument needs elaboration, since it has obvious counter-examples in writing and speaking, for which (up to a certain level of complexity) there are teachable general abilities even though they are always about some subject-matter. To make his argument convincing, McPeck needs to explain how thinking differs from writing and speaking in a way that does not permit useful abstraction of its components from the subject-matters with which it deals. He has not done so. Nevertheless, his position that the dispositions and abilities of a critical thinker are best developed in the context of subject-matter instruction is shared by many theorists of critical thinking, including Dewey (1910, 1933), Glaser (1941), Passmore (1980), Weinstein (1990), and Bailin et al. (1999b).

McPeck’s challenge prompted reflection on the extent to which critical thinking is subject-specific. McPeck argued for a strong subject-specificity thesis, according to which it is a conceptual truth that all critical thinking abilities are specific to a subject. (He did not however extend his subject-specificity thesis to critical thinking dispositions. In particular, he took the disposition to suspend judgment in situations of cognitive dissonance to be a general disposition.) Conceptual subject-specificity is subject to obvious counter-examples, such as the general ability to recognize confusion of necessary and sufficient conditions. A more modest thesis, also endorsed by McPeck, is epistemological subject-specificity, according to which the norms of good thinking vary from one field to another. Epistemological subject-specificity clearly holds to a certain extent; for example, the principles in accordance with which one solves a differential equation are quite different from the principles in accordance with which one determines whether a painting is a genuine Picasso. But the thesis suffers, as Ennis (1989) points out, from vagueness of the concept of a field or subject and from the obvious existence of inter-field principles, however broadly the concept of a field is construed. For example, the principles of hypothetico-deductive reasoning hold for all the varied fields in which such reasoning occurs. A third kind of subject-specificity is empirical subject-specificity, according to which as a matter of empirically observable fact a person with the abilities and dispositions of a critical thinker in one area of investigation will not necessarily have them in another area of investigation.

The thesis of empirical subject-specificity raises the general problem of transfer. If critical thinking abilities and dispositions have to be developed independently in each school subject, how are they of any use in dealing with the problems of everyday life and the political and social issues of contemporary society, most of which do not fit into the framework of a traditional school subject? Proponents of empirical subject-specificity tend to argue that transfer is more likely to occur if there is critical thinking instruction in a variety of domains, with explicit attention to dispositions and abilities that cut across domains. But evidence for this claim is scanty. There is a need for well-designed empirical studies that investigate the conditions that make transfer more likely.

It is common ground in debates about the generality or subject-specificity of critical thinking dispositions and abilities that critical thinking about any topic requires background knowledge about the topic. For example, the most sophisticated understanding of the principles of hypothetico-deductive reasoning is of no help unless accompanied by some knowledge of what might be plausible explanations of some phenomenon under investigation.

Critics have objected to bias in the theory, pedagogy and practice of critical thinking. Commentators (e.g., Alston 1995; Ennis 1998) have noted that anyone who takes a position has a bias in the neutral sense of being inclined in one direction rather than others. The critics, however, are objecting to bias in the pejorative sense of an unjustified favoring of certain ways of knowing over others, frequently alleging that the unjustly favoured ways are those of a dominant sex or culture (Bailin 1995). These ways favour:

• reinforcement of egocentric and sociocentric biases over dialectical engagement with opposing world-views (Paul 1981, 1984; Warren 1998)
• distancing from the object of inquiry over closeness to it (Martin 1992; Thayer-Bacon 1992)
• indifference to the situation of others over care for them (Martin 1992)
• orientation to thought over orientation to action (Martin 1992)
• being reasonable over caring to understand people’s ideas (Thayer-Bacon 1993)
• being neutral and objective over being embodied and situated (Thayer-Bacon 1995a)
• doubting over believing (Thayer-Bacon 1995b)
• reason over emotion, imagination and intuition (Thayer-Bacon 2000)
• solitary thinking over collaborative thinking (Thayer-Bacon 2000)
• written and spoken assignments over other forms of expression (Alston 2001)
• attention to written and spoken communications over attention to human problems (Alston 2001)
• winning debates in the public sphere over making and understanding meaning (Alston 2001)

A common thread in this smorgasbord of accusations is dissatisfaction with focusing on the logical analysis and evaluation of reasoning and arguments. While these authors acknowledge that such analysis and evaluation is part of critical thinking and should be part of its conceptualization and pedagogy, they insist that it is only a part. Paul (1981), for example, bemoans the tendency of atomistic teaching of methods of analyzing and evaluating arguments to turn students into more able sophists, adept at finding fault with positions and arguments with which they disagree but even more entrenched in the egocentric and sociocentric biases with which they began. Martin (1992) and Thayer-Bacon (1992) cite with approval the self-reported intimacy with their subject-matter of leading researchers in biology and medicine, an intimacy that conflicts with the distancing allegedly recommended in standard conceptions and pedagogy of critical thinking. Thayer-Bacon (2000) contrasts the embodied and socially embedded learning of her elementary school students in a Montessori school, who used their imagination, intuition and emotions as well as their reason, with conceptions of critical thinking as

thinking that is used to critique arguments, offer justifications, and make judgments about what are the good reasons, or the right answers. (Thayer-Bacon 2000: 127–128)

Alston (2001) reports that her students in a women’s studies class were able to see the flaws in the Cinderella myth that pervades much romantic fiction but in their own romantic relationships still acted as if all failures were the woman’s fault and still accepted the notions of love at first sight and living happily ever after. Students, she writes, should

be able to connect their intellectual critique to a more affective, somatic, and ethical account of making risky choices that have sexist, racist, classist, familial, sexual, or other consequences for themselves and those both near and far… critical thinking that reads arguments, texts, or practices merely on the surface without connections to feeling/desiring/doing or action lacks an ethical depth that should infuse the difference between mere cognitive activity and something we want to call critical thinking. (Alston 2001: 34)

Some critics portray such biases as unfair to women. Thayer-Bacon (1992), for example, has charged modern critical thinking theory with being sexist, on the ground that it separates the self from the object and causes one to lose touch with one’s inner voice, and thus stigmatizes women, who (she asserts) link self to object and listen to their inner voice. Her charge does not imply that women as a group are on average less able than men to analyze and evaluate arguments. Facione (1990c) found no difference by sex in performance on his California Critical Thinking Skills Test. Kuhn (1991: 280–281) found no difference by sex in either the disposition or the competence to engage in argumentative thinking.

The critics propose a variety of remedies for the biases that they allege. In general, they do not propose to eliminate or downplay critical thinking as an educational goal. Rather, they propose to conceptualize critical thinking differently and to change its pedagogy accordingly. Their pedagogical proposals arise logically from their objections. They can be summarized as follows:

• Focus on argument networks with dialectical exchanges reflecting contesting points of view rather than on atomic arguments, so as to develop “strong sense” critical thinking that transcends egocentric and sociocentric biases (Paul 1981, 1984).
• Foster closeness to the subject-matter and feeling connected to others in order to inform a humane democracy (Martin 1992).
• Develop “constructive thinking” as a social activity in a community of physically embodied and socially embedded inquirers with personal voices who value not only reason but also imagination, intuition and emotion (Thayer-Bacon 2000).
• In developing critical thinking in school subjects, treat as important neither skills nor dispositions but opening worlds of meaning (Alston 2001).
• Attend to the development of critical thinking dispositions as well as skills, and adopt the “critical pedagogy” practised and advocated by Freire (1968 [1970]) and hooks (1994) (Dalgleish, Girard, & Davies 2017).

A common thread in these proposals is treatment of critical thinking as a social, interactive, personally engaged activity like that of a quilting bee or a barn-raising (Thayer-Bacon 2000) rather than as an individual, solitary, distanced activity symbolized by Rodin’s The Thinker . One can get a vivid description of education with the former type of goal from the writings of bell hooks (1994, 2010). Critical thinking for her is open-minded dialectical exchange across opposing standpoints and from multiple perspectives, a conception similar to Paul’s “strong sense” critical thinking (Paul 1981). She abandons the structure of domination in the traditional classroom. In an introductory course on black women writers, for example, she assigns students to write an autobiographical paragraph about an early racial memory, then to read it aloud as the others listen, thus affirming the uniqueness and value of each voice and creating a communal awareness of the diversity of the group’s experiences (hooks 1994: 84). Her “engaged pedagogy” is thus similar to the “freedom under guidance” implemented in John Dewey’s Laboratory School of Chicago in the late 1890s and early 1900s. It incorporates the dialogue, anchored instruction, and mentoring that Abrami (2015) found to be most effective in improving critical thinking skills and dispositions.

What is the relationship of critical thinking to problem solving, decision-making, higher-order thinking, creative thinking, and other recognized types of thinking? One’s answer to this question obviously depends on how one defines the terms used in the question. If critical thinking is conceived broadly to cover any careful thinking about any topic for any purpose, then problem solving and decision making will be kinds of critical thinking, if they are done carefully. Historically, ‘critical thinking’ and ‘problem solving’ were two names for the same thing. If critical thinking is conceived more narrowly as consisting solely of appraisal of intellectual products, then it will be disjoint with problem solving and decision making, which are constructive.

Bloom’s taxonomy of educational objectives used the phrase “intellectual abilities and skills” for what had been labeled “critical thinking” by some, “reflective thinking” by Dewey and others, and “problem solving” by still others (Bloom et al. 1956: 38). Thus, the so-called “higher-order thinking skills” at the taxonomy’s top levels of analysis, synthesis and evaluation are just critical thinking skills, although they do not come with general criteria for their assessment (Ennis 1981b). The revised version of Bloom’s taxonomy (Anderson et al. 2001) likewise treats critical thinking as cutting across those types of cognitive process that involve more than remembering (Anderson et al. 2001: 269–270). For details, see the Supplement on History .

As to creative thinking, it overlaps with critical thinking (Bailin 1987, 1988). Thinking about the explanation of some phenomenon or event, as in Ferryboat , requires creative imagination in constructing plausible explanatory hypotheses. Likewise, thinking about a policy question, as in Candidate , requires creativity in coming up with options. Conversely, creativity in any field needs to be balanced by critical appraisal of the draft painting or novel or mathematical theory.

• Abrami, Philip C., Robert M. Bernard, Eugene Borokhovski, David I. Waddington, C. Anne Wade, and Tonje Person, 2015, “Strategies for Teaching Students to Think Critically: A Meta-analysis”, Review of Educational Research , 85(2): 275–314. doi:10.3102/0034654314551063
• Aikin, Wilford M., 1942, The Story of the Eight-year Study, with Conclusions and Recommendations , Volume I of Adventure in American Education , New York and London: Harper & Brothers. [ Aikin 1942 available online ]
• Alston, Kal, 1995, “Begging the Question: Is Critical Thinking Biased?”, Educational Theory , 45(2): 225–233. doi:10.1111/j.1741-5446.1995.00225.x
• –––, 2001, “Re/Thinking Critical Thinking: The Seductions of Everyday Life”, Studies in Philosophy and Education , 20(1): 27–40. doi:10.1023/A:1005247128053
• American Educational Research Association, 2014, Standards for Educational and Psychological Testing / American Educational Research Association, American Psychological Association, National Council on Measurement in Education , Washington, DC: American Educational Research Association.
• Anderson, Lorin W., David R. Krathwohl, Peter W. Airiasian, Kathleen A. Cruikshank, Richard E. Mayer, Paul R. Pintrich, James Raths, and Merlin C. Wittrock, 2001, A Taxonomy for Learning, Teaching and Assessing: A Revision of Bloom’s Taxonomy of Educational Objectives , New York: Longman, complete edition.
• Bailin, Sharon, 1987, “Critical and Creative Thinking”, Informal Logic , 9(1): 23–30. [ Bailin 1987 available online ]
• –––, 1988, Achieving Extraordinary Ends: An Essay on Creativity , Dordrecht: Kluwer. doi:10.1007/978-94-009-2780-3
• –––, 1995, “Is Critical Thinking Biased? Clarifications and Implications”, Educational Theory , 45(2): 191–197. doi:10.1111/j.1741-5446.1995.00191.x
• Bailin, Sharon and Mark Battersby, 2009, “Inquiry: A Dialectical Approach to Teaching Critical Thinking”, in Juho Ritola (ed.), Argument Cultures: Proceedings of OSSA 09 , CD-ROM (pp. 1–10), Windsor, ON: OSSA. [ Bailin & Battersby 2009 available online ]
• –––, 2016, “Fostering the Virtues of Inquiry”, Topoi , 35(2): 367–374. doi:10.1007/s11245-015-9307-6
• Bailin, Sharon, Roland Case, Jerrold R. Coombs, and Leroi B. Daniels, 1999a, “Common Misconceptions of Critical Thinking”, Journal of Curriculum Studies , 31(3): 269–283. doi:10.1080/002202799183124
• –––, 1999b, “Conceptualizing Critical Thinking”, Journal of Curriculum Studies , 31(3): 285–302. doi:10.1080/002202799183133
• Berman, Alan M., Seth J. Schwartz, William M. Kurtines, and Steven L. Berman, 2001, “The Process of Exploration in Identity Formation: The Role of Style and Competence”, Journal of Adolescence , 24(4): 513–528. doi:10.1006/jado.2001.0386
• Black, Beth (ed.), 2012, An A to Z of Critical Thinking , London: Continuum International Publishing Group.
• Bloom, Benjamin Samuel, Max D. Engelhart, Edward J. Furst, Walter H. Hill, and David R. Krathwohl, 1956, Taxonomy of Educational Objectives. Handbook I: Cognitive Domain , New York: David McKay.
• Casserly, Megan, 2012, “The 10 Skills That Will Get You Hired in 2013”, Forbes , Dec. 10, 2012. Available at https://www.forbes.com/sites/meghancasserly/2012/12/10/the-10-skills-that-will-get-you-a-job-in-2013/#79e7ff4e633d ; accessed 2017 11 06.
• Center for Assessment & Improvement of Learning, 2017, Critical Thinking Assessment Test , Cookeville, TN: Tennessee Technological University.
• Cohen, Jacob, 1988, Statistical Power Analysis for the Behavioral Sciences , Hillsdale, NJ: Lawrence Erlbaum Associates, 2nd edition.
• College Board, 1983, Academic Preparation for College. What Students Need to Know and Be Able to Do , New York: College Entrance Examination Board, ERIC document ED232517.
• Commission on the Relation of School and College of the Progressive Education Association, 1943, Thirty Schools Tell Their Story , Volume V of Adventure in American Education , New York and London: Harper & Brothers.
• Council for Aid to Education, 2017, CLA+ Student Guide . Available at http://cae.org/images/uploads/pdf/CLA_Student_Guide_Institution.pdf ; accessed 2017 09 26.
• Dalgleish, Adam, Patrick Girard, and Maree Davies, 2017, “Critical Thinking, Bias and Feminist Philosophy: Building a Better Framework through Collaboration”, Informal Logic , 37(4): 351–369. [ Dalgleish et al. available online ]
• Dewey, John, 1910, How We Think , Boston: D.C. Heath. [ Dewey 1910 available online ]
• –––, 1916, Democracy and Education: An Introduction to the Philosophy of Education , New York: Macmillan.
• –––, 1933, How We Think: A Restatement of the Relation of Reflective Thinking to the Educative Process , Lexington, MA: D.C. Heath.
• –––, 1936, “The Theory of the Chicago Experiment”, Appendix II of Mayhew & Edwards 1936: 463–477.
• –––, 1938, Logic: The Theory of Inquiry , New York: Henry Holt and Company.
• Dominguez, Caroline (coord.), 2018a, A European Collection of the Critical Thinking Skills and Dispositions Needed in Different Professional Fields for the 21st Century , Vila Real, Portugal: UTAD. Available at http://bit.ly/CRITHINKEDUO1 ; accessed 2018 04 09.
• ––– (coord.), 2018b, A European Review on Critical Thinking Educational Practices in Higher Education Institutions , Vila Real: UTAD. Available at http://bit.ly/CRITHINKEDUO2 ; accessed 2018 04 14.
• Dumke, Glenn S., 1980, Chancellor’s Executive Order 338 , Long Beach, CA: California State University, Chancellor’s Office. Available at https://www.calstate.edu/eo/EO-338.pdf ; accessed 2017 11 16.
• Ennis, Robert H., 1958, “An Appraisal of the Watson-Glaser Critical Thinking Appraisal”, The Journal of Educational Research , 52(4): 155–158. doi:10.1080/00220671.1958.10882558
• –––, 1962, “A Concept of Critical Thinking: A Proposed Basis for Research on the Teaching and Evaluation of Critical Thinking Ability”, Harvard Educational Review , 32(1): 81–111.
• –––, 1981a, “A Conception of Deductive Logical Competence”, Teaching Philosophy , 4(3/4): 337–385. doi:10.5840/teachphil198143/429
• –––, 1981b, “Eight Fallacies in Bloom’s Taxonomy”, in C. J. B. Macmillan (ed.), Philosophy of Education 1980: Proceedings of the Thirty-seventh Annual Meeting of the Philosophy of Education Society , Bloomington, IL: Philosophy of Education Society, pp. 269–273.
• –––, 1984, “Problems in Testing Informal Logic, Critical Thinking, Reasoning Ability”. Informal Logic , 6(1): 3–9. [ Ennis 1984 available online ]
• –––, 1987, “A Taxonomy of Critical Thinking Dispositions and Abilities”, in Joan Boykoff Baron and Robert J. Sternberg (eds.), Teaching Thinking Skills: Theory and Practice , New York: W. H. Freeman, pp. 9–26.
• –––, 1989, “Critical Thinking and Subject Specificity: Clarification and Needed Research”, Educational Researcher , 18(3): 4–10. doi:10.3102/0013189X018003004
• –––, 1991, “Critical Thinking: A Streamlined Conception”, Teaching Philosophy , 14(1): 5–24. doi:10.5840/teachphil19911412
• –––, 1996, “Critical Thinking Dispositions: Their Nature and Assessability”, Informal Logic , 18(2–3): 165–182. [ Ennis 1996 available online ]
• –––, 1998, “Is Critical Thinking Culturally Biased?”, Teaching Philosophy , 21(1): 15–33. doi:10.5840/teachphil19982113
• –––, 2011, “Critical Thinking: Reflection and Perspective Part I”, Inquiry: Critical Thinking across the Disciplines , 26(1): 4–18. doi:10.5840/inquiryctnews20112613
• –––, 2013, “Critical Thinking across the Curriculum: The Wisdom CTAC Program”, Inquiry: Critical Thinking across the Disciplines , 28(2): 25–45. doi:10.5840/inquiryct20132828
• –––, 2016, “Definition: A Three-Dimensional Analysis with Bearing on Key Concepts”, in Patrick Bondy and Laura Benacquista (eds.), Argumentation, Objectivity, and Bias: Proceedings of the 11th International Conference of the Ontario Society for the Study of Argumentation (OSSA), 18–21 May 2016 , Windsor, ON: OSSA, pp. 1–19. Available at http://scholar.uwindsor.ca/ossaarchive/OSSA11/papersandcommentaries/105 ; accessed 2017 12 02.
• –––, 2018, “Critical Thinking Across the Curriculum: A Vision”, Topoi , 37(1): 165–184. doi:10.1007/s11245-016-9401-4
• Ennis, Robert H., and Jason Millman, 1971, Manual for Cornell Critical Thinking Test, Level X, and Cornell Critical Thinking Test, Level Z , Urbana, IL: Critical Thinking Project, University of Illinois.
• Ennis, Robert H., Jason Millman, and Thomas Norbert Tomko, 1985, Cornell Critical Thinking Tests Level X & Level Z: Manual , Pacific Grove, CA: Midwest Publication, 3rd edition.
• –––, 2005, Cornell Critical Thinking Tests Level X & Level Z: Manual , Seaside, CA: Critical Thinking Company, 5th edition.
• Ennis, Robert H. and Eric Weir, 1985, The Ennis-Weir Critical Thinking Essay Test: Test, Manual, Criteria, Scoring Sheet: An Instrument for Teaching and Testing , Pacific Grove, CA: Midwest Publications.
• Facione, Peter A., 1990a, Critical Thinking: A Statement of Expert Consensus for Purposes of Educational Assessment and Instruction , Research Findings and Recommendations Prepared for the Committee on Pre-College Philosophy of the American Philosophical Association, ERIC Document ED315423.
• –––, 1990b, California Critical Thinking Skills Test, CCTST – Form A , Millbrae, CA: The California Academic Press.
• –––, 1990c, The California Critical Thinking Skills Test--College Level. Technical Report #3. Gender, Ethnicity, Major, CT Self-Esteem, and the CCTST , ERIC Document ED326584.
• –––, 1992, California Critical Thinking Skills Test: CCTST – Form B, Millbrae, CA: The California Academic Press.
• –––, 2000, “The Disposition Toward Critical Thinking: Its Character, Measurement, and Relationship to Critical Thinking Skill”, Informal Logic , 20(1): 61–84. [ Facione 2000 available online ]
• Facione, Peter A. and Noreen C. Facione, 1992, CCTDI: A Disposition Inventory , Millbrae, CA: The California Academic Press.
• Facione, Peter A., Noreen C. Facione, and Carol Ann F. Giancarlo, 2001, California Critical Thinking Disposition Inventory: CCTDI: Inventory Manual , Millbrae, CA: The California Academic Press.
• Facione, Peter A., Carol A. Sánchez, and Noreen C. Facione, 1994, Are College Students Disposed to Think? , Millbrae, CA: The California Academic Press. ERIC Document ED368311.
• Fisher, Alec, and Michael Scriven, 1997, Critical Thinking: Its Definition and Assessment , Norwich: Centre for Research in Critical Thinking, University of East Anglia.
• Freire, Paulo, 1968 [1970], Pedagogia do Oprimido . Translated as Pedagogy of the Oppressed , Myra Bergman Ramos (trans.), New York: Continuum, 1970.
• Glaser, Edward Maynard, 1941, An Experiment in the Development of Critical Thinking , New York: Bureau of Publications, Teachers College, Columbia University.
• Halpern, Diane F., 1998, “Teaching Critical Thinking for Transfer Across Domains: Disposition, Skills, Structure Training, and Metacognitive Monitoring”, American Psychologist , 53(4): 449–455. doi:10.1037/0003-066X.53.4.449
• –––, 2016, Manual: Halpern Critical Thinking Assessment , Mödling, Austria: Schuhfried. Available at https://drive.google.com/file/d/0BzUoP_pmwy1gdEpCR05PeW9qUzA/view ; accessed 2017 12 01.
• Hamby, Benjamin, 2014, The Virtues of Critical Thinkers , Doctoral dissertation, Philosophy, McMaster University. [ Hamby 2014 available online ]
• –––, 2015, “Willingness to Inquire: The Cardinal Critical Thinking Virtue”, in Martin Davies and Ronald Barnett (eds.), The Palgrave Handbook of Critical Thinking in Higher Education , New York: Palgrave Macmillan, pp. 77–87.
• Haynes, Ada, Elizabeth Lisic, Kevin Harris, Katie Leming, Kyle Shanks, and Barry Stein, 2015, “Using the Critical Thinking Assessment Test (CAT) as a Model for Designing Within-Course Assessments: Changing How Faculty Assess Student Learning”, Inquiry: Critical Thinking Across the Disciplines , 30(3): 38–48. doi:10.5840/inquiryct201530316
• Hitchcock, David, 2017, “Critical Thinking as an Educational Ideal”, in his On Reasoning and Argument: Essays in Informal Logic and on Critical Thinking , Dordrecht: Springer, pp. 477–497. doi:10.1007/978-3-319-53562-3_30
• hooks, bell, 1994, Teaching to Transgress: Education as the Practice of Freedom , New York and London: Routledge.
• –––, 2010, Teaching Critical Thinking: Practical Wisdom , New York and London: Routledge.
• Johnson, Ralph H., 1992, “The Problem of Defining Critical Thinking”, in Stephen P, Norris (ed.), The Generalizability of Critical Thinking , New York: Teachers College Press, pp. 38–53.
• Kahane, Howard, 1971, Logic and Contemporary Rhetoric: The Use of Reason in Everyday Life , Belmont, CA: Wadsworth.
• Kahneman, Daniel, 2011, Thinking, Fast and Slow , New York: Farrar, Straus and Giroux.
• Kenyon, Tim, and Guillaume Beaulac, 2014, “Critical Thinking Education and Debasing”, Informal Logic , 34(4): 341–363. [ Kenyon & Beaulac 2014 available online ]
• Krathwohl, David R., Benjamin S. Bloom, and Bertram B. Masia, 1964, Taxonomy of Educational Objectives, Handbook II: Affective Domain , New York: David McKay.
• Kuhn, Deanna, 1991, The Skills of Argument , New York: Cambridge University Press. doi:10.1017/CBO9780511571350
• Lipman, Matthew, 1987, “Critical Thinking–What Can It Be?”, Analytic Teaching , 8(1): 5–12. [ Lipman 1987 available online ]
• Loftus, Elizabeth F., 2017, “Eavesdropping on Memory”, Annual Review of Psychology , 68: 1–18. doi:10.1146/annurev-psych-010416-044138
• Martin, Jane Roland, 1992, “Critical Thinking for a Humane World”, in Stephen P. Norris (ed.), The Generalizability of Critical Thinking , New York: Teachers College Press, pp. 163–180.
• Mayhew, Katherine Camp, and Anna Camp Edwards, 1936, The Dewey School: The Laboratory School of the University of Chicago, 1896–1903 , New York: Appleton-Century. [ Mayhew & Edwards 1936 available online ]
• McPeck, John E., 1981, Critical Thinking and Education , New York: St. Martin’s Press.
• Nickerson, Raymond S., 1998, “Confirmation Bias: A Ubiquitous Phenomenon in Many Guises”, Review of General Psychology , 2(2): 175–220. doi:10.1037/1089-2680.2.2.175
• Nieto, Ana Maria, and Jorge Valenzuela, 2012, “A Study of the Internal Structure of Critical Thinking Dispositions”, Inquiry: Critical Thinking across the Disciplines , 27(1): 31–38. doi:10.5840/inquiryct20122713
• Norris, Stephen P., 1985, “Controlling for Background Beliefs When Developing Multiple-choice Critical Thinking Tests”, Educational Measurement: Issues and Practice , 7(3): 5–11. doi:10.1111/j.1745-3992.1988.tb00437.x
• Norris, Stephen P. and Robert H. Ennis, 1989, Evaluating Critical Thinking (The Practitioners’ Guide to Teaching Thinking Series), Pacific Grove, CA: Midwest Publications.
• Norris, Stephen P. and Ruth Elizabeth King, 1983, Test on Appraising Observations , St. John’s, NL: Institute for Educational Research and Development, Memorial University of Newfoundland.
• –––, 1984, The Design of a Critical Thinking Test on Appraising Observations , St. John’s, NL: Institute for Educational Research and Development, Memorial University of Newfoundland. ERIC Document ED260083.
• –––, 1985, Test on Appraising Observations: Manual , St. John’s, NL: Institute for Educational Research and Development, Memorial University of Newfoundland.
• –––, 1990a, Test on Appraising Observations , St. John’s, NL: Institute for Educational Research and Development, Memorial University of Newfoundland, 2nd edition.
• –––, 1990b, Test on Appraising Observations: Manual , St. John’s, NL: Institute for Educational Research and Development, Memorial University of Newfoundland, 2nd edition.
• Obama, Barack, 2014, State of the Union Address , January 28, 2014. [ Obama 2014 available online ]
• OCR [Oxford, Cambridge and RSA Examinations], 2011, AS/A Level GCE: Critical Thinking – H052, H452 , Cambridge: OCR. Information available at http://www.ocr.org.uk/qualifications/as-a-level-gce-critical-thinking-h052-h452/ ; accessed 2017 10 12.
• OECD [Organization for Economic Cooperation and Development] Centre for Educational Research and Innovation, 2018, Fostering and Assessing Students’ Creative and Critical Thinking Skills in Higher Education , Paris: OECD. Available at http://www.oecd.org/education/ceri/Fostering-and-assessing-students-creative-and-critical-thinking-skills-in-higher-education.pdf ; accessed 2018 04 22.
• Ontario Ministry of Education, 2013, The Ontario Curriculum Grades 9 to 12: Social Sciences and Humanities . Available at http://www.edu.gov.on.ca/eng/curriculum/secondary/ssciences9to122013.pdf ; accessed 2017 11 16.
• Passmore, John Arthur, 1980, The Philosophy of Teaching , London: Duckworth.
• Paul, Richard W., 1981, “Teaching Critical Thinking in the ‘Strong’ Sense: A Focus on Self-Deception, World Views, and a Dialectical Mode of Analysis”, Informal Logic , 4(2): 2–7. [ Paul 1981 available online ]
• –––, 1984, “Critical Thinking: Fundamental to Education for a Free Society”, Educational Leadership , 42(1): 4–14.
• –––, 1985, “McPeck’s Mistakes”, Informal Logic , 7(1): 35–43. [ Paul 1985 available online ]
• Paul, Richard W. and Linda Elder, 2006, The Miniature Guide to Critical Thinking: Concepts and Tools , Dillon Beach, CA: Foundation for Critical Thinking, 4th edition.
• Payette, Patricia, and Edna Ross, 2016, “Making a Campus-Wide Commitment to Critical Thinking: Insights and Promising Practices Utilizing the Paul-Elder Approach at the University of Louisville”, Inquiry: Critical Thinking Across the Disciplines , 31(1): 98–110. doi:10.5840/inquiryct20163118
• Possin, Kevin, 2008, “A Field Guide to Critical-Thinking Assessment”, Teaching Philosophy , 31(3): 201–228. doi:10.5840/teachphil200831324
• –––, 2013a, “Some Problems with the Halpern Critical Thinking Assessment (HCTA) Test”, Inquiry: Critical Thinking across the Disciplines , 28(3): 4–12. doi:10.5840/inquiryct201328313
• –––, 2013b, “A Serious Flaw in the Collegiate Learning Assessment (CLA) Test”, Informal Logic , 33(3): 390–405. [ Possin 2013b available online ]
• –––, 2014, “Critique of the Watson-Glaser Critical Thinking Appraisal Test: The More You Know, the Lower Your Score”, Informal Logic , 34(4): 393–416. [ Possin 2014 available online ]
• Rawls, John, 1971, A Theory of Justice , Cambridge, MA: Harvard University Press.
• Rousseau, Jean-Jacques, 1762, Émile , Amsterdam: Jean Néaulme.
• Scheffler, Israel, 1960, The Language of Education , Springfield, IL: Charles C. Thomas.
• Scriven, Michael, and Richard W. Paul, 1987, Defining Critical Thinking , Draft statement written for the National Council for Excellence in Critical Thinking Instruction. Available at http://www.criticalthinking.org/pages/defining-critical-thinking/766 ; accessed 2017 11 29.
• Sheffield, Clarence Burton Jr., 2018, “Promoting Critical Thinking in Higher Education: My Experiences as the Inaugural Eugene H. Fram Chair in Applied Critical Thinking at Rochester Institute of Technology”, Topoi , 37(1): 155–163. doi:10.1007/s11245-016-9392-1
• Siegel, Harvey, 1985, “McPeck, Informal Logic and the Nature of Critical Thinking”, in David Nyberg (ed.), Philosophy of Education 1985: Proceedings of the Forty-First Annual Meeting of the Philosophy of Education Society , Normal, IL: Philosophy of Education Society, pp. 61–72.
• –––, 1988, Educating Reason: Rationality, Critical Thinking, and Education , New York: Routledge.
• –––, 1999, “What (Good) Are Thinking Dispositions?”, Educational Theory , 49(2): 207–221. doi:10.1111/j.1741-5446.1999.00207.x
• Simpson, Elizabeth, 1966–67, “The Classification of Educational Objectives: Psychomotor Domain”, Illinois Teacher of Home Economics , 10(4): 110–144, ERIC document ED0103613. [ Simpson 1966–67 available online ]
• Skolverket, 2011, Curriculum for the Compulsory School, Preschool Class and the Recreation Centre , Stockholm: Ordförrådet AB. Available at http://malmo.se/download/18.29c3b78a132728ecb52800034181/pdf2687.pdf ; accessed 2017 11 16.
• Smith, B. Othanel, 1953, “The Improvement of Critical Thinking”, Progressive Education , 30(5): 129–134.
• Smith, Eugene Randolph, Ralph Winfred Tyler, and the Evaluation Staff, 1942, Appraising and Recording Student Progress , Volume III of Adventure in American Education , New York and London: Harper & Brothers.
• Splitter, Laurance J., 1987, “Educational Reform through Philosophy for Children”, Thinking: The Journal of Philosophy for Children , 7(2): 32–39. doi:10.5840/thinking1987729
• Stanovich Keith E., and Paula J. Stanovich, 2010, “A Framework for Critical Thinking, Rational Thinking, and Intelligence”, in David D. Preiss and Robert J. Sternberg (eds), Innovations in Educational Psychology: Perspectives on Learning, Teaching and Human Development , New York: Springer Publishing, pp 195–237.
• Stanovich Keith E., Richard F. West, and Maggie E. Toplak, 2011, “Intelligence and Rationality”, in Robert J. Sternberg and Scott Barry Kaufman (eds.), Cambridge Handbook of Intelligence , Cambridge: Cambridge University Press, 3rd edition, pp. 784–826. doi:10.1017/CBO9780511977244.040
• Tankersley, Karen, 2005, Literacy Strategies for Grades 4–12: Reinforcing the Threads of Reading , Alexandria, VA: Association for Supervision and Curriculum Development.
• Thayer-Bacon, Barbara J., 1992, “Is Modern Critical Thinking Theory Sexist?”, Inquiry: Critical Thinking Across the Disciplines , 10(1): 3–7. doi:10.5840/inquiryctnews199210123
• –––, 1993, “Caring and Its Relationship to Critical Thinking”, Educational Theory , 43(3): 323–340. doi:10.1111/j.1741-5446.1993.00323.x
• –––, 1995a, “Constructive Thinking: Personal Voice”, Journal of Thought , 30(1): 55–70.
• –––, 1995b, “Doubting and Believing: Both are Important for Critical Thinking”, Inquiry: Critical Thinking across the Disciplines , 15(2): 59–66. doi:10.5840/inquiryctnews199515226
• –––, 2000, Transforming Critical Thinking: Thinking Constructively , New York: Teachers College Press.
• Toulmin, Stephen Edelston, 1958, The Uses of Argument , Cambridge: Cambridge University Press.
• Turri, John, Mark Alfano, and John Greco, 2017, “Virtue Epistemology”, in Edward N. Zalta (ed.), The Stanford Encyclopedia of Philosophy (Winter 2017 Edition). URL = < https://plato.stanford.edu/archives/win2017/entries/epistemology-virtue/ >
• Warren, Karen J. 1988. “Critical Thinking and Feminism”, Informal Logic , 10(1): 31–44. [ Warren 1988 available online ]
• Watson, Goodwin, and Edward M. Glaser, 1980a, Watson-Glaser Critical Thinking Appraisal, Form A , San Antonio, TX: Psychological Corporation.
• –––, 1980b, Watson-Glaser Critical Thinking Appraisal: Forms A and B; Manual , San Antonio, TX: Psychological Corporation,
• –––, 1994, Watson-Glaser Critical Thinking Appraisal, Form B , San Antonio, TX: Psychological Corporation.
• Weinstein, Mark, 1990, “Towards a Research Agenda for Informal Logic and Critical Thinking”, Informal Logic , 12(3): 121–143. [ Weinstein 1990 available online ]
• –––, 2013, Logic, Truth and Inquiry , London: College Publications.
• Zagzebski, Linda Trinkaus, 1996, Virtues of the Mind: An Inquiry into the Nature of Virtue and the Ethical Foundations of Knowledge , Cambridge: Cambridge University Press. doi:10.1017/CBO9781139174763

How to cite this entry . Preview the PDF version of this entry at the Friends of the SEP Society . Look up this entry topic at the Internet Philosophy Ontology Project (InPhO). Enhanced bibliography for this entry at PhilPapers , with links to its database.

• Association for Informal Logic and Critical Thinking (AILACT)
• Center for Teaching Thinking (CTT)
• Critical Thinking Across the European Higher Education Curricula (CRITHINKEDU)
• Critical Thinking Definition, Instruction, and Assessment: A Rigorous Approach (criticalTHINKING.net)
• Critical Thinking Research (RAIL)
• Foundation for Critical Thinking
• Insight Assessment
• Partnership for 21st Century Learning (P21)
• The Critical Thinking Consortium
• The Nature of Critical Thinking: An Outline of Critical Thinking Dispositions and Abilities , by Robert H. Ennis

abilities | bias, implicit | children, philosophy for | civic education | decision-making capacity | Dewey, John | dispositions | education, philosophy of | epistemology: virtue | logic: informal

Copyright © 2018 by David Hitchcock < hitchckd @ mcmaster . ca >

Support SEP

Mirror sites.

View this site from another server:

• Info about mirror sites

The Stanford Encyclopedia of Philosophy is copyright © 2016 by The Metaphysics Research Lab , Center for the Study of Language and Information (CSLI), Stanford University

Library of Congress Catalog Data: ISSN 1095-5054