case control research method

• En español – ExME
• Em português – EME

Case-control and Cohort studies: A brief overview

Posted on 6th December 2017 by Saul Crandon

Introduction

Case-control and cohort studies are observational studies that lie near the middle of the hierarchy of evidence . These types of studies, along with randomised controlled trials, constitute analytical studies, whereas case reports and case series define descriptive studies (1). Although these studies are not ranked as highly as randomised controlled trials, they can provide strong evidence if designed appropriately.

Case-control studies

Case-control studies are retrospective. They clearly define two groups at the start: one with the outcome/disease and one without the outcome/disease. They look back to assess whether there is a statistically significant difference in the rates of exposure to a defined risk factor between the groups. See Figure 1 for a pictorial representation of a case-control study design. This can suggest associations between the risk factor and development of the disease in question, although no definitive causality can be drawn. The main outcome measure in case-control studies is odds ratio (OR) .

Figure 1. Case-control study design.

Cases should be selected based on objective inclusion and exclusion criteria from a reliable source such as a disease registry. An inherent issue with selecting cases is that a certain proportion of those with the disease would not have a formal diagnosis, may not present for medical care, may be misdiagnosed or may have died before getting a diagnosis. Regardless of how the cases are selected, they should be representative of the broader disease population that you are investigating to ensure generalisability.

Case-control studies should include two groups that are identical EXCEPT for their outcome / disease status.

As such, controls should also be selected carefully. It is possible to match controls to the cases selected on the basis of various factors (e.g. age, sex) to ensure these do not confound the study results. It may even increase statistical power and study precision by choosing up to three or four controls per case (2).

Case-controls can provide fast results and they are cheaper to perform than most other studies. The fact that the analysis is retrospective, allows rare diseases or diseases with long latency periods to be investigated. Furthermore, you can assess multiple exposures to get a better understanding of possible risk factors for the defined outcome / disease.

Nevertheless, as case-controls are retrospective, they are more prone to bias. One of the main examples is recall bias. Often case-control studies require the participants to self-report their exposure to a certain factor. Recall bias is the systematic difference in how the two groups may recall past events e.g. in a study investigating stillbirth, a mother who experienced this may recall the possible contributing factors a lot more vividly than a mother who had a healthy birth.

A summary of the pros and cons of case-control studies are provided in Table 1.

Table 1. Advantages and disadvantages of case-control studies.

Cohort studies

Cohort studies can be retrospective or prospective. Retrospective cohort studies are NOT the same as case-control studies.

In retrospective cohort studies, the exposure and outcomes have already happened. They are usually conducted on data that already exists (from prospective studies) and the exposures are defined before looking at the existing outcome data to see whether exposure to a risk factor is associated with a statistically significant difference in the outcome development rate.

Prospective cohort studies are more common. People are recruited into cohort studies regardless of their exposure or outcome status. This is one of their important strengths. People are often recruited because of their geographical area or occupation, for example, and researchers can then measure and analyse a range of exposures and outcomes.

The study then follows these participants for a defined period to assess the proportion that develop the outcome/disease of interest. See Figure 2 for a pictorial representation of a cohort study design. Therefore, cohort studies are good for assessing prognosis, risk factors and harm. The outcome measure in cohort studies is usually a risk ratio / relative risk (RR).

Figure 2. Cohort study design.

Cohort studies should include two groups that are identical EXCEPT for their exposure status.

As a result, both exposed and unexposed groups should be recruited from the same source population. Another important consideration is attrition. If a significant number of participants are not followed up (lost, death, dropped out) then this may impact the validity of the study. Not only does it decrease the study’s power, but there may be attrition bias – a significant difference between the groups of those that did not complete the study.

Cohort studies can assess a range of outcomes allowing an exposure to be rigorously assessed for its impact in developing disease. Additionally, they are good for rare exposures, e.g. contact with a chemical radiation blast.

Whilst cohort studies are useful, they can be expensive and time-consuming, especially if a long follow-up period is chosen or the disease itself is rare or has a long latency.

A summary of the pros and cons of cohort studies are provided in Table 2.

The Strengthening of Reporting of Observational Studies in Epidemiology Statement (STROBE)

STROBE provides a checklist of important steps for conducting these types of studies, as well as acting as best-practice reporting guidelines (3). Both case-control and cohort studies are observational, with varying advantages and disadvantages. However, the most important factor to the quality of evidence these studies provide, is their methodological quality.

• Song, J. and Chung, K. Observational Studies: Cohort and Case-Control Studies .  Plastic and Reconstructive Surgery.  2010 Dec;126(6):2234-2242.
• Ury HK. Efficiency of case-control studies with multiple controls per case: Continuous or dichotomous data .  Biometrics . 1975 Sep;31(3):643–649.
• von Elm E, Altman DG, Egger M, Pocock SJ, Gøtzsche PC, Vandenbroucke JP; STROBE Initiative.  The Strengthening the Reporting of Observational Studies in Epidemiology (STROBE) statement: guidelines for reporting observational studies.   Lancet 2007 Oct;370(9596):1453-14577. PMID: 18064739.

Saul Crandon

Leave a reply cancel reply.

Your email address will not be published. Required fields are marked *

Save my name, email, and website in this browser for the next time I comment.

No Comments on Case-control and Cohort studies: A brief overview

Very well presented, excellent clarifications. Has put me right back into class, literally!

Very clear and informative! Thank you.

very informative article.

Thank you for the easy to understand blog in cohort studies. I want to follow a group of people with and without a disease to see what health outcomes occurs to them in future such as hospitalisations, diagnoses, procedures etc, as I have many health outcomes to consider, my questions is how to make sure these outcomes has not occurred before the “exposure disease”. As, in cohort studies we are looking at incidence (new) cases, so if an outcome have occurred before the exposure, I can leave them out of the analysis. But because I am not looking at a single outcome which can be checked easily and if happened before exposure can be left out. I have EHR data, so all the exposure and outcome have occurred. my aim is to check the rates of different health outcomes between the exposed)dementia) and unexposed(non-dementia) individuals.

Very helpful information

Thanks for making this subject student friendly and easier to understand. A great help.

Thanks a lot. It really helped me to understand the topic. I am taking epidemiology class this winter, and your paper really saved me.

Happy new year.

Wow its amazing n simple way of briefing ,which i was enjoyed to learn this.its very easy n quick to pick ideas .. Thanks n stay connected

Saul you absolute melt! Really good work man

am a student of public health. This information is simple and well presented to the point. Thank you so much.

very helpful information provided here

really thanks for wonderful information because i doing my bachelor degree research by survival model

Quite informative thank you so much for the info please continue posting. An mph student with Africa university Zimbabwe.

Thank you this was so helpful amazing

Apreciated the information provided above.

So clear and perfect. The language is simple and superb.I am recommending this to all budding epidemiology students. Thanks a lot.

Great to hear, thank you AJ!

I have recently completed an investigational study where evidence of phlebitis was determined in a control cohort by data mining from electronic medical records. We then introduced an intervention in an attempt to reduce incidence of phlebitis in a second cohort. Again, results were determined by data mining. This was an expedited study, so there subjects were enrolled in a specific cohort based on date(s) of the drug infused. How do I define this study? Thanks so much.

thanks for the information and knowledge about observational studies. am a masters student in public health/epidemilogy of the faculty of medicines and pharmaceutical sciences , University of Dschang. this information is very explicit and straight to the point

Very much helpful

Subscribe to our newsletter

You will receive our monthly newsletter and free access to Trip Premium.

Related Articles

Cluster Randomized Trials: Concepts

This blog summarizes the concepts of cluster randomization, and the logistical and statistical considerations while designing a cluster randomized controlled trial.

Expertise-based Randomized Controlled Trials

This blog summarizes the concepts of Expertise-based randomized controlled trials with a focus on the advantages and challenges associated with this type of study.

An introduction to different types of study design

Conducting successful research requires choosing the appropriate study design. This article describes the most common types of designs conducted by researchers.

A Practical Overview of Case-Control Studies in Clinical Practice

Affiliations.

• 1 Department of Quantitative Health Sciences, Lerner Research Institute, Cleveland Clinic, Cleveland, OH; Center for Surgery and Public Health, Brigham and Women's Hospital, Harvard Medical School, Boston, MA. Electronic address: [email protected].
• 2 Department of Quantitative Health Sciences, Lerner Research Institute, Cleveland Clinic, Cleveland, OH; Department of Population and Quantitative Health Sciences, Case Western Reserve University, School of Medicine, Cleveland, OH.
• 3 Department of Statistics, University of Missouri, Columbia, MO.
• PMID: 32658653
• DOI: 10.1016/j.chest.2020.03.009

Case-control studies are one of the major observational study designs for performing clinical research. The advantages of these study designs over other study designs are that they are relatively quick to perform, economical, and easy to design and implement. Case-control studies are particularly appropriate for studying disease outbreaks, rare diseases, or outcomes of interest. This article describes several types of case-control designs, with simple graphical displays to help understand their differences. Study design considerations are reviewed, including sample size, power, and measures associated with risk factors for clinical outcomes. Finally, we discuss the advantages and disadvantages of case-control studies and provide a checklist for authors and a framework of considerations to guide reviewers' comments.

Keywords: OR; case-cohort; case-crossover; matching; nested case-control; relative risk.

Copyright © 2020 American College of Chest Physicians. Published by Elsevier Inc. All rights reserved.

Publication types

• Case-Control Studies*
• Guidelines as Topic
• Research Design / standards
• Research Design / statistics & numerical data*

Study Design 101: Case Control Study

• Case Report
• Case Control Study
• Cohort Study
• Randomized Controlled Trial
• Practice Guideline
• Systematic Review
• Meta-Analysis
• Helpful Formulas
• Finding Specific Study Types

A study that compares patients who have a disease or outcome of interest (cases) with patients who do not have the disease or outcome (controls), and looks back retrospectively to compare how frequently the exposure to a risk factor is present in each group to determine the relationship between the risk factor and the disease.

Case control studies are observational because no intervention is attempted and no attempt is made to alter the course of the disease. The goal is to retrospectively determine the exposure to the risk factor of interest from each of the two groups of individuals: cases and controls. These studies are designed to estimate odds.

Case control studies are also known as "retrospective studies" and "case-referent studies."

• Good for studying rare conditions or diseases
• Less time needed to conduct the study because the condition or disease has already occurred
• Lets you simultaneously look at multiple risk factors
• Useful as initial studies to establish an association
• Can answer questions that could not be answered through other study designs

Disadvantages

• Retrospective studies have more problems with data quality because they rely on memory and people with a condition will be more motivated to recall risk factors (also called recall bias).
• Not good for evaluating diagnostic tests because it's already clear that the cases have the condition and the controls do not
• It can be difficult to find a suitable control group

Design pitfalls to look out for

Care should be taken to avoid confounding, which arises when an exposure and an outcome are both strongly associated with a third variable. Controls should be subjects who might have been cases in the study but are selected independent of the exposure. Cases and controls should also not be "over-matched."

Is the control group appropriate for the population? Does the study use matching or pairing appropriately to avoid the effects of a confounding variable? Does it use appropriate inclusion and exclusion criteria?

Fictitious Example

There is a suspicion that zinc oxide, the white non-absorbent sunscreen traditionally worn by lifeguards is more effective at preventing sunburns that lead to skin cancer than absorbent sunscreen lotions. A case-control study was conducted to investigate if exposure to zinc oxide is a more effective skin cancer prevention measure. The study involved comparing a group of former lifeguards that had developed cancer on their cheeks and noses (cases) to a group of lifeguards without this type of cancer (controls) and assess their prior exposure to zinc oxide or absorbent sunscreen lotions.

This study would be retrospective in that the former lifeguards would be asked to recall which type of sunscreen they used on their face and approximately how often. This could be either a matched or unmatched study, but efforts would need to be made to ensure that the former lifeguards are of the same average age, and lifeguarded for a similar number of seasons and amount of time per season.

Real-life Examples

Boubekri, M., Cheung, I., Reid, K., Wang, C., & Zee, P. (2014). Impact of windows and daylight exposure on overall health and sleep quality of office workers: a case-control pilot study. Journal of Clinical Sleep Medicine : JCSM : Official Publication of the American Academy of Sleep Medicine, 10 (6), 603-611. https://doi.org/10.5664/jcsm.3780

This pilot study explored the impact of exposure to daylight on the health of office workers (measuring well-being and sleep quality subjectively, and light exposure, activity level and sleep-wake patterns via actigraphy). Individuals with windows in their workplaces had more light exposure, longer sleep duration, and more physical activity. They also reported a better scores in the areas of vitality and role limitations due to physical problems, better sleep quality and less sleep disturbances.

Togha, M., Razeghi Jahromi, S., Ghorbani, Z., Martami, F., & Seifishahpar, M. (2018). Serum Vitamin D Status in a Group of Migraine Patients Compared With Healthy Controls: A Case-Control Study. Headache, 58 (10), 1530-1540. https://doi.org/10.1111/head.13423

This case-control study compared serum vitamin D levels in individuals who experience migraine headaches with their matched controls. Studied over a period of thirty days, individuals with higher levels of serum Vitamin D was associated with lower odds of migraine headache.

Related Formulas

• Odds ratio in an unmatched study
• Odds ratio in a matched study

Related Terms

A patient with the disease or outcome of interest.

Confounding

When an exposure and an outcome are both strongly associated with a third variable.

A patient who does not have the disease or outcome.

Matched Design

Each case is matched individually with a control according to certain characteristics such as age and gender. It is important to remember that the concordant pairs (pairs in which the case and control are either both exposed or both not exposed) tell us nothing about the risk of exposure separately for cases or controls.

Observed Assignment

The method of assignment of individuals to study and control groups in observational studies when the investigator does not intervene to perform the assignment.

Unmatched Design

The controls are a sample from a suitable non-affected population.

Now test yourself!

1. Case Control Studies are prospective in that they follow the cases and controls over time and observe what occurs.

a) True b) False

2. Which of the following is an advantage of Case Control Studies?

a) They can simultaneously look at multiple risk factors. b) They are useful to initially establish an association between a risk factor and a disease or outcome. c) They take less time to complete because the condition or disease has already occurred. d) b and c only e) a, b, and c

Evidence Pyramid - Navigation

• Meta- Analysis
• Case Reports
• << Previous: Case Report
• Next: Cohort Study >>

• Last Updated: Sep 25, 2023 10:59 AM
• URL: https://guides.himmelfarb.gwu.edu/studydesign101

• Himmelfarb Intranet
• Privacy Notice
• Terms of Use
• GW is committed to digital accessibility. If you experience a barrier that affects your ability to access content on this page, let us know via the Accessibility Feedback Form .
• Himmelfarb Health Sciences Library
• 2300 Eye St., NW, Washington, DC 20037
• Phone: (202) 994-2850
• [email protected]
• https://himmelfarb.gwu.edu

Quantitative study designs: Case Control

Quantitative study designs.

• Introduction
• Cohort Studies
• Randomised Controlled Trial

Case Control

• Cross-Sectional Studies
• Study Designs Home

In a Case-Control study there are two groups of people: one has a health issue (Case group), and this group is “matched” to a Control group without the health issue based on characteristics like age, gender, occupation. In this study type, we can look back in the patient’s histories to look for exposure to risk factors that are common to the Case group, but not the Control group. It was a case-control study that demonstrated a link between carcinoma of the lung and smoking tobacco . These studies estimate the odds between the exposure and the health outcome, however they cannot prove causality. Case-Control studies might also be referred to as retrospective or case-referent studies. 

Stages of a Case-Control study

This diagram represents taking both the case (disease) and the control (no disease) groups and looking back at their histories to determine their exposure to possible contributing factors.  The researchers then determine the likelihood of those factors contributing to the disease.

(FOR ACCESSIBILITY: A case control study is likely to show that most, but not all exposed people end up with the health issue, and some unexposed people may also develop the health issue)

Which Clinical Questions does Case-Control best answer?

Case-Control studies are best used for Prognosis questions.

For example: Do anticholinergic drugs increase the risk of dementia in later life? (See BMJ Case-Control study Anticholinergic drugs and risk of dementia: case-control study )

What are the advantages and disadvantages to consider when using Case-Control?

* Confounding occurs when the elements of the study design invalidate the result. It is usually unintentional. It is important to avoid confounding, which can happen in a few ways within Case-Control studies. This explains why it is lower in the hierarchy of evidence, superior only to Case Studies.

What does a strong Case-Control study look like?

A strong study will have:

• Well-matched controls, similar background without being so similar that they are likely to end up with the same health issue (this can be easier said than done since the risk factors are unknown). 
• Detailed medical histories are available, reducing the emphasis on a patient’s unreliable recall of their potential exposures. 

What are the pitfalls to look for?

• Poorly matched or over-matched controls.  Poorly matched means that not enough factors are similar between the Case and Control. E.g. age, gender, geography. Over-matched conversely means that so many things match (age, occupation, geography, health habits) that in all likelihood the Control group will also end up with the same health issue! Either of these situations could cause the study to become ineffective. 
• Selection bias: Selection of Controls is biased. E.g. All Controls are in the hospital, so they’re likely already sick, they’re not a true sample of the wider population. 
• Cases include persons showing early symptoms who never ended up having the illness. 

Critical appraisal tools 

To assist with critically appraising case control studies there are some tools / checklists you can use.

CASP - Case Control Checklist

JBI – Critical appraisal checklist for case control studies

CEBMA – Centre for Evidence Based Management  – Critical appraisal questions (focus on leadership and management)

STROBE - Observational Studies checklists includes Case control

SIGN - Case-Control Studies Checklist

Real World Examples

Smoking and carcinoma of the lung; preliminary report

• Doll, R., & Hill, A. B. (1950). Smoking and carcinoma of the lung; preliminary report.  British Medical Journal ,  2 (4682), 739–748. Retrieved from  https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2038856/
• Key Case-Control study linking tobacco smoking with lung cancer
• Notes a marked increase in incidence of Lung Cancer disproportionate to population growth.
• 20 London Hospitals contributed current Cases of lung, stomach, colon and rectum cancer via admissions, house-physician and radiotherapy diagnosis, non-cancer Controls were selected at each hospital of the same-sex and within 5 year age group of each.
• 1732 Cases and 743 Controls were interviewed for social class, gender, age, exposure to urban pollution, occupation and smoking habits.
• It was found that continued smoking from a younger age and smoking a greater number of cigarettes correlated with incidence of lung cancer.

Anticholinergic drugs and risk of dementia: case-control study

• Richardson, K., Fox, C., Maidment, I., Steel, N., Loke, Y. K., Arthur, A., . . . Savva, G. M. (2018). Anticholinergic drugs and risk of dementia: case-control study. BMJ , 361, k1315. Retrieved from  http://www.bmj.com/content/361/bmj.k1315.abstract .
• A recent study linking the duration and level of exposure to Anticholinergic drugs and subsequent onset of dementia.
• Anticholinergic Cognitive Burden (ACB) was estimated in various drugs, the higher the exposure (measured as the ACB score) the greater likeliness of onset of dementia later in life.
• Antidepressant, urological, and antiparkinson drugs with an ACB score of 3 increased the risk of dementia. Gastrointestinal drugs with an ACB score of 3 were not strongly linked with onset of dementia.
• Tricyclic antidepressants such as Amitriptyline have an ACB score of 3 and are an example of a common area of concern.

Omega-3 deficiency associated with perinatal depression: Case-Control study 

• Rees, A.-M., Austin, M.-P., Owen, C., & Parker, G. (2009). Omega-3 deficiency associated with perinatal depression: Case control study. Psychiatry Research , 166(2), 254-259. Retrieved from  http://www.sciencedirect.com/science/article/pii/S0165178107004398 .
• During pregnancy women lose Omega-3 polyunsaturated fatty acids to the developing foetus.
• There is a known link between Omgea-3 depletion and depression
• Sixteen depressed and 22 non-depressed women were recruited during their third trimester
• High levels of Omega-3 were associated with significantly lower levels of depression.
• Women with low levels of Omega-3 were six times more likely to be depressed during pregnancy.

References and Further Reading

Doll, R., & Hill, A. B. (1950). Smoking and carcinoma of the lung; preliminary report. British Medical Journal, 2(4682), 739–748. Retrieved from https://www.ncbi.nlm.nih.gov/pmc/articles/PMC2038856/

Greenhalgh, Trisha. How to Read a Paper: the Basics of Evidence-Based Medicine, John Wiley & Sons, Incorporated, 2014. ProQuest Ebook Central, http://ebookcentral.proquest.com/lib/deakin/detail.action?docID=1642418 .

Himmelfarb Health Sciences Library. (2019). Study Design 101: Case-Control Study. Retrieved from https://himmelfarb.gwu.edu/tutorials/studydesign101/casecontrols.cfm   

Hoffmann, T., Bennett, S., & Del Mar, C. (2017). Evidence-Based Practice Across the Health Professions (Third edition. ed.): Elsevier. 

Lewallen, S., & Courtright, P. (1998). Epidemiology in practice: case-control studies. Community Eye Health, 11(28), 57.  https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1706071/  

Pelham, B. W. a., & Blanton, H. (2013). Conducting research in psychology : measuring the weight of smoke /Brett W. Pelham, Hart Blanton (Fourth edition. ed.): Wadsworth Cengage Learning. 

Rees, A.-M., Austin, M.-P., Owen, C., & Parker, G. (2009). Omega-3 deficiency associated with perinatal depression: Case control study. Psychiatry Research, 166(2), 254-259. Retrieved from http://www.sciencedirect.com/science/article/pii/S0165178107004398

Richardson, K., Fox, C., Maidment, I., Steel, N., Loke, Y. K., Arthur, A., … Savva, G. M. (2018). Anticholinergic drugs and risk of dementia: case-control study. BMJ, 361, k1315. Retrieved from http://www.bmj.com/content/361/bmj.k1315.abstract

Statistics How To. (2019). Case-Control Study: Definition, Real Life Examples. Retrieved from https://www.statisticshowto.com/case-control-study/  

• << Previous: Randomised Controlled Trial
• Next: Cross-Sectional Studies >>
• Last Updated: May 15, 2024 11:37 AM
• URL: https://deakin.libguides.com/quantitative-study-designs

Case Study Research Method in Psychology

Saul Mcleod, PhD

Editor-in-Chief for Simply Psychology

BSc (Hons) Psychology, MRes, PhD, University of Manchester

Saul Mcleod, PhD., is a qualified psychology teacher with over 18 years of experience in further and higher education. He has been published in peer-reviewed journals, including the Journal of Clinical Psychology.

Learn about our Editorial Process

Olivia Guy-Evans, MSc

Associate Editor for Simply Psychology

BSc (Hons) Psychology, MSc Psychology of Education

Olivia Guy-Evans is a writer and associate editor for Simply Psychology. She has previously worked in healthcare and educational sectors.

On This Page:

Case studies are in-depth investigations of a person, group, event, or community. Typically, data is gathered from various sources using several methods (e.g., observations & interviews).

The case study research method originated in clinical medicine (the case history, i.e., the patient’s personal history). In psychology, case studies are often confined to the study of a particular individual.

The information is mainly biographical and relates to events in the individual’s past (i.e., retrospective), as well as to significant events that are currently occurring in his or her everyday life.

The case study is not a research method, but researchers select methods of data collection and analysis that will generate material suitable for case studies.

Freud (1909a, 1909b) conducted very detailed investigations into the private lives of his patients in an attempt to both understand and help them overcome their illnesses.

This makes it clear that the case study is a method that should only be used by a psychologist, therapist, or psychiatrist, i.e., someone with a professional qualification.

There is an ethical issue of competence. Only someone qualified to diagnose and treat a person can conduct a formal case study relating to atypical (i.e., abnormal) behavior or atypical development.

 Famous Case Studies

• Anna O – One of the most famous case studies, documenting psychoanalyst Josef Breuer’s treatment of “Anna O” (real name Bertha Pappenheim) for hysteria in the late 1800s using early psychoanalytic theory.
• Little Hans – A child psychoanalysis case study published by Sigmund Freud in 1909 analyzing his five-year-old patient Herbert Graf’s house phobia as related to the Oedipus complex.
• Bruce/Brenda – Gender identity case of the boy (Bruce) whose botched circumcision led psychologist John Money to advise gender reassignment and raise him as a girl (Brenda) in the 1960s.
• Genie Wiley – Linguistics/psychological development case of the victim of extreme isolation abuse who was studied in 1970s California for effects of early language deprivation on acquiring speech later in life.
• Phineas Gage – One of the most famous neuropsychology case studies analyzes personality changes in railroad worker Phineas Gage after an 1848 brain injury involving a tamping iron piercing his skull.

Clinical Case Studies

• Studying the effectiveness of psychotherapy approaches with an individual patient
• Assessing and treating mental illnesses like depression, anxiety disorders, PTSD
• Neuropsychological cases investigating brain injuries or disorders

Child Psychology Case Studies

• Studying psychological development from birth through adolescence
• Cases of learning disabilities, autism spectrum disorders, ADHD
• Effects of trauma, abuse, deprivation on development

Types of Case Studies

• Explanatory case studies : Used to explore causation in order to find underlying principles. Helpful for doing qualitative analysis to explain presumed causal links.
• Exploratory case studies : Used to explore situations where an intervention being evaluated has no clear set of outcomes. It helps define questions and hypotheses for future research.
• Descriptive case studies : Describe an intervention or phenomenon and the real-life context in which it occurred. It is helpful for illustrating certain topics within an evaluation.
• Multiple-case studies : Used to explore differences between cases and replicate findings across cases. Helpful for comparing and contrasting specific cases.
• Intrinsic : Used to gain a better understanding of a particular case. Helpful for capturing the complexity of a single case.
• Collective : Used to explore a general phenomenon using multiple case studies. Helpful for jointly studying a group of cases in order to inquire into the phenomenon.

Where Do You Find Data for a Case Study?

There are several places to find data for a case study. The key is to gather data from multiple sources to get a complete picture of the case and corroborate facts or findings through triangulation of evidence. Most of this information is likely qualitative (i.e., verbal description rather than measurement), but the psychologist might also collect numerical data.

1. Primary sources

• Interviews – Interviewing key people related to the case to get their perspectives and insights. The interview is an extremely effective procedure for obtaining information about an individual, and it may be used to collect comments from the person’s friends, parents, employer, workmates, and others who have a good knowledge of the person, as well as to obtain facts from the person him or herself.
• Observations – Observing behaviors, interactions, processes, etc., related to the case as they unfold in real-time.
• Documents & Records – Reviewing private documents, diaries, public records, correspondence, meeting minutes, etc., relevant to the case.

2. Secondary sources

• News/Media – News coverage of events related to the case study.
• Academic articles – Journal articles, dissertations etc. that discuss the case.
• Government reports – Official data and records related to the case context.
• Books/films – Books, documentaries or films discussing the case.

3. Archival records

Searching historical archives, museum collections and databases to find relevant documents, visual/audio records related to the case history and context.

Public archives like newspapers, organizational records, photographic collections could all include potentially relevant pieces of information to shed light on attitudes, cultural perspectives, common practices and historical contexts related to psychology.

4. Organizational records

Organizational records offer the advantage of often having large datasets collected over time that can reveal or confirm psychological insights.

Of course, privacy and ethical concerns regarding confidential data must be navigated carefully.

However, with proper protocols, organizational records can provide invaluable context and empirical depth to qualitative case studies exploring the intersection of psychology and organizations.

• Organizational/industrial psychology research : Organizational records like employee surveys, turnover/retention data, policies, incident reports etc. may provide insight into topics like job satisfaction, workplace culture and dynamics, leadership issues, employee behaviors etc.
• Clinical psychology : Therapists/hospitals may grant access to anonymized medical records to study aspects like assessments, diagnoses, treatment plans etc. This could shed light on clinical practices.
• School psychology : Studies could utilize anonymized student records like test scores, grades, disciplinary issues, and counseling referrals to study child development, learning barriers, effectiveness of support programs, and more.

How do I Write a Case Study in Psychology?

Follow specified case study guidelines provided by a journal or your psychology tutor. General components of clinical case studies include: background, symptoms, assessments, diagnosis, treatment, and outcomes. Interpreting the information means the researcher decides what to include or leave out. A good case study should always clarify which information is the factual description and which is an inference or the researcher’s opinion.

1. Introduction

• Provide background on the case context and why it is of interest, presenting background information like demographics, relevant history, and presenting problem.
• Compare briefly to similar published cases if applicable. Clearly state the focus/importance of the case.

2. Case Presentation

• Describe the presenting problem in detail, including symptoms, duration,and impact on daily life.
• Include client demographics like age and gender, information about social relationships, and mental health history.
• Describe all physical, emotional, and/or sensory symptoms reported by the client.
• Use patient quotes to describe the initial complaint verbatim. Follow with full-sentence summaries of relevant history details gathered, including key components that led to a working diagnosis.
• Summarize clinical exam results, namely orthopedic/neurological tests, imaging, lab tests, etc. Note actual results rather than subjective conclusions. Provide images if clearly reproducible/anonymized.
• Clearly state the working diagnosis or clinical impression before transitioning to management.

3. Management and Outcome

• Indicate the total duration of care and number of treatments given over what timeframe. Use specific names/descriptions for any therapies/interventions applied.
• Present the results of the intervention,including any quantitative or qualitative data collected.
• For outcomes, utilize visual analog scales for pain, medication usage logs, etc., if possible. Include patient self-reports of improvement/worsening of symptoms. Note the reason for discharge/end of care.

4. Discussion

• Analyze the case, exploring contributing factors, limitations of the study, and connections to existing research.
• Analyze the effectiveness of the intervention,considering factors like participant adherence, limitations of the study, and potential alternative explanations for the results.
• Identify any questions raised in the case analysis and relate insights to established theories and current research if applicable. Avoid definitive claims about physiological explanations.
• Offer clinical implications, and suggest future research directions.

5. Additional Items

• Thank specific assistants for writing support only. No patient acknowledgments.
• References should directly support any key claims or quotes included.
• Use tables/figures/images only if substantially informative. Include permissions and legends/explanatory notes.
• Provides detailed (rich qualitative) information.
• Provides insight for further research.
• Permitting investigation of otherwise impractical (or unethical) situations.

Case studies allow a researcher to investigate a topic in far more detail than might be possible if they were trying to deal with a large number of research participants (nomothetic approach) with the aim of ‘averaging’.

Because of their in-depth, multi-sided approach, case studies often shed light on aspects of human thinking and behavior that would be unethical or impractical to study in other ways.

Research that only looks into the measurable aspects of human behavior is not likely to give us insights into the subjective dimension of experience, which is important to psychoanalytic and humanistic psychologists.

Case studies are often used in exploratory research. They can help us generate new ideas (that might be tested by other methods). They are an important way of illustrating theories and can help show how different aspects of a person’s life are related to each other.

The method is, therefore, important for psychologists who adopt a holistic point of view (i.e., humanistic psychologists ).

Limitations

• Lacking scientific rigor and providing little basis for generalization of results to the wider population.
• Researchers’ own subjective feelings may influence the case study (researcher bias).
• Difficult to replicate.
• Time-consuming and expensive.
• The volume of data, together with the time restrictions in place, impacted the depth of analysis that was possible within the available resources.

Because a case study deals with only one person/event/group, we can never be sure if the case study investigated is representative of the wider body of “similar” instances. This means the conclusions drawn from a particular case may not be transferable to other settings.

Because case studies are based on the analysis of qualitative (i.e., descriptive) data , a lot depends on the psychologist’s interpretation of the information she has acquired.

This means that there is a lot of scope for Anna O , and it could be that the subjective opinions of the psychologist intrude in the assessment of what the data means.

For example, Freud has been criticized for producing case studies in which the information was sometimes distorted to fit particular behavioral theories (e.g., Little Hans ).

This is also true of Money’s interpretation of the Bruce/Brenda case study (Diamond, 1997) when he ignored evidence that went against his theory.

Breuer, J., & Freud, S. (1895).  Studies on hysteria . Standard Edition 2: London.

Curtiss, S. (1981). Genie: The case of a modern wild child .

Diamond, M., & Sigmundson, K. (1997). Sex Reassignment at Birth: Long-term Review and Clinical Implications. Archives of Pediatrics & Adolescent Medicine , 151(3), 298-304

Freud, S. (1909a). Analysis of a phobia of a five year old boy. In The Pelican Freud Library (1977), Vol 8, Case Histories 1, pages 169-306

Freud, S. (1909b). Bemerkungen über einen Fall von Zwangsneurose (Der “Rattenmann”). Jb. psychoanal. psychopathol. Forsch ., I, p. 357-421; GW, VII, p. 379-463; Notes upon a case of obsessional neurosis, SE , 10: 151-318.

Harlow J. M. (1848). Passage of an iron rod through the head.  Boston Medical and Surgical Journal, 39 , 389–393.

Harlow, J. M. (1868).  Recovery from the Passage of an Iron Bar through the Head .  Publications of the Massachusetts Medical Society. 2  (3), 327-347.

Money, J., & Ehrhardt, A. A. (1972).  Man & Woman, Boy & Girl : The Differentiation and Dimorphism of Gender Identity from Conception to Maturity. Baltimore, Maryland: Johns Hopkins University Press.

Money, J., & Tucker, P. (1975). Sexual signatures: On being a man or a woman.

Further Information

• Case Study Approach
• Case Study Method
• Enhancing the Quality of Case Studies in Health Services Research
• “We do things together” A case study of “couplehood” in dementia
• Using mixed methods for evaluating an integrative approach to cancer care: a case study

Related Articles

Research Methodology

Qualitative Data Coding

What Is a Focus Group?

Cross-Cultural Research Methodology In Psychology

What Is Internal Validity In Research?

Research Methodology , Statistics

What Is Face Validity In Research? Importance & How To Measure

Criterion Validity: Definition & Examples

• Download PDF
• Share X Facebook Email LinkedIn
• Permissions

Case-Control Studies : Using “Real-world” Evidence to Assess Association

• 1 Office of Biostatistics and Epidemiology, Center for Biologics Evaluation and Research (CBER), US Food and Drug Administration, Silver Spring, Maryland
• JAMA Guide to Statistics and Methods Mediation Analysis Hopin Lee, PhD; Robert D. Herbert, PhD; James H. McAuley, PhD JAMA
• Comment & Response Case-Control Studies Tony Blakely, PhD; Neil Pearce, PhD; John Lynch, PhD JAMA
• Comment & Response Case-Control Studies—Reply Telba Z. Irony, PhD JAMA
• Original Investigation Cardiovascular Risk and Inhaled Long-Acting Bronchodilators Meng-Ting Wang, PhD; Jun-Ting Liou, MD; Chen Wei Lin, BS; Chen-Liang Tsai, MD; Yun-Han Wang, MS, BPharm; Yu-Juei Hsu, MD, PhD; Jyun-Heng Lai, MS, BPharm JAMA Internal Medicine

Associations between patient characteristics or treatments received and clinical outcomes are often first described using observational data, such as data arising through usual clinical care without the experimental assignment of treatments that occurs in a randomized clinical trial (RCT). These data based on usual clinical care are referred to by some as “real-world” data. A key strategy for efficiently finding such associations is to use a case-control study. 1 In a recent issue of JAMA Internal Medicine , Wang et al 2 assessed the association between cardiovascular disease (CVD) and use of inhaled long-acting β 2 -agonists (LABAs) or long-acting antimuscarinic antagonists (LAMAs) in chronic obstructive pulmonary disease (COPD), utilizing a nested case-control study.

Read More About

Irony TZ. Case-Control Studies : Using “Real-world” Evidence to Assess Association . JAMA. 2018;320(10):1027–1028. doi:10.1001/jama.2018.12115

Manage citations:

© 2024

Artificial Intelligence Resource Center

Cardiology in JAMA : Read the Latest

Browse and subscribe to JAMA Network podcasts!

Others Also Liked

Select your interests.

Customize your JAMA Network experience by selecting one or more topics from the list below.

• Academic Medicine
• Acid Base, Electrolytes, Fluids
• Allergy and Clinical Immunology
• American Indian or Alaska Natives
• Anesthesiology
• Anticoagulation
• Art and Images in Psychiatry
• Artificial Intelligence
• Assisted Reproduction
• Bleeding and Transfusion
• Caring for the Critically Ill Patient
• Challenges in Clinical Electrocardiography
• Climate and Health
• Climate Change
• Clinical Challenge
• Clinical Decision Support
• Clinical Implications of Basic Neuroscience
• Clinical Pharmacy and Pharmacology
• Complementary and Alternative Medicine
• Consensus Statements
• Coronavirus (COVID-19)
• Critical Care Medicine
• Cultural Competency
• Dental Medicine
• Dermatology
• Diabetes and Endocrinology
• Diagnostic Test Interpretation
• Drug Development
• Electronic Health Records
• Emergency Medicine
• End of Life, Hospice, Palliative Care
• Environmental Health
• Equity, Diversity, and Inclusion
• Facial Plastic Surgery
• Gastroenterology and Hepatology
• Genetics and Genomics
• Genomics and Precision Health
• Global Health
• Guide to Statistics and Methods
• Hair Disorders
• Health Care Delivery Models
• Health Care Economics, Insurance, Payment
• Health Care Quality
• Health Care Reform
• Health Care Safety
• Health Care Workforce
• Health Disparities
• Health Inequities
• Health Policy
• Health Systems Science
• History of Medicine
• Hypertension
• Images in Neurology
• Implementation Science
• Infectious Diseases
• Innovations in Health Care Delivery
• JAMA Infographic
• Law and Medicine
• Leading Change
• Less is More
• LGBTQIA Medicine
• Lifestyle Behaviors
• Medical Coding
• Medical Devices and Equipment
• Medical Education
• Medical Education and Training
• Medical Journals and Publishing
• Mobile Health and Telemedicine
• Narrative Medicine
• Neuroscience and Psychiatry
• Notable Notes
• Nutrition, Obesity, Exercise
• Obstetrics and Gynecology
• Occupational Health
• Ophthalmology
• Orthopedics
• Otolaryngology
• Pain Medicine
• Palliative Care
• Pathology and Laboratory Medicine
• Patient Care
• Patient Information
• Performance Improvement
• Performance Measures
• Perioperative Care and Consultation
• Pharmacoeconomics
• Pharmacoepidemiology
• Pharmacogenetics
• Pharmacy and Clinical Pharmacology
• Physical Medicine and Rehabilitation
• Physical Therapy
• Physician Leadership
• Population Health
• Primary Care
• Professional Well-being
• Professionalism
• Psychiatry and Behavioral Health
• Public Health
• Pulmonary Medicine
• Regulatory Agencies
• Reproductive Health
• Research, Methods, Statistics
• Resuscitation
• Rheumatology
• Risk Management
• Scientific Discovery and the Future of Medicine
• Shared Decision Making and Communication
• Sleep Medicine
• Sports Medicine
• Stem Cell Transplantation
• Substance Use and Addiction Medicine
• Surgical Innovation
• Surgical Pearls
• Teachable Moment
• Technology and Finance
• The Art of JAMA
• The Arts and Medicine
• The Rational Clinical Examination
• Tobacco and e-Cigarettes
• Translational Medicine
• Trauma and Injury
• Treatment Adherence
• Ultrasonography
• Users' Guide to the Medical Literature
• Vaccination
• Venous Thromboembolism
• Veterans Health
• Women's Health
• Workflow and Process
• Wound Care, Infection, Healing
• Register for email alerts with links to free full-text articles
• Access PDFs of free articles
• Manage your interests
• Save searches and receive search alerts

• Chapter 8. Case-control and cross sectional studies

Case-control studies

Selection of cases, selection of controls, ascertainment of exposure, cross sectional studies.

• Chapter 1. What is epidemiology?
• Chapter 2. Quantifying disease in populations
• Chapter 3. Comparing disease rates
• Chapter 4. Measurement error and bias
• Chapter 5. Planning and conducting a survey
• Chapter 6. Ecological studies
• Chapter 7. Longitudinal studies
• Chapter 9. Experimental studies
• Chapter 10. Screening
• Chapter 11. Outbreaks of disease
• Chapter 12. Reading epidemiological reports
• Chapter 13. Further reading

Follow us on

Content links.

• Collections
• Health in South Asia
• Women’s, children’s & adolescents’ health
• News and views
• BMJ Opinion
• Rapid responses
• Editorial staff
• BMJ in the USA
• BMJ in South Asia
• Submit your paper
• BMA members
• Subscribers
• Advertisers and sponsors

Explore BMJ

• Our company
• BMJ Careers
• BMJ Learning
• BMJ Masterclasses
• BMJ Journals
• BMJ Student
• Academic edition of The BMJ
• BMJ Best Practice
• The BMJ Awards
• Email alerts
• Activate subscription

Information

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here .

Loading metrics

Open Access

Peer-reviewed

Research Article

The effects of person-centred active rehabilitation on symptoms of suspected Chronic Traumatic Encephalopathy: A mixed-methods single case design

Roles Conceptualization, Data curation, Formal analysis, Investigation, Methodology, Writing – original draft, Writing – review & editing

Affiliation Department of Health Professions, Faculty of Health and Education, Manchester Metropolitan University, Manchester, United Kingdom

Roles Conceptualization, Formal analysis, Methodology, Supervision, Writing – review & editing

Roles Conceptualization, Formal analysis, Methodology, Supervision

Affiliation Department of Psychology, Faculty of Health and Education, Manchester Metropolitan University, Manchester, United Kingdom

Roles Conceptualization, Formal analysis, Methodology, Supervision, Writing – original draft, Writing – review & editing

* E-mail: [email protected]

• Rachael Hearn, 
• James Selfe, 
• Maria I. Cordero, 
• Nick Dobbin

• Published: May 30, 2024
• https://doi.org/10.1371/journal.pone.0302260
• Reader Comments

The objective was to investigate the effectiveness of a person-centred active rehabilitation programme on symptoms associated with suspected Chronic Traumatic Encephalopathy (CTE). This was accomplished by (1) assessing the effect that a person-centred active rehabilitation programme had on participant symptoms, and (2) exploring how temporal contextual factors affected the participants’ experience with, and perceived effectiveness of, the active rehabilitation programme.

A twelve-month mixed-methods single case experimental research design was used with six cases (participants). Individual cases were involved in a 51-week study period including an initial interview and three-week baseline phase. Cases were then randomly allocated to one of two n-of-1 study designs (i.e., A-B, B-A, B-A, A-B or B-A, A-B, A-B, B-A) where A and B represent a non-intervention and intervention phase, respectively. Interviews were conducted regularly throughout the study whilst outcome measures were assessed at each follow-up. Analysis of the data included visual, statistical, and qualitative analysis.

Visual and statistical analysis of cognitive and executive function, and mindful attention, demonstrated trivial-to-large effects with the summary reflecting positive or unclear results. A mixed picture was observed for mood and behaviour with effects considered trivial-to-large, and the summary demonstrating positive, unclear and negative effects. Qualitative analysis indicated a perceived improvement in outcome measures such as memory, attention, anxiety, and emotional control despite mixed quantitative findings whilst a clear impact of contextual factors, such as COVID-19, the political atmosphere, exercise tolerance, programme progression, and motivation were evident during the intervention.

Conclusions

This study has provided primary-level evidence to suggest active rehabilitation as a potential intervention for the management of suspected CTE symptoms. This study has also demonstrated the benefit of a person-centred approach to both clinical research and practice, particularly by considering contextual factors for a better understanding of an intervention effect.

Citation: Hearn R, Selfe J, Cordero MI, Dobbin N (2024) The effects of person-centred active rehabilitation on symptoms of suspected Chronic Traumatic Encephalopathy: A mixed-methods single case design. PLoS ONE 19(5): e0302260. https://doi.org/10.1371/journal.pone.0302260

Editor: Simone Varrasi, University of Catania, ITALY

Received: December 21, 2023; Accepted: March 31, 2024; Published: May 30, 2024

Copyright: © 2024 Hearn et al. This is an open access article distributed under the terms of the Creative Commons Attribution License , which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: The data underlying the results presented in this study are available from Manchester Metropolitan University e-space: https://e-space.mmu.ac.uk/view/datasets/ .

Funding: The author(s) received no specific funding for this work.

Competing interests: The authors have declared that no competing interests exist.

Introduction

Chronic Traumatic Encephalopathy (CTE) was formally defined by the National Institute of Neurological Disorders and Stroke/National Institute of Biomedical Imaging and Bioengineering (NINDS/NIBIB) consensus panel in 2015. CTE is a neurodegenerative pathology defined by its unique, irregular pattern of tau protein accumulation around small blood vessels at the base of the cortical sulci [ 1 ]. While CTE has been known by many since at least 1928, the clinical profile has consistently been linked to an exposure to repetitive brain injury. CTE has now been diagnosed post-mortem in former American football, football (soccer), rugby, Australian rules football, ice hockey, baseball, and wrestling athletes, as well as military personnel and domestic abuse victims [ 1 – 3 ]. The inability to diagnose CTE pre-death has led to the development of a clinical profile termed Traumatic Encephalopathy Syndrome (TES). This clinical profile has enabled researchers to identify individuals with symptoms associated with suspected CTE allowing for early management and development of active rehabilitation or treatment options [ 2 , 4 – 6 ].

Currently, no evidence-based therapy has been developed to treat or manage symptoms associated with CTE to our knowledge. Cantu and Budson [ 4 ] provided the first expert review outlining potential lifestyle modifications and symptom management strategies for CTE. Recommendations included exercise, active rehabilitation, diet, cognitive rehabilitation, mood/behavioural therapy, occupational therapy, vestibular and motor therapy, and pharmacological therapy. This expert opinion has been reiterated by Fusco et al. [ 7 ] and Rossi et al. [ 8 ] in expert opinions or narrative reviews. Themes including cognitive and motor rehabilitation therapy, mindfulness, mood/behavioural therapy, occupational therapy, diet, exercise, and active rehabilitation were suggested to help manage neuropsychiatric symptoms of CTE [ 7 , 8 ]. Finally, an umbrella review [ 9 ] has reported the effect that active rehabilitation has on other tauopathies with symptoms associated with CTE. This review found that various forms of active rehabilitation had a positive effect (standardised mean difference ranging from 0.11 to 0.88) on symptoms of cognitive and motor function in populations diagnosed with Alzheimer’s and Parkinson’s disease.

Despite being an area of growing interest amongst researchers and clinicians, current evidence is largely limited to secondary level research. Whilst various forms of management strategies have been suggested (e.g., active rehabilitation), few have examined the efficacy of this. It is also important to consider the patient within a rehabilitation or management approach. A person-centred care approach allows for the management of patients with a unique set of symptoms which is likely to be particularly important when managing individuals with suspected CTE. Accordingly, the objective of this study was to investigate the effectiveness of person-centred active rehabilitation on symptoms of TES, providing the first primary level research exploring a potential management for CTE symptoms. This was accomplished by: (1) assessing the effect that a person-centred active rehabilitation programme had on participant symptoms suspected to be associated with the development of CTE, and (2) exploring how proximal (individual—interpersonal and intrapersonal) and distal (environmental—socio-economic, rural-urban differences and immigration background) factors affected the participants’ experience with, and perceived effectiveness of, the active rehabilitation programme through repeated interviews during the course of the intervention.

Materials and methods

A mixed-methods single case research (MMSCR) design was used for this study with an n-of-1 framework. The study was designed and reported in line with the Consolidated Standards of Reporting Trials (CONSORT) extension for N-of-1 trials [ 10 , 11 ]. Ethical approval was granted by the Faculty of Health, Psychology and Social Care Research Ethics and Governance Committee at Manchester Metropolitan University (ID: 11822).

Eligibility criteria

Individuals between the age 20 and 60 years who met the 2014 TES criteria [ 6 ], spoke/read English, and were at least one year retired from competitive sport were eligible for inclusion. The 2014 TES criteria was used as the updated criteria by Katz and colleagues [ 5 ] was not available at the start of the study. An age range of 20 to 60 years allowed for sufficient exposure to mTBI/contact sport while minimising the chance of concomitant neurodegenerative disease, presence of dementia, and other neurological disorders. Individuals diagnosed with dementia were excluded.

At any time during data collection, withdrawal could be explicitly expressed by the participant. Withdrawal was assumed if the participant (1) missed more than two follow-up interviews in a row, or (2) did not respond to at least two contact attempts made by the researcher seeking to schedule a follow-up interview. Withdrawal was also discussed if the participant expressed dissent with the study procedures. In the event of participant withdrawal, any data where a full data collection cycle had been completed (A-B matched pair) was included in the analysis.

Materials and procedures

General procedure..

Individual cases (participants) were involved for 51 weeks, with recruitment and study commencement occurring on a rolling basis starting in April 2020 and ending in June 2021. The study began with an initial interview, followed by a three-week baseline phase. Participants were then randomly allocated to one of two systematic counterbalanced n-of-1 study designs, the first being A-B, B-A, B-A, A-B and the second being B-A, A-B, A-B, B-A, where ‘A’ indicates a non-intervention phase and ‘B’ indicates an intervention phase. Each paired phase (A-B or B-A) lasted twelve weeks and consisted of six interviews which took place every two weeks. A schematic overview of the study is illustrated in Fig 1 .

• PPT PowerPoint slide
• PNG larger image
• TIFF original image

Note: A = non-intervention phase. B–intervention phase.

https://doi.org/10.1371/journal.pone.0302260.g001

Initial interview and screening assessments.

An initial interview was conducted to (1), screen the participant for study eligibility, ensuring they met clinical criteria for the presence of TES, and (2), to screen the participant for evidence of potential cognitive impairment, changes in mood/behaviour, or motor impairment associated with CTE which could be assessed during the three-week baseline period. To date, there is no established battery of assessments relevant to the population used in this study; thus, a pragmatic approach was adopted. The initial interview began with three core screening assessments to measure levels of cognitive function, mood/behavioural symptoms, and motor function ( S1 Table ). The initial meeting concluded with a semi-structured interview that sought to elicit further information related to the eligibility criteria and provided the participant with an opportunity to share information on relevant sporting history, medical history, family medical history, and any other further symptoms or concerns. In line with a person-centred care approach, participants also gave information regarding activity capabilities and preferences to inform the design of the active rehabilitation programme.

Baseline phase and follow-ups.

The three-week baseline period sought to establish the presence of measurable impairment. The assessments used were individualised for each case based on relevance and symptoms reported in the initial interview. Any symptom of TES had the potential for inclusion. A description of all assessments used in the study can be found in S2 Table . To reduce the burden on participants, a five-assessment limit was implemented. For those who did not reach the five-assessment limit, the Perceived Stress Scale (PSS) was included to provide further support to the contextual information gathered during the follow-up interviews.

During each follow-up, participants completed an online survey consisting of self-report symptom assessments. The online survey also gave open box to submit a daily activity log. This log allowed participants to report daily log of their physical activity as well as additional contextual information related to their symptoms and programme experience. Participants also took part in a semi-structured follow-up interview at each data collection point. The semi-structured interviews primarily sought to create a person-centred care environment, promoting factors such as understanding the participant as a person and encouraging involvement in the co-design of the active rehabilitation. The aim of these interviews was to understand i) how the presence of the person-centred active rehabilitation programme affected the symptoms of interest, ii) how the participant described their experience with the rehabilitation programme and prescription, iii) how proximal and distal factors may have influenced the participants symptom levels, and iv) how proximal and distal factors may have influenced their experience with or effect of the programme.

Intervention delivery.

The setting of the study was entirely online. Any outcome assessments used were recorded via Qualtrics XM (Qualtrics, Provo, Utah, USA). The intervention programme was distributed via email and accompanied with online tutorial videos.

During an intervention phase (B phase), participants completed one resistance training session and one cardiovascular session each week. During a non-intervention phase (A phase) the prescribed exercises were removed, but participants were allowed to continue with habitual activities. Care was taken to ensure programme prescriptions resulted in a greater training load during intervention phases relative to the non-intervention phases.

The training programme was tailored by mode, duration, and intensity based on participant needs, preferences, facilities, and strength/fitness levels as understood from the semi-structured interview. Intensity for the resistance training programmes was prescribed using a modified rating of perceived exertion (RPE) scale [ 12 ]. The intensity of the cardiovascular training programme was prescribed using the Borg 6–20 scale given its linear relationship with heart rate [ 12 ].

Data analysis

Quantitative analysis..

Visual analysis of the quantitative data followed a modified framework presented by Wolfe and colleagues [ 13 ], created in accordance with The What Works Clearinghouse (WWC) Single-Case Design Standards [ 14 ]. Initially, a trend in the data was established for each of the paired non-intervention and intervention phases using a split middle trend (SMT) line [ 15 ]. The SMT line was then used to predict outcome measures for the subsequent phase. The size of the effect was estimated using a modified point system [ 13 ] based on the following:

• Evidence of change in level, trend and/or variability = 1.0 point
• Change was immediate, there was less than 30% of data overlap, or there was evidence of consistency between phase-types (intervention/non-intervention) = 0.25 point

After summing the scores, 0–2 indicates unclear behavioural change; 3–4 a small behavioural change; 5–6 a moderate behavioural change; 7–8 a large behavioural change.

To support the visual analysis and future research (e.g., sample size, meta-analyses), within-case standardized mean difference (WC-SMD) and non-overlap of all pairs (NAP) with 95% confidence limits were calculated [ 16 ]. SMDs were classified as: <2.0 trivial; 0.20–0.50, small; 0.51–0.80, moderate; >0.80, large [ 17 ]. NAP was interpreted as a probability that a randomly selected datapoint in the intervention (B) phase was above or below (depending on if an increase or decrease is desirable) a randomly selected datapoint in the non-intervention (A) phase [ 16 ].

Qualitative analysis.

All interviews were transcribed verbatim . Transcriptions were then read to identify content related to the topics of interest based on deductive analysis a-priori . Additional points of interest that also emerged were included via inductive analysis. To maintain trustworthiness through confirmability, all authors were given a random selection of transcripts to analyse. Qualitative analysis followed the explanation building approach outlined by Yin [ 18 ]. The following propositions were initially determined:

• The presence of the person-centred active rehabilitation programme had a positive effect on the participant’s symptoms of interest.
• The needs and preferences of the participant regarding the rehabilitation mode and prescription were met.
• Proximal and distal factors influenced i) the participant’s reported symptom levels, ii) the participant’s experience with the active rehabilitation programme, and iii) the effect of the active rehabilitation programme.

While a total of twenty-four follow-up interviews were available, a pragmatic approach for qualitative analysis was adopted. The sample of interviews used in this study consisted of those that took place at the end of each A-B paired phase (e.g., A1.3, B2.3, B3.3, and A4.3) to provide a more global view of the participant perspective as it related to the phase (whether A or B) and proximal and distal factors. A summary of results is presented in a narrative and visual format.

Participant characteristics

Ten participants were recruited for the study. Two participants did not demonstrate a measurable impairment during the screening or baseline phase in one of the three core clinical features and therefore did not meet eligibility criteria. A further two participants withdrew from the study before they completed an entire A-B phase. In both instances, the participant did not respond to at least two contact attempts made by the primary author to schedule follow-up interviews. A reason for withdrawal was not stated by either participant. In line with the study protocol, their data was withdrawn from the study. A total of six participants completed all aspects and were included in the final analysis.

Participant characteristics, informed by the initial semi-structured interview and baseline phase, can be found in Table 1 . Results are presented using pseudonyms to maintain anonymity.

https://doi.org/10.1371/journal.pone.0302260.t001

Fig 2 illustrates a complete study timeline across all participants along with the sequence of intervention (B) and non-intervention (A) phases. This figure also provides key information on the COVID pandemic and the restrictions in place for each participant at the time of data collection.

https://doi.org/10.1371/journal.pone.0302260.g002

Fig 3 presents results from the PSS assessment.

https://doi.org/10.1371/journal.pone.0302260.g003

Participant activity levels

Participant were habitually active during the intervention engaging in various forms of physical activity including leisure, exercise, and sport. Table 2 presents a summary of the activities reported by participants during both intervention (B) and non-intervention (A) phases with bolded text being the prescribed activities.

https://doi.org/10.1371/journal.pone.0302260.t002

Quantitative analysis

Table 3 provides a summary of the six individual cases, including visual and statistical analysis, across each symptom of interest. Individual case results can be found in S3 – S8 Tables. The effect that active rehabilitation had on symptoms of motor function was not reported as no participants included in the study had any measurable motor impairments.

https://doi.org/10.1371/journal.pone.0302260.t003

Figs 4 and 5 provide visual results of outcome measures related to cognitive function. Effects on general cognitive function ( Fig 4 ) varied, with one participant (Niall) demonstrating a positive effect, one participant (Kristen) demonstrating a negative effect, and one participant (Simon) demonstrating an unclear effect. It should be noted that levels of general cognitive function consistently demonstrated an upward trend across intervention phases throughout the study for Simon, indicating a potential positive effect; however, statistical analysis supports an unclear effect. Visual and statistical analysis observed consistent overlap in all three cases as well. Levels of cognitive function were higher in the final phase for all three cases compared to those levels reported in the first study phase. Effects on mindful attention ( Fig 4 ) also varied, with one case demonstrating negative visual and statistical analysis (Simon) and one demonstrating a small visual but trivial statistical effect (Abel). High levels of variability throughout the study should be noted here along with consistent overlap between phases; therefore, it is difficult to determine the effect active rehabilitation had on levels of mindful attention.

https://doi.org/10.1371/journal.pone.0302260.g004

https://doi.org/10.1371/journal.pone.0302260.g005

Though the WC-SMD ranged from trivial to large, four of the six participants demonstrated a positive effect (Niall, Luigi, Abel, Simon) for measures of executive function ( Fig 5 ). Three of these participants (Niall, Luigi, Simon) also reported higher levels of executive function at the final phase of the study compared to the start of the study. Only Abel reported worse scores during the final phase of the study; however, a large WC-SMD (1.69) and NAP value of 0.79 suggests a positive effect of active rehabilitation overall. It should be noted that Kristen consistently demonstrated decreased variability of scores in non-intervention phases indicating more stable results for executive function in the absence of an active rehabilitation programme. This would suggest that the presence of an active rehabilitation programme had the potential to influence levels of outcome measures, albeit with a lack of statistical certainty. All cases demonstrated consistent overlap with visual analysis; however, NAP values in three cases (Niall, Abel, Simon) demonstrated a higher probability of increased executive function in intervention phases.

Figs 6 and 7 provide visual results of outcome measures related to mood and behaviour. The effect of active rehabilitation on outcome measures considered ‘core clinical features’ according to TES clinical criteria (depression, irritability, social isolation/loneliness) ( Fig 6 ) varied. One participant (Luigi) demonstrated a large positive effect on levels of depression, supported by both visual and statistical analysis; however, three participants (Kristen, Abel, Simon) demonstrated a small, negative effect on levels of depression. Interestingly, all participants aside from one (Simon) reported lower levels of depression during the final phase of the study compared to initial levels. There was no consistency of patterns in variability or trend, but overlap between phases was consistently present. NAP values also varied, indicating a variation in the likelihood of a randomly selected point taken during an intervention phase resulting in lower depression scores. Only one participant reported symptoms of loneliness and social isolation (Niall), and one participant reported symptoms of irritability (Luigi). In both instances, visual analysis indicated a positive effect. Levels of loneliness demonstrated a consistent downward trend during intervention phases despite increased variability. It should also be noted that Luigi demonstrated decreased variability for levels of irritability across intervention phases. A positive effect is further supported by improved levels reported during the final phase of the study compared to initial levels, as well as a general decline observed across the entire study; however, statistical analysis reports a trivial effect in both outcome measures.

https://doi.org/10.1371/journal.pone.0302260.g006

https://doi.org/10.1371/journal.pone.0302260.g007

The effect of active rehabilitation on outcome measures considered ‘supportive clinical features’ according to TES clinical criteria (anxiety, sleep quality/insomnia) ( Fig 7 ) also varied. One participant (Luigi) reported a small-to-moderate effect on levels of anxiety, supported by visual and statistical analysis; however, Kristen and Abel reported a moderate, negative effect. The effect of active rehabilitation on Gemma’s level of anxiety was unclear. Visual analysis demonstrated a small effect; however, statistical analysis reports the effect as trivial. Interestingly, all participants demonstrated reduced levels of anxiety during the final phase of the study compared to those levels reported during the initial phase; however, overlap was present between all phases across all cases suggesting a temporal pattern of change. Only Kristen reported NAP levels which could suggest a greater probability of lower levels of anxiety in non-intervention phases (NAP = 0.25). It should be noted that Gemma consistently demonstrated a downward trend in levels of anxiety in intervention phases. Only one participant reported symptoms of reduced sleep quality and insomnia (Kristen). Despite WC-SMD reporting a small negative effect, visual analysis reported a trivial effect. Further, NAP suggests the probability randomly selected data point for sleep score being lower during an intervention phase was 43%. It should be noted that Kristen’s sleep score was lower at the end of the study compared to those reported at the start of the study, indicating an improvement in symptoms.

Qualitative data

Fig 8 presents a visual summary of the topics emerged from the semi-structured interviews, including information relevant to the context of the study (8A), intervention experience (8B), and perception towards the effect of the intervention on symptoms (8C). Topics of discussion highlighted in green present information that may have contributed to a positive participant experience and generally included improved memory, coping skills emotional control and anxiety, the role of goal setting, and satisfaction with the active rehabilitation programme. Topics of discussion highlighted in red present a potentially negative experience. These include information such as COVID-19 and the political atmosphere as well as exercise tolerance, progression, and motivation. Topics of discussion highlighted in yellow presented points that were not clearly linked to a positive or negative experience (e.g., weather, unclear or neutral feelings, and preferences).

https://doi.org/10.1371/journal.pone.0302260.g008

The most obvious influence that emerged from the results which impacted every participant was the presence of the COVID-19 pandemic and subsequent lockdown restrictions. This seemed to influence Niall and Luigi the most. These two participants began the study at the very start of the pandemic where restrictions were the harshest and the levels of uncertainty were the highest. It seemed to affect Kristen the least, who lives in a quite secluded part of New York (state). The presence of the pandemic had both a direct and indirect effect on the success of the intervention. The pandemic influenced some outcomes of interest such as symptoms of anxiety and depression. Lockdown restrictions could also influence executive function as illustrated by Niall, who reported that motivation, attention, and concentration were disrupted. It should be noted that two participants contracted COVID-19 during the study (Kristen, Abel), resulting in further direct effects of the global pandemic. Indirectly, COVID-19 and subsequent lockdowns also impacted socialisation which had a notable impact on Niall, Abel, and Gemma. That said, we do also acknowledge that the pandemic and ‘stay at home’ messaging did facilitate a home-based active rehabilitation programme whereby greater leisure time was available for some due to reduced travel to and from working and social commitments.

The political discourse present in the United States affected the three American participants to varying degrees. Abel considers himself passionate about, and therefore hyper-involved in, the political culture of the US. Therefore, the BLM protests, the Capitol attack, and the Presidential election resulted in increased feelings of stress. This effect was vastly reduced in the second half of the study following the end of the election cycle, as evidenced in the PSS scores. While still affected by the same factors, Gemma was less involved and therefore found that the political discourse had more of an ‘exhausting’ effect and produced feelings of irritability. Finally, Kristen reported some levels of stress concerning the present circumstances; however, she was better able to cope as she was not as directly exposed to some of the unrest living in a secluded area compared to Abel and Gemma who were more city-based.

Work was another factor that was mentioned by several participants. As someone working in the healthcare industry, who also received a work promotion during the study, Luigi felt direct pressure from the COVID-19 pandemic. This was evidenced in his PSS scores. Abel and Gemma both suffered from job instability. Abel changed jobs at least three times during the study, as each position was more seasonal in nature and induced a degree of anxiety and frustration as well as resulting in periods of physically demanding work. Gemma’s work was directly impacted by the COVID-19 pandemic. After the onset of the pandemic, her job position became less stable. She tried taking on per diem work, but regardless her work situation was unstable largely throughout the study. In contrast, Simon reported a positive effect on his general well-being after returning to work following a prolonged period of medical leave.

Programme prescription and activity selection was another factor that emerged from the data. Exercise intolerance was present in two participants (Kristen, Simon) who both reported a history of moderate-to-severe TBI. Kristen struggled with fully participating in the study as a direct result of her exercise intolerance. While she was willing to ‘push through’ some short-term adverse reactions, she expressed that any time she overcame her exercise intolerance, the phase would change to a non-intervention phase and she had to start all over again during the next intervention phase. This sentiment is further supported by reported outcome measures in phase B4, which occurred immediately after phase B3 resulting in a double intervention phase. When given an additional six weeks of intervention, levels were improved for cognitive function, anxiety, and depression. Simon’s exercise intolerance was less severe, but it did have an impact on what activities could be prescribed. He did not report any adverse events because of the programme, aside from his difficulty with constant disruption and lack of control over his programmes. COVID-19 and subsequent lockdowns resulted in the disruption of preferred activities for several participants. Niall was not able to go to the gym or consistently take part in field hockey which he reported was a source of frustration. Not only were these his preferred activities, but both involved an element of socialisation and associated social capital. Luigi noted that he struggled to find ‘me time’ as he never felt comfortable returning to the gym even when it was open despite this being his preferred setting due to a fear of contracting COVID-19. Simon struggled with the inconsistency of gym availability which resulted in constant changes to his programmes. Finally, the weather had both a positive and a negative impact on activity selection depending on the time of year. Niall, Luigi, Abel, and Gemma all preferred outdoor activities, which meant participants enjoyed activity more in the warmer and less rainy months of the year when they could get outside. Despite these factors, various participants still expressed satisfaction with their programmes (Niall, Luigi, Gemma). Positive experiences were increased when the intervention utilised a goal-setting element to increase motivation (Niall, Luigi, Simon).

This mixed-methods single case design provides the first primary level evidence to suggest that active rehabilitation has potential as an intervention for those suffering from symptoms related to suspected CTE. Overall, active rehabilitation had a largely positive effect on measures of cognitive function whilst the results for mood and behaviour across the intervention period was mixed and heavily influenced by factors such as the COVID-19 pandemic.

Quantitative analysis suggests that active rehabilitation had an almost equally positive and unclear effect on overall cognitive function. These findings reflect that reported by Hearn et al. [ 9 ] who observed a pooled estimate across three studies as moderately positive but with confidence intervals crossing positive, unclear, and negative effects on cognitive function in populations with tau pathologies. Considering the outcome measures of general cognitive function (one positive, one trivial, one negative), the effect of active rehabilitation was inconclusive and suggestive of individual variation. The effect of active rehabilitation on measures of general cognitive function were inconclusive, with one trivial effect and one negative effect reported. Across the study, the only impairment observed in all participants was executive dysfunction. This supports the inclusion of memory impairment and executive dysfunction in the updated TES criterion presented by Katz and colleagues [ 5 ]. Four of the six participants reported a positive effect for executive function overall (Niall, Luigi, Abel, Simon) with two others being unclear (Kristen, Gemma). Importantly, there was no negative effect observed on measures of executive function. Overall, the effect of active rehabilitation on cognitive function has yet to be fully determined but is promising. While the underlying physiological mechanisms are yet to be fully understood, exercise is known to have a positive impact on neurogenesis and angiogenesis [ 19 , 20 ] which can have a positive influence on executive function.

The results for mood and behaviour symptoms were largely inconclusive. When considering the individual outcome measures, one positive, one trivial, and two negative effect were observed for anxiety. Whilst this outcome is surprising from a purely quantitative viewpoint, based on the qualitative results, it is unlikely that these effects are attributed solely to the active rehabilitation programme. For example, the qualitative data suggests COVID-19 and subsequent lockdowns had both a direct and indirect effect on many of these outcomes. Participants expressed how the consequences of lockdowns in response to the pandemic directly influenced symptom levels. Specifically, lockdowns negatively influenced components of executive function, as well as levels of anxiety and depression. One participant (Gemma) further expressed her feelings of fear and anxiety about the presence of the virus itself, specifically about the danger it posed to her loved ones. These findings reflect similar observations in various populations where the COVID-19 pandemic resulted in higher reported levels of stress, anxiety, and depression [ 21 , 22 ]. Supported by the PSS scores and participant interviews, the influence was likely stronger earlier on in the intervention period as participants eventually began to adapt to life with COVID-19. For those who contracted the virus (Kristen, Abel), symptom levels were also likely influenced by the presence of the virus. In addition to the respiratory and inflammatory symptoms associated with COVID-19, contracting COVID-19 has also been associated with the development of fatigue, anxiety, depression, and cognitive disturbances [ 23 ].

Despite the various proximal and distal factors being discussed, some of which were unprecedented and, at times, had a substantial influence on participant symptoms, this study was still able to provide positive results. Supported by the quantitative results, participants directly reported that they felt the active rehabilitation programme had a positive effect on symptoms of memory, attention, depression, and anxiety. It was suggested the effect may have been cumulative across the twelve-month intervention period (Kristen, Abel, Gemma). This is supported by the improvement of various outcome measures at the end of the study despite conflicting visual and/or statistical analysis, such as the improvement in levels of cognitive function in the case of Kristen or the improvement in levels of depression in the case of Abel. At the very least the programme offered a way to cope with or better manage symptoms as suggested by Luigi, Abel, Gemma. This is demonstrated by the consistent decreased variability in levels of irritability (Luigi) and loneliness (Niall) consistently observed during intervention phases despite a reported trivial effect. If repeated under more stable conditions, there is a potential for a greater, and perhaps stronger, number of observed positive effects, especially when considering the greater understanding of the context and individual preferences that emerged in this study. Further, some of the participants may not have met the eligibility criteria had these unprecedented factors not been present. For example, Niall’s levels of loneliness were only above reported population average during the baseline phase and proceeded to drop throughout the first half of the study independent of the active rehabilitation programme. Therefore, future research and clinical practice might consider symptoms and inclusion criteria that are evident outside of other factors such as the environment and personal circumstances.

This study also supports the use of a person-centred approach in future research and clinical care, as evidenced by the enhanced understanding of participant context and study experience. Utilising a mixed methods approach with data integration has allowed for an in-depth understanding of the observed visual and statistical results. This was further enhanced by the study length (twelve months). A person-centred approach to programme prescription ensured participant satisfaction and reduced dropouts. This is evidenced by the six participants who completed the twelve-month study and reported programme satisfaction despite many not having access to preferred activities.

Whilst this study is the first empirical investigation within this area, this study is not without some limitations. Firstly, we note that some features of the study might limit the generalisability of the findings. Indeed, the TES criteria was updated after the study commenced, the study was carried out during a pandemic, and the outcomes lack specificity to a a suspected CTE population. Secondly, whilst online methods were essential for the success of this study, there is an inherent lack of control over aspects such as other activities and the intensity these are performed which would impact some outcomes used in the study. Also, the online nature meant that outcomes were limited to questionnaires and qualitative data whereas objective measure to supplement these might have strengthened the validity of these findings. Future research should continue to utilise a person-centred approach within this area of enquiry to improved intervention efficacy while continuing to maintain study adherence and participant satisfaction. Also, future work should seek substantiate the findings in this study to allow for pooling of data.

In conclusion, this study has provided evidence to establish the potential use of active rehabilitation for the management of suspected CTE using a mixed-methods single case research design. Based on the results of this study, a narrative summary of the integrated qualitative and quantitative results has been provided in Table 4 . This table provides a holistic understanding of individual cases and observed effects supporting an overall conclusion, though we do refer readers to the individual cases in S3 – S8 Tables given the person-centred approach of the study. This study has offered preliminary evidence which suggests active rehabilitation may offer some benefit to individuals with symptoms of suspected CTE and warrants further investigation using standardised and innovative methodologies. This study has also demonstrated the benefit of a person-centred approach to both clinical research and practice. Considering factors such as personal circumstances, cultural climate, and a detailed intervention response allows for a better understanding of an intervention effect within the context of the study.

https://doi.org/10.1371/journal.pone.0302260.t004

Supporting information

S1 table. screening assessment..

MCI: mild cognitive impairment; PROMIS: Patient-reported outcomes measurement information system. N/A: not applicable due to multiple subscales used.

https://doi.org/10.1371/journal.pone.0302260.s001

S2 Table. Outcome assessments.

https://doi.org/10.1371/journal.pone.0302260.s002

S3 Table. Niall’s summary of results.

https://doi.org/10.1371/journal.pone.0302260.s003

S4 Table. Luigi’s summary of results.

https://doi.org/10.1371/journal.pone.0302260.s004

S5 Table. Kristen’s summary of results.

https://doi.org/10.1371/journal.pone.0302260.s005

S6 Table. Abel’s summary of results.

https://doi.org/10.1371/journal.pone.0302260.s006

S7 Table. Gemma’s summary of results.

https://doi.org/10.1371/journal.pone.0302260.s007

S8 Table. Simon’s summary of results.

https://doi.org/10.1371/journal.pone.0302260.s008

Acknowledgments

We would like to thank the participants who demonstrated such courage and dedication, especially in such chaotic and unprecedented times. We would also like to pay tribute to our co-author, Maria I Cordero, who sadly passed before this manuscript could be completed. Maria your patience, your kindness, and your pursuit of excellence will never be forgotten.

• View Article
• PubMed/NCBI
• Google Scholar
• 12. Borg G. Borg’s perceived exertion and pain scales. Human kinetics; Champaign, USA 1998.
• 18. Yin RK. Case study research: Design and methods. Vol. 5. Thousand Oaks, CA: Sage; 2009.

Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.

• View all journals
• My Account Login
• Explore content
• About the journal
• Publish with us
• Sign up for alerts
• Open access
• Published: 31 May 2024

What difference does one course make? Assessing the impact of content-based instruction on students’ sustainability literacy

• Inan Deniz Erguvan   ORCID: orcid.org/0000-0001-8713-2935 1  

Humanities and Social Sciences Communications volume  11 , Article number:  708 ( 2024 ) Cite this article

Metrics details

• Language and linguistics

Composition studies, with their cross-disciplinary role in students’ academic lives, can be essential in placing sustainability at the center of students’ learning. This research assessed the impact of content-based instruction on students’ sustainability literacy in a first-year composition course through a mixed-method design. In the quantitative part of this case study, 221 students in different classes of a first-year writing course in a higher education institute in Kuwait during the Fall term of 2022 were first given a pretest to determine their sustainability literacy levels. During a 6-week period, 121 students participated in the content-based instruction emphasizing sustainability, while 100 students comprised the control group, receiving curriculum without any emphasis on sustainability. The allocation of students in these two groups was random, determined solely by the classes they were enrolled in at the beginning of the semester. At the end of the semester, both the experimental and control groups were given a posttest to measure the impact of the instruction on their sustainability literacy levels. For the qualitative component, 60 students from the experimental group and 60 students from the control group were tasked with composing an essay identifying Kuwait’s major sustainability challenges and proposing corresponding solutions. The impact of content-based instruction on students’ literacy levels was measured by conducting a qualitative and quantitative content analysis on their writing. The results showed that the experimental group students made statistically significant improvements in their sustainable literacy levels, scored better on the posttest, used more sustainability terms and concepts, and identified more sustainability-related challenges and solutions in their essays.

Introduction

Our planet faces a critical emergency, evident in ecosystem devastation, species extinction, the depletion and destruction of vital resources, widespread pollution, and extreme poverty affecting billions of people. Scientists attribute these challenges significantly to our ignorance of the limits of Earth’s resources.

With its 17 Sustainable Development Goals (SDGs), the United Nations’ 2030 Sustainable Development Agenda is one of the most important attempts to solve the intricate global issues of our day. Reaching the objectives of sustainable development (SD) requires education. According to the UN, sustainability literacy includes the mindsets, abilities, and information people need to genuinely commit to creating a sustainable future and make wise decisions in that direction (Decamps, 2017 ).

The Education for Sustainable Development (ESD) framework, developed by the UN, serves as a roadmap for institutions and educators to revise curricula and teaching pedagogies based on sustainability principles. This framework is employed by hundreds of universities worldwide (Yuan and Zuo, 2013 ). ESD has gained political and institutional acceptability in many parts of the world recognizing the potential of promoting sustainability literacy to foster creative solutions to the world’s problems.

There is consensus that higher education institutions (HEIs) ought to promote sustainable development via research and activism since all students, regardless of their field of study, have the capacity to be social change agents. As a result, pupils need to acquire the skills necessary to contribute to a sustainable future (Buckler and Creech, 2014 ).

HEIs are responsible for teaching sustainability literacy and producing environmentally conscious citizens, given their ability to shape students’ attitudes and perspectives (Stephens et al. 2008 ). This endeavor holds particular significance in creating a new generation keenly aware of the global environmental challenges we are going through (Koehn and Uitto, 2017 ).

Leal Filho ( 2010 ) argues that universities cannot avoid dealing with the biggest problems that humanity is currently experiencing. Additionally, he contends that ESD is especially important in higher education since students will soon be pursuing careers in a variety of fields and will need to understand how their careers can contribute to the solution of sustainability issues. According to Leal Filho ( 2010 ), ESD will inspire students “to take action both during their time as students and, later on, as professionals” (p. 2). Therefore, in order to effectively address the difficulties they will experience in their various disciplines, undergraduates should develop competence-based sustainability awareness and literacy.

Sustainable development is not restricted to a single science. Composition studies, with their inherent cross-disciplinary and distinctive purpose in students’ academic lives, can play an important role in making sustainability a core focus of the curriculum. Composition instructors have the freedom to teach in a variety of contexts and disciplines. While teaching composition is as labor-intensive as any other subject in higher education, writing instructors have more clout to urge students to investigate a wide range of topics than academics who teach in more specialized fields (Owens, 2001 ).

Although certain curricular initiatives have been the subject of research, the impact of curriculum design on improving sustainability understanding has not received as much attention. Because of this, there are currently no guidelines in the literature for developing curriculum that specifically address sustainability learning objectives. Therefore, the goal of this project is to enhance students’ sustainability literacy by supporting the development of a structured curriculum in a first-year writing course. In order to do this, this study examined the benefits of utilizing textual and audiovisual materials in content-based instruction to introduce students to the three dimensions of sustainability as well as the United Nations Sustainable Development Goals.

Content-based instruction (CBI) is a popular approach to language education that combines content and language learning. Content-based education varies from standard language classes in that language comes second to content. This teaching technique is deemed effective because it employs English as a medium to impart content knowledge while providing various opportunities for students to use English in class (Brinton et al. 2003 ). Thus, the use of English stems from meaningful purposes (content learning) and frequent practice (opportunities to use English), resulting in an environment conducive to rich discussions, ultimately improving language fluency while reinforcing the content taught in a variety of academic areas. In other words, content-based language acquisition gives pupils a valid or relevant purpose to use the language they are learning (Kennedy 2006 ).

The CBI is seen as an effective tool for preparing students for higher education studies in a new language and context. Song ( 2006 ) conducted a long-term study to find out how well CBI worked for ESL students at a community college in the United States. According to the study, students enrolled in the content-linked ESL program passed the ESL course with greater marks and pass rates. They also performed better in follow-up ESL and developmental English classes. Overall, compared to their peers, the ESL students who were linked to content demonstrated higher levels of long-term academic performance. Higher GPA overall, graduation and retention rates, and English proficiency exam pass rates were all indicators of this achievement.

According to Stoller ( 2004 ), CBI stands out for its dedication to both language and content-learning objectives. Over the years, the program has garnered support as a result of students’ improved language skills and content-area knowledge at the elementary, middle, and post-secondary education levels, which attests to its perceived successes. According to Kennedy ( 2006 ), kids who study languages in addition to other subjects perform better academically and are able to make links between their studies and the real world. Multiple teaching methodologies are employed in content-based foreign language instruction, which also accounts for the variety of learning styles and intelligences present in the classroom (Kennedy, 2006 ).

However, despite the abundance of interest in using CBI to increase students’ awareness of certain topics and concepts, there is still a lack of research in assessing the impact of CBI on sustainability literacy. There are a few case studies, several reports of individual attempts and class practices to implement CBI in EFL classes to familiarize students with sustainability concepts (Vorholt, 2018 ; Schneider, 2017 ), and the empirical studies tend to focus on assessing teachers’ perspectives on teaching sustainability to their students (Shah et al., 2022 ; Maijala et al., 2023 ). Additionally, studies assessing the impact of CBI on students’ sustainability literacy with an experimental research design are very rare in the literature. Thus, this research is expected to make an important contribution to the field of sustainability education in higher education institutions.

This research employs a case study approach due to its ability to allow in depth, multifaceted explorations of complex issues in a real-world context (Crowe et al., 2011 ). This methodology aligns with the exploratory nature of this research, enabling us to generate a contextualized understanding that contributes to the existing body of knowledge. The selected case institution offers a valuable opportunity to examine a real-world scenario that is both relevant to our research questions and has generated practical implications for decision-makers in the field.

The main research questions that will guide the study are as follows:

Did the content-based instruction have any significant effect on the participants’ sustainability literacy levels?

Are there any differences between the control and experimental groups’ essays in terms of students’ perceptions of sustainability challenges and their solutions in Kuwait?

Sustainability

The idea of sustainability is not so new; it existed before the field of environmental sciences as we know it today. Nonetheless, the need to use resources sustainably has become more widely recognized due to factors including population growth, increased consumption following the Industrial Revolution, and the threat of the depletion of essential resources like coal, oil, and wood. Fears that living standards would not be maintained for current or future generations sparked a style of thinking that led to the creation and acceptance of sustainable development (Du Pisani, 2006 ).

Although there is still no commonly accepted definition of sustainability, its context has eventually widened to include “three pillars”; namely the social, economic, and environmental aspects of sustainability (Purvis et al., 2019 ). Initially, the focus was mainly on the environmental dimension of sustainability and many researchers considered this dimension more important than the other two, however, later, the economic, and social dimensions started to attract similar amounts of attention (Colantonio, 2007 ).

Following the 1972 United Nations Conference on the Human Environment, which was the first UN conference devoted to environmental issues, there have been global efforts to redefine sustainability. There are many definitions of sustainable development, but the one that is most often cited comes from the 1987 Brundtland Report, also known as Our Common Future: “Sustainable development is development that meets the needs of the present without compromising the ability of future generations to meet their own needs” (Brundtland, 1987 ).

Solutions to sustainability issues, whether ecological, social, or economic, hinge on decision-making processes at both the organizational and individual levels. It is important to recognize that organizational decisions stem from individual choices (Carley and Behrens, 1999 ). Therefore, the success of sustainability goals largely relies on individual decision-making, particularly in consumer behavior. By opting for sustainable choices, consumers can drive demand for sustainable products and services, articulate their values, reduce their environmental footprint, and contribute to building a culture of sustainability. The positive effect of education on pro-environmental consumption behaviors is evident in the literature (AlNuaimi and AlGhamdi, 2022 ; Adjengdia and Schlegelmilch, 2020 ; Achola et al., 2020 ) and was recognized in the Brundtland Report ( 1987 ).

Furthermore, in 2002, the World Summit on Sustainable Development and the Commission on Sustainable Development emphasized the crucial role of information in informed decision-making. Hence, education emerges as a vital instrument in achieving sustainability goals. It empowers individuals and communities to take meaningful action and make informed choices that safeguard the environment while promoting social and economic development.

Sustainability literacy

Organizations from a variety of sectors have prioritized educational projects aimed at improving people’s understanding of sustainability because they believe that a sustainable future requires a society that is knowledgeable about sustainability. The significance of sustainability education has been emphasized recently by international organizations, private companies, and most significantly, higher education institutions. Renewing interest in creating trustworthy assessments of sustainability literacy and knowledge has coincided with the increased emphasis on sustainability education.

Various approaches have been used to develop a valid assessment tool for sustainability literacy. One noteworthy example is the SULITEST (Sustainability Literacy Test), established after the Rio+20 Conference (Decamps et al., 2017 ). SULITEST is an online standardized set of multiple-choice questions that can be used globally, alongside specialized modules tailored to specific national, regional, and cultural contexts. Décamps et al. ( 2017 ) outlined the structure of this tool and highlighted its potential for measuring sustainability literacy on a global scale, recommending its adoption by educational institutions.

Similarly, Zwickle and Jones ( 2017 ) developed a web-based survey tool to assess the sustainability knowledge of undergraduate students at Ohio State University; this tool involved 1000 participants and comprised 16 multiple-choice questions. In the United States, the American Association for the Advancement of Sustainability in Higher Education (AASHE) introduced the Sustainability Tracking Assessment Rating System (STARS) in 2010, with participation from more than a thousand institutions by 2022. The STARS evaluates the sustainability efforts of colleges and universities in the U.S., rewarding institutions that offer a greater number of sustainability-related courses or even require students to complete at least one sustainability course as part of their general education requirements (Bullock and Wilder, 2016 ). Participating institutions assess the sustainability literacy of their students, focusing on their knowledge of sustainability topics and challenges.

As higher education institutions and society at large increasingly prioritize the importance of individuals’ understanding of sustainability, the need for accurate assessments of sustainability knowledge becomes more significant. The development of improved measures of sustainability knowledge is anticipated to enhance sustainability education and ultimately cultivate a population with higher levels of sustainability literacy (Kuehl et al., 2023 ).

Sustainability in Kuwait

Kuwait is identified as one of the wealthiest countries in the world, owing to its substantial revenues derived from the oil sector. The country enjoys an abundance of wealth from the oil sector which make up more than 90% of Kuwait’s export earnings, a dependence that makes it difficult to diversify the economy and develop other industries that are less reliant on fossil fuels (Eltony, 2002 ). Consequently, Kuwait encounters various sustainability challenges, primarily stemming from its heavy reliance on oil revenues (AlOthman and Palliam, 2018 ). Some of the major environmental challenges faced by Kuwait are air pollution, water scarcity, and waste management. The country has high levels of air pollution due to its petrochemical industry activities and transportation. Water scarcity is a significant issue in Kuwait, where desalination plants are relied upon to meet water needs. Nevertheless, Kuwaitis consume a staggering 520 l of freshwater per capita per day, one of the highest in the world (Kuwait National Development Plan, 2017 ). Waste management poses another significant challenge, as Kuwait generates large amounts of waste due to high mass consumption, necessitating proper disposal and recycling methods (Al Yaqout et al., 2002 ; Koushki et al., 2004 ). Currently, water and energy consumption, along with waste production per capita, rank among the highest globally in Kuwait.

The country has launched several initiatives to promote sustainable development, and the most significant initiative is the Kuwait National Development Plan (KNDP) that serves as a roadmap for sustainable development in Kuwait. The KNDP emphasizes the importance of economic, social, and environmental sustainability and sets targets for reducing carbon emissions, improving waste management, and promoting renewable energy (Kuwait National Development Plan, 2017 ). Kuwait officially embraced the SDGs in September 2015, subsequently integrating them into its Vision 2035 plan.

Despite these efforts, Kuwait currently holds the 101st position out of 163 countries, with an overall score of 64.53 (Sachs et al., 2022 ). Furthermore, there remains a gap in the implementation of sustainable practices by government agencies and a lack of sustainable awareness among the public. Very few studies exist in this domain, with the overarching message emphasizing the need for greater awareness of sustainability in Kuwait. For example, a study by Al Qattan and Gray ( 2021 ) revealed that government policies and practices inadequately address pollution issues, particularly in Kuwaiti water bodies. Similarly, AlSanad ( 2015 ) found that lack of awareness acts as a main barrier to adopting sustainable construction approaches in Kuwait and stresses the need for governmental initiatives such as standards, policies, and incentives to promote sustainability. According to similar research (Koushki et al., 2004 ; AlSulalili et al., 2014 ; Al Beeshi et al., 2020 ), there is a dearth of public knowledge of sustainable waste management techniques and municipal programs for waste prevention, reduction, or recycling.

Kuwait’s overall score of 64.53 places it 101st out of 163 countries, notwithstanding these efforts (Sachs et al., 2022 ). In addition, there is still a lack of public understanding of sustainability issues and a gap in the way government agencies are implementing sustainable practices. There are very few studies in this field, and most of them emphasize how important it is for Kuwaitis to be more conscious of sustainability. Al Qattan and Gray’s study from 2021, for instance, showed that pollution problems are not sufficiently addressed by government policies and practices, especially when it comes to Kuwaiti water bodies. Similar findings were made by AlSanad ( 2015 ), who highlighted the necessity of governmental initiatives such as standards, rules, and incentives and discovered that a major obstacle to Kuwait’s adoption of sustainable construction practices is a lack of awareness.

In conclusion, despite bourgeoning awareness of sustainability among businesses and the government’s initiatives to promote sustainability, Kuwait still requires heightened awareness and implementation of sustainable practices and concerted efforts to address the nation’s oil reliance and propel towards a more sustainable future.

Methodology

This study has a true experimental research design with random assignment of students in control and experimental groups, with a pretest and posttest administered to both groups. A mixed-method sequential explanatory approach was adopted to collect the data, which were first quantitative and then qualitative in two consecutive phases of the study (Creswell, 2012 ; Creswell and Clark, 2011 ). Using mixed methods helps to provide a more comprehensive framework of the phenomenon by enabling rich and informative data and validating and triangulating the data by analyzing the same issue through both quantitative and qualitative methods (Silverman, 2000 ).

Research population

The research population of the study consisted of students at a private university in Kuwait based on an American-style model of higher education that offers instruction in English. A total of 221 first-year composition students participated and were divided into experimental and control groups, with 100 students assigned to the experimental group and 121 to the control group. The allocation of students into these groups was random and determined by their enrollment in specific course sections at the beginning of the semester. The discrepancy in group sizes reflects variations in the number of students per course section, typically ranging from 20 to 25.

In the experimental group, 100 students received specialized content-based instruction focused on sustainability, while the remaining 121 students in the control group completed regular assignments as outlined in the course syllabus, covering various predetermined topics assigned by their writing instructors. Both groups underwent a pretest before the commencement of content-based instruction and a posttest at the conclusion of the semester.

The participants’ demographic information is displayed in Table 1 .

For the qualitative part of the study, the researcher collected essays from students at the end of the Fall semester of 2022. The research population consisted of students in both the experimental and control groups who attended the class and signed the consent form on the day of data collection, week 15 of Fall 2022. There were 65 students who produced an essay in the experimental group and 67 in the control group. Five essays from the experimental group and seven essays from the control group were eliminated because they had a very low word count (less than 100 words), thus, 120 were left for analysis.

Data collection

The quantitative section collected data through an adapted version of the Sustainability Literacy Assessment, prepared by a committee at the University of Wisconsin-Oshkosh to measure the university’s sustainability performance, within the Sustainability Tracking, Assessment & Rating System (STARS) framework ( 2018 ). The assessment form included four sections, testing the knowledge level with five multiple-choice questions, and assessing students’ self-reported skills, attitudes and familiarity with some sustainability topics and concepts on a five-point Likert scale. The Institutional Review Board (IRB) approval (case number 278674) was obtained, and students signed the consent form before the data collection. A total of 221 students completed the questionnaire—121 in the control group and 100 in the experimental group.

Table 2 shows the reliability scores of these sections of the data collection tool. When the scales are examined, it is determined that they have a good level of reliability. A Cronbach’s alpha greater than 0.50 indicates that the scale used is reliable. This also indicates that the internal consistency of the scale used in the study is good.

In the qualitative data collection, students in both the experimental and control groups were asked to write a short essay identifying the major sustainability challenge of Kuwait and offering solutions to this problem. This session was conducted during the scheduled class time of 50 min, on the computer under the instructor’s supervision.

Data analysis

The quantitative data were analyzed using the SPSS Statistics (Statistical Package for Social Sciences) for Windows 25.0 program. Along with descriptive statistical methods (numbers, percentages, minimum-maximum values, median, mean, standard deviation), chi-square analysis was applied to test the homogeneity of the groups. The data were checked for the normal distribution compatibility with Q–Q plot drawing for its skewness and kurtosis values (±3).

For quantitative data comparison in normally distributed data, an independent t test was used for comparisons between two independent groups, and a dependent t test was used for comparisons between two dependent stages. One-way ANOVA was applied for comparisons of more than two independent groups.

Three processes comprise the data analysis process in qualitative research: arranging and prepping the data for analysis, coding and condensing the data to reduce the data into themes and presenting the data in tables and figures (Creswell, 2012 ). The content analysis method was used to assess the data collected for this study. The methodical, impartial, and, if feasible, quantitative examination of the content of different documents is known as content analysis (Bilgin, 2006 ). Content analysis’s primary goal is to find ideas and connections that will contribute to the explanation of the information gathered.

The student essays were imported into the MAXQDA 2022 program, which utilizes visual analysis tools extensively and offers a more structured approach to data analysis than manual analysis (Kuckartz and Rädiker, 2019 ). To identify the most frequent words and word combinations in the essays, a quantitative content analysis was performed using the MaxDictio function of the software. For the qualitative content analysis, a combined approach incorporating both inductive and deductive methods was employed. The researcher thoroughly reviewed the data multiple times, generating initial codes. Codes that were related to each other were then grouped together under relevant themes and assigned appropriate names. Subsequently, the obtained themes were elaborated upon in detail and the findings were interpreted.

Research Question 1

Did the content-based instruction have a significant effect on the participants’ sustainability literacy levels?

The first section of the questionnaire included five questions testing students’ knowledge of sustainability. Table 3 below shows the percentages of correct and incorrect answers in the control and experimental groups according to the pretest and posttest scores.

According to Table 3 , the experimental group of students in the posttest scored the highest percentage in the knowledge questions. The percentage of correct answers produced by the students in the control group did not show a consistent pattern, while it increased in Q1 and Q3, it decreased in Q2, Q4, and Q5.

However, for the experimental group, the students’ correct answers to all the questions increased. To assess whether these differences were statistically significant, an independent t test was conducted between the pretest and posttest scores of the two groups.

Table 4 shows the result of the independent t test conducted to compare the average knowledge of the participants before and after the CBI. There was no statistically significant difference between the control group and experimental group participants’ pretest knowledge averages, but there was a statistically significant difference in posttest averages ( p  < 0.05).

The questionnaire also included questions asking students to evaluate their literacy in skills, attitudes, and topics and concepts regarding sustainability. Table 5 shows the results of the independent t test conducted to compare the skills, attitudes, and topic and concept scores of the participants according to their groups. According to these findings, the posttest scores for skills, attitudes, and familiarity with topics and concepts were significantly greater for the experimental group participants than for the control group participants.

Research Question 2

Are there any differences between the control and experimental groups’ essays in terms of their perceptions of sustainability challenges and their solutions in Kuwait?

To analyze this question, students were asked to write an essay identifying the major sustainability challenge of Kuwait and offering some solutions to it. The essays were processed through MAXQDA, and the frequency distributions of the control and experimental group student essays are shown in Table 6 .

According to student perspectives, the major sustainability challenges in Kuwait were dependence on oil, donating money to other countries and unemployment in the economic area. In the environmental realm, pollution and littering were the most frequently mentioned problems, followed by climate change. Loss of biodiversity and scarcity of resources were the other two major environmental sustainability challenges. Social sustainability issues in Kuwait, as per student views, could be listed as health and wellbeing problems, corruption, lack of quality infrastructure, quality of education, gender inequality and discrimination and human rights issues.

Table 6 also shows the number of essays mentioning the coded sustainability problems in each group type. According to these findings, students in the control group identified similar codes for environmental and economic sustainability problems, except for unemployment, with varying frequencies. However, regarding social sustainability problems, no control group student mentioned quality of education, reducing inequality and discrimination, or traffic accidents, and only one student mentioned malnutrition and obesity, corruption and gender equality. These issues were identified by a larger number of students in the experimental group. Overall, in the sustainability problems content analysis, 94 codes were included in the control group, and 137 codes were included in the experimental group.

Table 7 below shows the codes in the student essays for solutions to Kuwait’s major sustainability problems. According to the content analysis of the essays, the control group students did not mention five solutions that were mentioned by the experimental group. These were Kuwaitization and creating jobs in the economic sustainability domain, improving the quality of education, reducing inequality and discrimination, and reducing traffic accidents in the social sustainability domain. Both groups proposed the same solutions in the environmental domain, with control group 84, and experimental 68 codes. However, overall, the control group students had 104 codes, and the experimental group students had 137 codes for sustainability solutions.

A final content analysis was conducted quantitatively, via the MAXDictio function of MAXQDA to test how many sustainability related terms and concepts the students used in their essays. The list prepared by The Association for the Advancement of Sustainability (AASHE)’s Suggested Keywords for Sustainability Course and Research Inventories (The Association for the Advancement of Sustainability in Higher Education, 2023 ) was uploaded to the software and the student essays were analyzed based on these keywords.

According to Table 8 , the word count of the essays in the experimental group reached 27,751, and that of the control group reached 28,303. Despite the higher word count, dictionary-based content analysis revealed that the experimental group used more sustainability related keywords, as listed in the inventory. Students in the experimental group used 122, and the control group used 97 of these suggested sustainability keywords in their essays.

This study aimed to assess the impact of a 6-week course on content-based instruction (CBI) on the sustainability literacy levels of composition students. Our findings indicate that CBI significantly improved the sustainability literacy of the experimental group, as evidenced by their post-test scores and written work.

The first research question was addressed quantitatively, revealing significant improvements in the experimental group’s knowledge levels, skills, attitudes, and familiarity with sustainability concepts compared to those of the control group. This finding suggested that CBI effectively enhanced students’ sustainability literacy.

The second research question was explored qualitatively through the analysis of student essays. The experimental group demonstrated a greater ability to identify sustainability problems facing their country and propose solutions, particularly in the social sustainability domain. Additionally, they used more sustainability-related keywords in their essays, despite the control group having longer essays.

The results of our data analysis for both research questions revealed the positive effect of CBI on student learning. Content-based instruction is indeed widely recognized for its potential to enhance language learning outcomes and our findings are consistent with those of several previous studies in the field. While sustainable development is not frequently included in language education or promoted as part of teacher preparation for language learners (Maijala et al. 2023 ), it can readily succumb to CBI. CBI has begun transforming language-learning environments into places where students utilize language to research urgent global challenges, such as climate change (Turpin, 2022 ). A wide range of curricular approaches are included in CBI, ranging from language-focused programs where content is viewed as a helpful tool for extending the goals of the language curriculum to content-focused programs where content acquisition is prioritized over language learning (Met, 1999 ). As a result, teaching environmental and sustainable education in English as a foreign language (EFL) classes is growing in popularity.

Vorholt ( 2018 ) designed and taught a 6-week CBI course titled, “Environmental Issues” to undergraduate students at Lewis & Clark College, USA. The course focused on ecology versus economy, sustainability, and activism, which involved activities such as service learning and speaking. However, although she published her experiences and guidelines for designing the course, she did not assess the impact of student learning at the end of the course. Another review involved evaluating the opportunities for using an online German class as a vehicle for sustainability education in Ecuador, through content-based instruction (Schneider, 2017 ). This paper proposes adjusting the content of an online class and offering activities that will promote sustainability in a developing economy such as Ecuador.

A similar study was conducted in Switzerland, where SULITEST was administered to first-semester students in an HE institution, both before and after the survey (Zizka and Varga, 2021 ). Although the method used was not content-based instruction, the authors suggested that students from various nationalities and linguistic backgrounds in the Swiss HEI received an introductory course in English and French to sustainable hospitality culture aimed at providing insight into hospitality and tourism challenges and to reflecting on their sustainable solutions. The course did not specifically target the SDGs, but according to the posttest results, students’ knowledge about sustainability in general improved, and even exceeded the worldwide averages overall.

An attempt to incorporate environmental sustainability was made by task-based teaching in a translation course at two universities in Indonesia (Siregar et al., 2022 ). At the end of the course, the posttest demonstrated that the student’s confidence, one of the keys to acquiring a language, increased when using specific terms. The combination of task-based learning with appropriate content that is relevant to personal life, such as environmental sustainability increased the students’ motivation to learn and benefit from the translation activity.

Task-based learning was used in a translation course at two Indonesian institutions in an effort to include environmental sustainability (Siregar et al. 2022 ). The post-test at the end of the course showed that utilizing particular terms boosted the student’s confidence, which is one of the cornerstones to learning a language. Students were more motivated to learn and gain from the translation exercise when task-based learning was combined with relevant, real-world topics, including environmental sustainability.

A closer look at the findings of the second research question highlights the fact that students in both the experimental and control groups produced the highest number of codes for sustainability problems and solutions in the environmental pillar of sustainability. This aligns with the literature which suggests that the environmental pillar of sustainability is most often the one that students are more aware of (Zizka and Varga, 2021 ). For example, Chaplin and Wyton ( 2014 ) found that university students strongly associate recycling and sustainable living, and in many cases, they are believed to be the same thing. According to Drayson et al. ( 2014 ) the environmental dimension is the most prominent dimension in university students’ understanding of sustainable development. Another study conducted in China (Yuan and Zuo, 2013 ) showed that the students’ perceptions of the top priorities for higher education for sustainable development are generally environmentally oriented.

One interesting finding of the content analysis of the essays is that students in the experimental group mentioned more social sustainability problems and solutions than did those in the control group. These essays produced codes such as corruption, gender inequality and (lack of) quality of education, which are indeed some major social sustainability challenges Kuwait is facing, as reported in the Sustainable Development Report by the UN (Sachs et al., 2022 ). Gender inequality in Kuwait has been described as “significant challenges stagnating” by the UN, scoring particularly low in indicators such as the “ratio of female-to-male labor force participation” and ‘seats held by women in the national parliament’. Despite the growing achievements of Kuwaiti women, they still face challenges in social, cultural, and political arenas (Al Zuabi, 2016 ). In his study, Al Zuabi explored the Kuwaiti women’s challenges in attaining participation in the sociopolitical development of Kuwait and found that there are barriers preventing their empowerment and effective participation in national development. The fact that four students in the experimental group presented this problem and offered solutions to ensuring gender equality in the country as opposed to zero students in the control group could be interpreted as a positive influence of the sustainability-focused CBI.

Another major social problem that the country is facing and that emerged in the experimental group essays is corruption. Kuwait’s score in the Corruption Perceptions Index is decreasing (Sachs et al., 2022 ) and is defined as a significant challenge indicator. According to Al Saif ( 2020 ), corruption is a multilayered system in Kuwait that involves more than embezzlement and money laundering, with “wasta” (the Arabic word for the use of connections and influence to gain favors) serving as the cornerstone. Although corruption poses an existential threat to the country, it remains widespread to the extent that it has “become a staple of governance and a feature of everyday life in Kuwait” (Al Saif, 2020 ). Kuwait’s ranking in corruption indices falls every year, and this major social problem was identified solely by experimental group students, rather than by the control group.

The quality of education was another social sustainability problem mentioned by the experimental group students, but not by the control group. Despite some challenges, Quality Education (SDG 4) is a domain in which Kuwait seems to be doing better according to UN standards, with its high literacy and school enrollment rates. However, the Kuwaiti education system falls far below international standards and is quite inefficient, resulting in a higher cost per student. Among 141 countries, Kuwait has been ranked 112th globally in the skillset of graduates and 83rd in the quality of vocational training, according to the Global Competitiveness Report (World Economic Forum, 2019 ). Kuwait University, the only state university in the country, was ranked 9th in the GCC region, 19th in the Arab World, and 83rd in the MENA region (Abualrub, 2016 ). The major underlying reasons include a short school year, a high repetition rate, and low expenditure on school textbooks and teaching materials (Burney et al., 2013 ). The education system would benefit from increased use of technology, improved educational curriculum, and higher recruitment standards for teachers and their teaching skills (Murad and AlAwadhi, 2018 ; AlFelaij, 2016 ; AlHashem and AlHouti, 2021 ).

Foreign language teachers can play a crucial role in promoting sustainability; however, there are certainly some obstacles to implementing sustainability education in foreign language classes. Academics’ attitudes and level of awareness play a key role in shaping the successful implementation of a range of pedagogical techniques for ESD goals (Crosling et al. 2020 ). Currently, the greatest challenge is teachers’ lack of knowledge of sustainability concepts and their limited experience in teaching sustainability (Maijala et al., 2023 ; Shah et al., 2022 ).

In some countries where sustainability issues are on the political and educational agenda, in-service courses aiming to strengthen university teachers’ competence in integrating sustainable development (SD) into their classes are underway. At Uppsala University, Sweden, such a course was open to diverse participants from different faculties and allowed for stimulating exchanges of knowledge and perspectives (Rehn, 2018 ).

Conclusions and recommendations

In conclusion, this study aimed to evaluate the effects of a six-week content-based instruction (CBI) on the sustainability literacy of first-year composition students. The results from the experimental group showed significant enhancements in knowledge, skills, attitudes, and familiarity with sustainability concepts, as evidenced by the independent t test and content analysis findings.

Quantitative analysis revealed a clear increase in students’ sustainability literacy, aligning with CBI’s recognized potential to enhance language learning outcomes. Qualitative examination of the student essays further highlighted a deeper grasp of sustainability issues, particularly in the environmental domain, echoing existing literature regarding heightened environmental awareness among students.

Additionally, the experimental group demonstrated a heightened awareness of pressing social sustainability challenges in Kuwait, such as gender inequality, corruption, and educational quality. These topics were less emphasized or absent in the control group essays, indicating the positive influence of sustainability-focused CBI on students’ understanding of the social dimension of sustainability.

This study contributes to the existing research in two significant ways. First, it highlights the effectiveness of integrating sustainability into language education through CBI within an ESL context. The observed positive impact suggests that targeted interventions can effectively enhance students’ sustainability literacy, even within traditional language-focused curricula.

Second, the study emphasizes the potential of interdisciplinary approaches to bolster sustainability education in higher education. Collaborative efforts, workshops, and training opportunities across departments can equip writing and composition instructors with the pedagogical tools to integrate sustainability into their curriculum, fostering a more sustainable language-teaching culture.

This study is subject to several limitations. These include the relatively short duration of the CBI, and a small research population focusing on a specific group of students. Importantly, this was a case study in which one faculty member designed her own course materials to integrate sustainability into a first-year writing course at a higher education institution. Despite these constraints, the results were positive. Students exposed to CBI with a sustainability theme demonstrated increased sustainability literacy, evident in their improved scores on knowledge tests, incorporation of sustainability concepts, and the identification of sustainability problems and solutions in their essays. While these findings may not be broadly applicable, they suggest the potential impact of dedicated teachers designing courses to enhance student learning. Additionally, the scope of the study was limited because the effects of CBI were measured shortly before the semester ended, precluding assessment of students’ retention levels in subsequent semesters or years. Therefore, further research is necessary to explore this aspect.

Higher education institutions have a powerful opportunity to equip students in all disciplines with the knowledge and skills needed to achieve the UN’s SDGs by 2030 (Briens et al., 2022 ). By integrating sustainability education across all programs, universities can create a generation of graduates prepared to tackle global challenges.

To this end, preparing teachers and faculty to integrate sustainability issues in language teaching is essential. Higher education institutions should create collaborative programs and training for faculty to boost their understanding of sustainability. These initiatives should educate participants on integrating environmental, social, and economic issues into their teaching subjects and encourage them to develop activities that facilitate integrated teaching approaches (Nur et al., 2022 ; Hauschild et al., 2012 ; Çetinkaya et al., 2015 ).

Finally, educators should be encouraged to conduct similar case studies to contribute to a growing body of evidence showing the positive impact of dedicated teaching efforts in promoting sustainability literacy.

Data availability

https://data.mendeley.com/datasets/fhny7btkg7/1 .

Abualrub SY (2016) The problems with Kuwait’s struggling educational system. Kuwait Times. https://kuwaittimes.com/problems-kuwaits-struggling-educational-system/

Achola G, Asamoah-Manu N, Tani YD, Weiss A (2020) Consumer education can lead to behaviour change. Field Actions Sci Rep 22:96–103. http://journals.openedition.org/factsreports/6426

Google Scholar  

Adjengdia BB, Schlegelmilch BB (2020) Linking sustainable product attributes and consumer decision-making: Insights from a systematic review, J Clean Prod, 245, https://doi.org/10.1016/j.jclepro.2019.118902

Al Beeshi A, Alsulaili A, Al-Fadhli F (2020) Food waste management in Kuwait: current situation and future needs. 5th Eurasia waste management symposium, 26–28 October 2020. Proceedings, Istanbul, Turkey, https://www.eurasiasymposium.com/EWMS/files/Hall2/Session-6/167_Alsulaili-14.pdf

Al Felaij B (2016) Why integrating technology has been unsuccessful in Kuwait? An exploratory study. E-Learn Digit Media 13(3–4):126–139. https://doi.org/10.1177/2042753016672901

Article   Google Scholar  

Al Hashem F, Alhouti I (2021) Endless education reform: the case of Kuwait. International perspectives on education and society, 345–367. https://doi.org/10.1108/s1479-367920210000040019

Al Othman S, Palliam R (2018) Environmental degradation: Waste disposal and management in Kuwait. SSRN Electron J https://doi.org/10.2139/ssrn.3272799

AlNuaimi SR, AlGhamdi SG (2022) Sustainable consumption and education for sustainability in higher education. Sustainability 14(12):7255. https://doi.org/10.3390/su14127255

Al Qattan MEA, Gray TS (2021) Marine pollution in Kuwait and its impacts on fish-stock decline in Kuwaiti waters: reviewing the Kuwaiti government’s policies and practices. Front Sustain 2. https://doi.org/10.3389/frsus.2021.667822

Al Saif B (2020) Another Invasion of Kuwait: Corruption may represent an existential threat to the country, but it still remains widespread. Diwan. Malcolm H. Kerr Carnegie Middle East Center. https://carnegie-mec.org/diwan/82453

Al Sanad S (2015) Awareness, drivers, actions, and barriers of sustainable construction in Kuwait. Procedia Eng 118:969–983. https://doi.org/10.1016/j.proeng.2015.08.538

Al Sulaili A, AlSager B, Albanwan H, Almeer A, AlEssa L (2014) An integrated solid waste management system in Kuwait. Presented at the 5th International Conference on Environmental Science and Technology, IPCBEE vol. 69 (2014). IACSIT Press, Singapore, 10.7763/IPCBEE

Al Yaqout AF, Koushki PA, Hamoda MF (2002) Public opinion and siting solid waste landfills in Kuwait. Resour Conserv Recycl 35(4):215–227. https://doi.org/10.1016/s0921-3449(01)00111-2

Al Zuabi AZ (2016) Sociopolitical participation of Kuwaiti women in the development process: current state and challenges ahead. J Soc Serv Res 42(5):689–702. https://doi.org/10.1080/01488376.2016.1212775

Bilgin N (2006) Sosyal Bilimlerde İçerik Analizi: Teknikler ve Örnek Çalışmalar. Siyasal Kitabevi

Briens EC, Chiu Y, Braun D, Verma P, Fiegel G, Pompeii B, Singh K (2022) Assessing sustainability knowledge for undergraduate students in different academic programs and settings. Int J Sustain High Educ 24(1):69–95. https://doi.org/10.1108/ijshe-10-2021-0455

Brinton D, Snow, MA, Wesche MB (2003) Content-based second language instruction. University of Michigan Press ELT

Brundtland G (1987) Report of the World Commission on Environment and Development: our common future. United Nations General Assembly Document A/42/427

Buckler C, Creech H (2014) Shaping the future we want: UN decade of education for sustainable development; final report. UNESCO. https://sustainabledevelopment.un.org/content/documents/1682Shaping%20the%20

Bullock G, Wilder N (2016) The comprehensiveness of competing higher education sustainability assessments. Int J Sustain High Educ 17(3):282–304. https://doi.org/10.1108/ijshe-05-2014-0078

Burney NA, Johnes J, Al-Enezi M, Al-Musallam M (2013) The efficiency of public schools: the case of Kuwait. Educ Econ 21(4):360–379. https://doi.org/10.1080/09645292.2011.595580

Carley KM, Behrens D (1999) Organizational and individual decision-making. In Handbook of systems engineering and management. John Wiley and Sons. http://www.casos.cs.cmu.edu/publications/papers/ORGTHEO25.pdf

Chaplin G, Wyton P (2014) Student engagement with sustainability: understanding the value-action gap. Int J Sustain High Educ 15(4):404–417. https://doi.org/10.1108/IJSHE-04-2012-0029

Colantonio A (2007) Social sustainability: an exploratory analysis of its definition, assessment methods metrics and tools. EIBURS Working Paper Series (2007/01). Oxford Brooks University, Oxford Institute for Sustainable Development (OISD)—International Land Markets Group, Oxford, UK

Creswell JW, Clark VL (2011) Designing and conducting mixed methods research. SAGE

Creswell JW (2012) Educational research: planning, conducting, and evaluating quantitative and qualitative research. Pearson Education

Crosling G, Atherton G, Shuib M, Rahim AA, Azizan SN, & Nasir MI (2020) The teaching of sustainability in higher education: improving environmental resilience in Malaysia. Innovations in Higher Education Teaching and Learning, 17–38. https://doi.org/10.1108/s2055-364120200000022002

Crowe S, Cresswell K, Robertson A, Huby G, Avery A, Sheikh A (2011) The case study approach. BMC Med Res Methodol, 11(1). https://doi.org/10.1186/1471-2288-11-100

Çetinkaya G, Ozturk E, Ismail E (2015) Introducing environmental education in teaching foreign language to young learners. Int J Educ Res Rev 3(6):373–378. https://www.internationalscholarsjournals.com/articles/introducing-environmental-education-in-teaching-foreign-language-to-young-learners.pdf

Decamps A (2017) Analysis of determinants of a measure of sustainability literacy (2017/8). UNESCO, https://unesdoc.unesco.org/ark:/48223/pf0000259581

Decamps A, Barbat G, Carteron J-C, Hands V, Parkes C (2017) Sulitest: a collaborative initiative to support and assess sustainability literacy in higher education. Int J Manag Educ 15, 138–152. https://doi.org/10.1016/j.ijme.2017.02.006

Drayson R, Bone E, Agombar J, Kemp S (2014) Student attitudes towards and skills for sustainable development. The Higher Education Academy. Retrieved from www.heacademy.ac.uk

Du Pisani JA (2006) Sustainable development—historical roots of the concept. Environ Sci 3(2):83–96. https://doi.org/10.1080/15693430600688831

Eltony MN (2002) Can an oil-based economy be diversified? A case study of Kuwait. J Energy Dev 27(2):197–211. http://www.jstor.org/stable/24808708

Hauschild S, Poltavtchenko E, Stoller FL (2012) Going green: merging environmental education and language instruction. Engl Teach Forum 50(2):2–13. https://americanenglish.state.gov/files/ae/resource_files/50_2_3_hauschild-et-al.pdf

Kennedy TJ (2006) Language learning and its impact on the brain: connecting language learning with the mind through content-based instruction. Foreign Lang Ann 39(3):471–486. https://doi.org/10.1111/j.1944-9720.2006.tb02900.x

Koehn PH, Uitto JI (2017) Universities and the sustainable development future: evaluating higher-education contributions to the 2030 agenda. Routledge

Koushki PA, AlHumoud JM, AlDuaij U (2004) Municipal solid waste in Kuwait: trends and attitudes on collection, separation, and willingness to pay. Kuwait J Sci Eng 31(2):173–188

Kuckartz U, Rädiker S (2019) Analyzing qualitative data with MAXQDA: text, audio, and video. Springer, The USA, 10.1007/978-3-030-15671-8

Book   Google Scholar  

Kuehl C, Sparks AC, Hodges H, Smith ER (2023) Exploring sustainability literacy: developing and assessing a bottom-up measure of what students know about sustainability. Front Sustain 4. https://doi.org/10.3389/frsus.2023.1167041

Kuwait National Development Plan (2017) the Government of Kuwait. https://www.newkuwait.gov.kw/image/NewKuwait_CampaignLaunchEvent.pdf

Leal Filho W (2010) Universities and climate change: Introducing climate change to university programmes. Springer Science & Business Media

Maijala M, Heikkola LM, Kuusalu S-R, Laine P, Mutta M, Mäntylä K (2023) Pre-service language teachers’ perceptions of sustainability and its implementation in language teaching. Lang Teach Res , https://doi.org/10.1177/13621688231170682

Met M (1999) Content-based instruction: defining terms, making decisions. Johns Hopkins University, National Foreign Language Center. https://carla.umn.edu/cobaltt/modules/principles/decisions.html

Murad D, Al Awadhi L (2018) Report: the quality of education in Kuwait. The Cross-Cultural Diwaniya. https://static1.squarespace.com/static/5ae5dab125bf0240808df943/t/5baca382e4966bf0037ab2b0/1538040718718/Report+-+The+Quality+of+Education+in+Kuwait.pdf

Nur S, Anas I, Pilu R (2022) The call for environmentally based language teaching and green pedagogy: climate actions in language education. J Engl Lang Stud 4(1):77–85

Purvis B, Mao Y, Robinson D (2019) Three pillars of sustainability: In search of conceptual origins. Sustain Sci 14(3):681–695. https://doi.org/10.1007/s11625-018-0627-5

Owens D (2001) Composition and sustainability: teaching for a threatened generation. National Council of Teachers of English. https://eric.ed.gov/?id=ED458601

Rehn J (2018) Teaching teachers to teach sustainability—a cross-disciplinary course for integrating ESD in Higher Education. Global University Network for Innovation. https://www.guninetwork.org/articles/teaching-teachers-teach-sustainability-cross-disciplinary-course-integrating-esd-higher

Sachs J, Kroll C, Lafortune G, Fuller G, Woelm F (2022) Sustainable development report 2022. Cambridge University Press. https://doi.org/10.1017/9781009210058

Schneider K (2017) Evaluation of the opportunities for sustainability education through content-based learning in online German classes in Ecuador. Eur J Sustain Dev 6(3):373–382. https://doi.org/10.14207/ejsd.2017.v6n3p373

Shah Z, Kennedy-Clark S, Xie Y, Rahim MS, Mahdavi M, Levula A (2022) Teacher views on teaching sustainability in higher education institutes in Australia. Sustainability 14(14):8431. https://doi.org/10.3390/su14148431

Silverman D (2000) Doing qualitative research: a practical handbook. SAGE

Siregar R, Nuraida N, Kalsum EU (2022) Incorporating environment sustainability content in translation teaching through a task-based approach. J Linguist Lit Lang Teach 6(2):251–261. https://doi.org/10.30743/ll.v6i2.5669

Song B (2006) Content-based ESL instruction: long-term effects and outcomes. Engl Specif Purp 25(4):420–437. https://doi.org/10.1016/j.esp.2005.09.002

Stephens JC, Hernandez ME, Román M, Graham AC, Scholz RW (2008) Higher education as a change agent for sustainability in different cultures and contexts. Int J Sustain High Educ 9(3):317–338. https://doi.org/10.1108/14676370810885916

Stoller FL (2004) Content-based instruction: perspectives on curriculum planning. Annu Rev Appl Linguist 24:261–283. https://doi.org/10.1017/S0267190504000108

The Association for the Advancement of Sustainability in Higher Education (2023) AASHE’s suggested keywords for sustainability course and research inventories. Google Docs. https://docs.google.com/spreadsheets/d/1CINYGQ-8nov8qhGx_fQxn-KTSbC4BgpMHLsBgplaQGo/edit#gid=1359647972

The Sustainability Tracking, Assessment & Rating System (2018) University of Wisconsin-Oshkosh|Scorecard|Institutions|STARS reports. https://reports.aashe.org/institutions/university-of-wisconsin-oshkosh-wi/report/2018-01-29/

Turpin KM (2022) Multiliteracies pedagogy: theory to practice for scaffolding sustainability literacies. In Education for sustainable development in foreign language learning: content-based instruction in college-level curricula (pp 34–49). Routledge. https://doi.org/10.4324/9781003080183

Vorholt J (2018) New ways in teaching speaking (2nd ed) TESOL Press

World Economic Forum (2019) The global competitiveness report 2019. https://www3.weforum.org/docs/WEF_TheGlobalCompetitivenessReport2019.pdf

Yuan X, Zuo J (2013) A critical assessment of the higher education for sustainable development from students’ perspectives—a Chinese study. J Clean Prod 48:108–115. https://doi.org/10.1016/j.jclepro.2012.10.041

Zizka L, Varga P (2021) Teaching sustainability in higher education institutions: assessing hospitality students’ sustainability literacy. J Hosp Tour Educ 33(4):242–257. https://doi.org/10.1080/10963758.2020.1726771

Zwickle A, Jones K (2017) Sustainability knowledge and attitudes—assessing latent constructs. World Sustain Ser, 435–451. https://doi.org/10.1007/978-3-319-67122-2_25

Download references

Author information

Authors and affiliations.

Gulf University for Science and Technology, Mubarak Al-Abdullah, Kuwait

Inan Deniz Erguvan

You can also search for this author in PubMed   Google Scholar

Contributions

Inan Deniz Erguvan as the sole author of this manuscript wrote the literature review, collected and analyzed the data and produced the discussion section of the manuscript.

Corresponding author

Correspondence to Inan Deniz Erguvan .

Ethics declarations

Competing interests.

This study has been funded by Gulf University for Science and Technology, ISG case 4.

Ethical approval

The study used human subjects and the ethics approval from the Institutional Review Board (IRB) of Gulf University for Science and Technology was obtained prior to the application of the survey on students, with the registration number: IRB-278674/2023-24.

Informed consent

Informed consent was obtained from all participants. Students signed a consent form before they completed the data collection process.

Additional information

Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Cite this article.

Erguvan, I.D. What difference does one course make? Assessing the impact of content-based instruction on students’ sustainability literacy. Humanit Soc Sci Commun 11 , 708 (2024). https://doi.org/10.1057/s41599-024-03149-4

Download citation

Received : 07 September 2023

Accepted : 07 May 2024

Published : 31 May 2024

DOI : https://doi.org/10.1057/s41599-024-03149-4

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

Quick links

• Explore articles by subject
• Guide to authors
• Editorial policies

A framework and process for community-engaged, mixed-methods cancer needs assessments

• Original Paper
• Open access
• Published: 29 May 2024

Cite this article

You have full access to this open access article

• Todd Burus 1 , 12 ,
• Jessica R. Thompson 1 ,
• Caree R. McAfee 1 ,
• Lovoria B. Williams 1 , 2 ,
• Jennifer Redmond Knight 3 , 4 ,
• Bin Huang 5 , 6 ,
• Sarojini Kanotra 7 ,
• Natalie P. Wilhite 1 ,
• Elaine Russell 1 , 3 ,
• Melinda Rogers 8 ,
• Connie L. Sorrell 9 ,
• Christine Stroebel 1 ,
• Rachael King 10 &
• Pamela C. Hull 1 , 11  

38 Accesses

Explore all metrics

Community health needs assessments are required for most state and local public health agencies and non-profit hospitals. Typically based on community health improvement planning models, these assessments encompass overall community health and multiple diseases to inform program planning. National Cancer Institute (NCI)-designated Cancer Centers and community-based cancer-focused programs share the goal of reducing cancer burden in the catchment areas they serve. However, to date, no published models exist to guide cancer-specific needs assessments for a determined geographic area that can inform both public health and research initiatives. The purpose of this article is to outline a cancer needs assessment (CNA) framework and community-engaged, mixed-methods process, along with a case study of how we applied it in Kentucky.

We convened a steering committee of key organizational partners to provide input throughout the process. We developed a conceptual framework of multi-level determinants affecting cancer-related outcomes. We incorporated both quantitative and qualitative data gathered through a variety of means, including a novel application of group concept mapping to guide definition of priorities.

The resulting CNA has helped guide strategic planning and priorities for Kentucky’s Cancer Action Plan, Markey Cancer Center, state agencies, and community-based organizations.

This framework and process can be used collaboratively by cancer center Community Outreach and Engagement offices, public health agencies, oncology programs, and community partners to plan impactful cancer control programs and research in their catchment areas. Universities can also use them to inform the planning of community engagement and health equity research efforts.

Avoid common mistakes on your manuscript.

Introduction

Needs assessments arose in the mid-twentieth century as a useful tool for performing program planning and evaluation. Their purpose is to, first, identify needs (the gap between the current and future desired conditions) and then prioritize how to address them [ 1 ]. State and local public health agencies were early adopters of community health needs assessments (CHNA), and since 2011, the national Public Health Accreditation Board has required CHNAs [ 2 ]. In recent years, needs assessments have taken a more prominent role in healthcare with the requirement that nonprofit hospitals perform a CHNA every 3 years in accordance with the Patient Protection and Affordable Care Act (ACA) of 2010 [ 3 ]. Hospitals are also required to develop a corresponding implementation plan to act on priorities identified through their CHNAs. Combined, this process is meant as an accountability measure for facilities receiving federal funding, pushing them to focus on health outcome improvements for individuals living within their service areas. In 2012 the Commission on Cancer (CoC) under the American College of Surgeons initiated new requirements for CoC-accredited oncology treatment facilities to perform a CHNA once every 3 years [ 4 ]. While the 2020 CoC standards no longer require CHNAs, their implementation is encouraged to guide outreach and psychosocial programs that address barriers to cancer care [ 5 ].

To promote local accountability and community benefit, in 2013, the National Cancer Institute (NCI) introduced the concept of catchment areas in the Cancer Center Support Grant (CCSG) funding requirements for NCI-Designated Cancer Centers. A cancer center catchment area is a population- or geographically based area in which the cancer center does or desires to serve patients and perform research to reduce the cancer burden [ 6 ]. The subsequent 2016 CCSG guidelines expanded on prior requirements for cancer prevention and control by establishing a new Community Outreach and Engagement (COE) component [ 7 ]. The COE component was charged with continuously generating a comprehensive profile of the catchment area’s unique cancer needs (i.e., factors influencing poor cancer-related health outcomes and disparities) and opportunities for improving them. This profile should catalyze both cancer control activities and basic, clinical, translational, and population research priorities to address identified needs in collaboration with community partners [ 8 ]. Conducting a needs assessment can help cancer center COE offices accomplish these objectives [ 9 , 10 ].

Public health agencies typically follow CHNA processes outlined by community health improvement planning (CHIP) models, which emphasize involving local residents and partner organizations to assess community health status; identify health priorities; and plan, implement, and evaluate city- or statewide health improvement initiatives [ 11 , 12 , 13 ]. In contrast, the non-profit hospital CHNA requirement did not come with much guidance and, as such, has produced varied results during its first decade of existence [ 14 ]. Furthermore, CHIP-based CHNA processes are designed to encompass the entire spectrum of community health and select priorities among multiple diseases, with a narrow focus on planning public health agency programs. Additionally, the National Comprehensive Cancer Control Program requires funded states, tribes and territories and their cancer coalitions to review cancer-related data and develop a jurisdiction-wide cancer plan [ 15 ]. No formal process is in place to fulfill this requirement, and it varies considerably among states and jurisdictions [ 16 ]. To date, no published models exist to guide cancer-specific needs assessments that can inform public health initiatives, cancer plans, and research.

To fill this gap, the objectives of this article are to: (1) present a cancer needs assessment (CNA) framework and community-engaged, mixed-methods process to guide the identification of priorities for both community-based cancer control activities and research agendas aimed at reducing cancer burden and disparities in a specific catchment area, and (2) illustrate a case study of how we applied the CNA framework and process in Kentucky through a partnership among the University of Kentucky Markey Cancer Center (UKMCC) COE team and a steering committee comprised of academic, public health, and community partners. While separate articles will detail the methods and results of specific components of the Kentucky CNA in greater depth, this article provides a high-level overview of how to conduct a community-engaged, multi-method CNA that integrates various sources and types of data.

Conceptual framework

We drew on several existing models to develop a conceptual framework of social and individual factors that influence cancer outcomes to guide the collection, analysis, interpretation, and presentation of data in Cancer Needs Assessments (CNA). For example, Rodriguez et al. adapted McLeroy’s Social-Ecological Model and illustrated how nested interactions at the patient, community, and policy levels impact patient health outcomes and health disparities [ 17 ]. Hiatt and Breen went a step further by considering the direct influence different levels of analysis, including social determinants of health (SDOH), have on the cancer care continuum, and where opportunities for intervention exist [ 18 , 19 ]. Additionally, the American Association for Cancer Research put forward a model which recognized the roles both personal and population evidence-based actions can play in effective cancer prevention [ 20 ]. Finally, Alcaraz et al. created a framework for advancing cancer health equity through understanding and addressing SDOH [ 21 , 22 ]. The Alcaraz et al. model introduced the idea of an upstream/downstream intervention orientation in which they argue that “[to] achieve cancer health equity, more focused efforts are needed upstream to address social factors for population-level impact.”

The UKMCC COE synthesized these concepts to develop a draft CNA conceptual framework, the Multilevel Determinants of Cancer-related Outcomes Framework, and the steering committee gave input to refine it (Fig.  1 ). This framework organizes several intertwined levels of consideration for assessing cancer needs and identifying potential interventions to reduce cancer burden and disparities. On the left, the SDOH reflect the overall context in which an individual lives. We classified SDOH as falling into three levels—society, environment, and community (see Fig.  1 ), which influence one another in a complex, circular manner over time [ 23 ]. In the middle, under individual-level factors, one’s personal characteristics and behaviors interact with their body’s internal biological mechanisms, which could potentially create somatic mutations and result in cancer. The complex interplay of contextual SDOH factors and individual factors interact with cancer-related outcomes along the cancer care continuum, shown on the right. On the bottom left, the green to red continuum indicates varying degrees of health equity in contextual SDOH factors for a given geographic area, which influence the risk of exposures and behaviors for individuals in different social groups (e.g., gender, race/ethnicity, rural/urban). The degree of health equity in SDOH impacts the degree of health disparities in population-level cancer-related outcomes across social groups, as indicated on the bottom right. Along the top of the figure, we indicate the continuum of upstream actions directed toward SDOH to downstream actions targeting the individual level.

Multilevel determinants of cancer-related outcomes framework. Society includes social context (e.g., culture; social norms; the meaning of socially constructed concepts like race, ethnicity, and gender), economic conditions (e.g., median income, unemployment), institutions (e.g., education system, criminal justice system), and policies (e.g., Affordable Care Act, Medicaid, institutional policies). Environment includes the “natural environment” (e.g., water, air, soil) and substances (natural or man-made) that individuals are exposed to through these means, and the “built” environment constructed by humans (e.g., roads, sidewalks, parks, buildings). Community includes the local health care system (e.g., primary care, hospitals, cancer clinics, and other facilities) and interpersonal relationships (e.g., family, neighborhoods)

Cancer needs assessment mixed method process

Mixed methods research brings together the strengths of quantitative breadth and qualitative depth of data within the same study to gain greater insights [ 24 ]. In needs assessments, mixed methods provide a comprehensive picture of community health needs and issues, adding lived experience to identified quantitative patterns [ 25 ]. Considering the availability of existing cancer-related quantitative data and important gaps that need to be filled with qualitative data, we recommend the process illustrated in Fig.  2 , which represents a community-engaged, explanatory sequential mixed methods approach. The first phase of the CNA process includes assessment of patterns in existing quantitative data plus simultaneous gathering of qualitative community insights through focus groups and review of hospital CHNAs. While extensive secondary quantitative data exists in publicly available sources, groups may also choose to perform additional primary quantitative data collection (e.g., population health surveys), if they have sufficient financial and personnel resources [ 8 , 26 ]. The second phase is a group concept mapping process with quantitative and qualitative steps to generate consensus on priorities. The final phase involves summarizing and disseminating CNA findings. A CNA steering committee of key stakeholders should ideally provide input throughout all phases. Below we describe the methodology of each component and illustrate the case study of how we applied this process for the Kentucky CNA (KY CNA).

Process workflow for performing a community-engaged, mixed-methods cancer needs assessment

Community partner input

An essential preliminary step for a community-engaged CNA process is identifying a collaborative steering committee, which could be a new or existing committee, task force, or community advisory board. This group is typically comprised of key stakeholders within the catchment area representing a variety of points of view, such as local non-profit organizations, government agencies, universities, healthcare providers, policy makers, and other community leaders or members [ 25 ]. The committee’s responsibility is to provide input throughout the CNA process on design, recruitment, data collection, interpretation of findings, and dissemination. This collaborative approach grounds the CNA process in existing community insights and assets, ensures representation, and builds credibility and investment in the results among stakeholder organizations.

Given that the catchment area for the 2021 KY CNA encompassed the entire state, the UKMCC COE formed a KY CNA steering committee consisting of organizations and programs with a statewide reach, a focus that includes cancer, access to data sources or networks of organizations and community members, and a common interest in a statewide CNA. The 27-member steering committee included representatives of the UKMCC, Kentucky Cancer Consortium (KCC, the state cancer coalition), Kentucky Cancer Program (KCP, a statewide outreach program), Kentucky Cancer Registry (KCR), Kentucky Department for Public Health, American Cancer Society, Foundation for a Healthy Kentucky, and University of Louisville. From March 2021 to March 2022, through all steps of the process, UKMCC COE convened monthly steering committee meetings and also sought their input through email. In addition, UKMCC COE regularly updated and gathered feedback during the process from the UKMCC Community Advisory Board, comprised of 12 lay community members and community organizations.

Quantitative methods

In performing a CNA, the quantitative methods provide a numerical view of the target catchment area. This assists in, among other things, measuring the observed degree of cancer burden, inequities in SDOH, and disparities in cancer-related outcomes that exists among certain population subgroups. For the 2021 KY CNA we chose to only gather secondary quantitative data due to practical considerations surrounding the large size of our catchment area, the ongoing COVID-19 pandemic at the time, and to focus available resources on gathering needed qualitative data.

Gathering secondary data

While developing the conceptual framework for the KY CNA, we began to brainstorm available quantitative data from secondary sources that correspond to components of the framework. We collected these ideas in an indicator list with notes about available sources and the additional demographic information each contained. Subsequently, we organized indicators based on the conceptual framework. Throughout the needs assessment process, we regularly revisited this list. By the end, this list contained over 100 quantitative variables to consider beyond just cancer incidence and mortality (Suppl. Table S1 ). We compiled this data, developed summaries, and analyzed it for the presence of significant disparities between various subgroups of the population. In a parallel project, the UKMCC COE streamlined the process of gathering of multiple publicly available data sources through the creation of a catchment area data collection software called Cancer InFocus [ 26 ].

Geographic levels

Given that the scope of our needs assessment was the entire state, we processed all of the data collected at the state, county, and/or census tract level (as available) to aid in creating geographic visualizations. Kentucky has a unique level of administrative boundaries utilized in this process known as Area Development Districts (ADDs). ADDs combine several counties into larger sub-state regions, making it possible to report on certain variables that would otherwise need to be suppressed at the smaller county level. Depending on the makeup of the area being assessed, it may be useful to collect available data for other geographic levels (such as public use microdata areas) or non-standard administrative regions.

Visualizations

We constructed numerous tables, charts, and maps with the quantitative data we collected. We shared these visualizations with our steering committee during monthly meetings to get feedback on ease of comprehension and how to best capture the story being told by the data. Consistent colors were used across visualizations to assist in developing a visual narrative that was easy to follow with minimal need for written text in the final document.

Dissemination follow-up survey

To evaluate the reach and impact of our CNA, a follow-up survey was sent to individuals who volunteered to be recontacted when downloading a digital copy of resources from the 2021 KY CNA website ( www.kycancerneeds.org ). The survey was constructed in REDCap and consisted of multiple questions intended to assess the usage of needs assessment findings throughout the state [ 27 ].

Data processing

Data processing was performed in the R statistical programming language, and visualizations were created in R (version 4.2.1, R Core Team, Vienna, Austria), Tableau (version 2021.3, Tableau, Seattle, WA), and ArcGIS Pro (version 2.9, Environmental Systems Research Institute, Redlands, California). When formal statistical tests were performed to compare values, a p -value of 0.05 was used to assess statistical significance. Otherwise, statistically significant differences were noted using non-overlapping 95% confidence intervals.

Qualitative methods

The qualitative aspect of a mixed methods CNA allows researchers to examine the perceived needs in the community and add additional context to the inequities and disparities observed using quantitative methods. Importantly, qualitative methods allow for the inclusion of voices in needs assessments from underrepresented portions of the population whose experiences may not be captured by solely relying on quantitative population-level data [ 25 ]. Additionally, through open-ended questions, qualitative data can provide details on the ‘how’ and ‘why’ certain quantitative trends exist in order to develop the community-specific strategies necessary to address identified needs [ 24 ].

Hospital community health needs assessments scoping review

We conducted a scoping review of hospital CHNAs from all non-profit hospitals across the state of Kentucky to improve our understanding of statewide priority health needs and implementation strategies and to identify where priority needs aligned with the cancer care continuum. The CHNA review process included: (1) Conducting a literature review of the CHNA process; (2) Collecting the most recent CHNA and implementation strategy reports for Kentucky hospitals; (3) Documenting information about hospital catchment areas from CHNAs and building hospital profiles; (4) Building a glossary of terms and definitions, and grouping them into categories under priority health needs and implementation strategies; (5) Training staff on how to review and code priority health needs and implementation; (6) Performing two reviews per CHNA; and (7) Reconciling discrepancies in categorization between initial reviews with a third reviewer.

Focus groups

To incorporate more community participation beyond our steering committee, we conducted online focus groups with adult (age 18+) residents of Kentucky who did not work in a healthcare profession ( n  = 51). From July to September 2021, we recruited participants utilizing our existing KCC and KCP partnership networks and ResearchMatch, an online volunteer research registry [ 28 ]. Our study team contacted eligible participants, gathered study consent, and assigned participants to a specific scheduled focus group. As we sought to identify needs by race and ethnicity, geographic area, and sexual orientation and gender identity, we intentionally stratified groups by these characteristics based on a brief demographic questionnaire, resulting in 11 focus group sessions.

The focus group discussions lasted approximately two hours and took place via Zoom. Discussions were led by a trained facilitator and supported by a research assistant. Our semi-structured questions focused on participant awareness of existing resources and needs across the cancer care continuum. We supported this discussion using a visual graphic displaying the various areas of the cancer care continuum (risk reduction, screening, treatment, and follow-up/survivorship). Participants received a $50 e-gift card for their participation. Each focus group discussion was recorded and transcribed for qualitative coding analysis. The facilitator and research assistant conducted a line-by-line review of the transcripts and double-coded 20% of the transcripts, resulting in over 90% agreement. Any discrepancies were resolved through consensus.

Concept mapping

After focus group data collection, we designed a concept mapping project to prioritize the wide variety of identified cancer needs and strategies. A participatory mixed method, concept mapping utilizes a series of survey-based activities and group discussions to generate consensus on a particular topic of interest [ 29 , 30 , 31 ]. From September to December 2021, we recruited community members (adults, non-health professionals, and Kentucky residents) and community organization employees who work in cancer services in Kentucky to participate in concept mapping remotely ( n  = 162). The community member participants were individuals who previously participated in the KY CNA focus groups. Community organizations were identified by KCC and KCP and included representatives from health departments, nonprofit organizations, advocacy groups, insurance companies, health systems, and educational organizations.

We recruited participants by email and provided a link to the online concept mapping activities utilizing the Groupwisdom concept mapping platform [ 32 ]. Prior to beginning the first set of activities, a working group of COE, KCP, and KCC members compiled a list of 80 items from the previous quantitative findings, qualitative focus group discussion themes, and concerns raised at partner meetings. The survey-based activities asked participants to sort these 80 items into thematic groups and to rate each item on two five-point Likert-type scales: (1) importance for Kentucky communities and (2) how easy it would be to address in Kentucky communities (i.e., feasibility). With these data, the study team generated concept maps (e.g., point and cluster maps) using cluster analysis and rating assessments (e.g., correlational comparisons), which were shown to participants in group discussions. We conducted six discussion groups via Zoom, including three with community members and three with organizational partners. The community member participants received up to $60 in e-gift cards for their participation ($30 for the online survey-based activities and $30 for the group discussion).

Dissemination

We used a multi-pronged strategy for disseminating the KY CNA report and district profiles across Kentucky. This approach included distributing physical copies of the summary report and district profiles and creating a website with the digital summary report, district profiles, and interactive data dashboard. In addition, we hosted meetings and webinars on the findings of the report and how they could be used to inform research, guide strategic planning and outreach, and impact the community.

Institutional review board statement

UK Institutional Review Board (IRB) approved the research procedures under three protocols: Kentucky Behavioral Risk Factor Survey analysis (#69894), focus groups (#65451), and concept mapping (#73420). The IRB issued a non-human research determination for using the publicly available, aggregated secondary data and the CNA dissemination follow-up survey.

Quantitative results

We gathered secondary data from over 18 different sources including KCR, a population-based central cancer registry for the Commonwealth of Kentucky and member of the NCI Surveillance, Epidemiology, and End Results program since 2000 (Table  1 ). KCR provided cancer incidence and mortality data for 2014–2018 for all primary cancer sites stratified by combinations of sex, race, rurality, Appalachian residence, and county or ADD. They also calculated new combined incidence and mortality rates for three groups of cancer sites associated with the major cancer risk factors of tobacco, obesity, and human papillomavirus (HPV) [ 33 , 34 , 35 ]. Twelve of the top 20 incidence rate cancers, and 12 of the top 20 mortality rate cancers, were observed to have significantly higher rates in Kentucky than in the rest of the U.S. (Suppl. Table S2; Fig. S1 ). Of particular interest, we found that Kentucky lung cancer incidence rates were 78.7% higher than U.S. rates, and lung cancer mortality rates were 81.9% higher. Kentucky also experienced significantly higher rates in incidence and mortality for the groupings of tobacco-, obesity-, and HPV-related cancers.

Likewise, we noted significant disparities among several behavioral risk factors and socioeconomic factors, though Kentucky’s rates for cancer screening were comparable to U.S. rates (Suppl. Table S3; Fig. S2). KCR provided relative survival rates for cancer patients for the years of 2012–2018, defined as the percentage of patients with a particular cancer diagnosis surviving 5-years after being diagnosed compared to a similar population of individuals without a cancer diagnosis. This revealed that Black Kentucky females with a breast cancer diagnosis had a significant 10% lower 5-year survival rate than White Kentucky females.

Qualitative results

Community health needs assessments scoping review.

One-hundred and ten non-profit hospitals in the state of Kentucky met the criteria for needing to have performed a CHNA at the time of our review. Research staff successfully located CHNAs for 73 of these hospitals from websites and requesting copies. We analyzed these CHNAs for content on their strategic priorities and implementation strategies (Fig.  3 ). While 60.3% of them noted tobacco/smoking cessation as a strategic priority, only about a quarter (27.4%) specifically highlighted cancer. Lung cancer screening was a priority for 24.7% of hospitals reviewed, breast cancer screening for 13.7%, and colorectal cancer screening for 12.3% (no hospital highlighted a priority for cervical cancer screening despite Kentucky ranking first among U.S. states in cervical cancer incidence). Mention of focusing on SDOHs appeared on 28.8% of CHNAs reviewed, while health equity was only found in 1 of the 73.

Bar chart showing the results of the Kentucky non-profit hospitals Community Health Needs Assessments (CHNA) review, indicating the percentage of CHNAs that include these items as a priority and/or strategy

Using an online format, we uncovered a wide variety of factors affecting Kentuckians across the cancer care continuum. For example, continued novel approaches are needed to make information accessible and to utilize messaging that will not be interpreted as blaming or shaming. Likewise, screening efforts need to continue reaching individuals where they are and include messaging from individuals who engender trust. Continued efforts are also needed to address practical concerns for both screening and treatment, such as cost and transportation as well as lack of knowledge of which screening tools and treatments are covered by insurance. Participants expressed unique concerns based on their race and ethnicity, rurality, sexual orientation and gender identity, and age. These differences suggest uncovering ways to promote positive, understanding communication between patients and providers and to create safe care spaces that consider ways cultural norms affect cancer care to fight stigma and to improve health equity [ 36 ].

Concept mapping results

The concept mapping process successfully grouped the 80 items, which span factors across the cancer care continuum, into six thematic clusters. We also identified potential community-driven strategies to improve cancer risk reduction, treatment, and follow-up in Kentucky (Fig.  4 ). Specifically, participants commonly identified key areas for continued efforts, such as lung cancer screening, smoking cessation, HPV vaccination, and disparities driven by social determinants among rural, Appalachian, Black, and Hispanic Kentuckians. Community member and partner-driven strategies to affect these areas include a continued focus on health communication strategies, supports for treatment navigation, ways to overcome barriers to accessing care, and methods for increasing trust in patient–provider relationships. Moving forward, healthcare professionals dedicated to improving cancer in Kentucky can consider ways to build upon these strategies.

Community prioritization of needs and strategies from the 2021 Kentucky Cancer Needs Assessment

Final synthesis

Results from the quantitative and qualitative data collection were synthesized into a final KY CNA report entitled, “2021 Kentucky Cancer Needs Assessment: The Story of Cancer in Kentucky.” This 59-page report was broken down into five sections, including an executive summary of findings. Emphasis was placed on presenting information in a visual format with minimal prose where possible. Additional one-page district profiles were created using data from the report for each of Kentucky’s 15 ADDs.

Dissemination results

The KY CNA website ( www.kycancerneeds.org ) launched in April 2022 with a downloadable version of the summary report, along with two media toolkits breaking down the report visualizations into individual images for use in presentations and grant proposals. To facilitate tracking, we asked users to complete a request form to access the downloads, with the option of sharing contact information for a follow-up survey to be sent approximately 2 months later. Between April 2022 and January 2023, 304 people submitted the request form and 650+ report documents were downloaded. We sent 148 follow-up surveys. Among respondents ( n  = 64), the top uses of the CNA reported were to guide program and strategic planning (48.4%), to inform grant applications (29.7%), and using the web portal to get local data (23.4%) (Fig.  5 ). Use of the KY CNA was spread broadly across individuals in healthcare professions, university settings, state and local government, and cancer-focused non-profit organizations.

Bar chart showing how individuals used the 2021 Kentucky Cancer Needs Assessment ( n  = 64)

In addition, 750 physical copies of the KY CNA report were distributed throughout Kentucky, including 138 given to the current members of Kentucky’s state legislature. Over 2,000 copies of the district profiles were printed for dissemination by KCP Regional Cancer Control Specialists to local partners. Report findings were shared by UKMCC COE staff to over 800+ persons across 26 oral presentations.

We sought to outline a conceptual framework and process that cancer centers, cancer coalitions and other organizations can follow to perform community-engaged CNAs in their catchment areas. In particular, this project provided important methodological insight on how to triangulate quantitative and qualitative data and gather community feedback on setting research and outreach priorities. Applying this iterative process in the KY CNA, we were able to incorporate the perspectives of diverse community members across the state. Moreover, the contributions from focus groups, concept mapping, and our steering committee helped ensure that we could maximally assess the assets, opportunities, and barriers for cancer prevention and control. The KY CNA represented a significant advancement in understanding the story of cancer in Kentucky and laid the groundwork for future efforts to reduce the cancer burden in Kentucky.

Previous research has highlighted the importance of engaging community stakeholders in needs assessments and identification of priorities [ 12 , 37 ]. However, in a review of CHNAs among Texas non-profit hospitals, Pennel et al. observed that 18% made no attempt to engage community members, and only four out of 76 involved community members in strategy selection [ 14 ]. Within the KY CNA steering committee, the Kentucky Cancer Program—which already operates within Kentucky communities—played an important role in facilitating the involvement of lay community members. These individuals represented key population subgroups in the state and provided crucial feedback in setting priorities and strategies—reflecting an “interpretive approach” to defining community as highlighted by Franz et al. [ 38 ]. Although a simple look at incidence and mortality rates reveals the disproportionate burden of cancer in Kentucky, the KY CNA findings illustrate that the causes of this problem and the avenues for addressing it require a much deeper analysis. This finding supports the argument of Pennel et al. that grappling with the broader social determinants of health is necessary to achieve a beneficial impact on cancer outcomes and the improvement of health equity [ 39 ].

Our CNA process also adds several methodological contributions to the identification and prioritization of catchment area needs through community-engaged and mixed method approaches. Specifically, our novel approach includes: (1) the use of virtual focus groups to engage a wide-range of community members throughout our catchment area, including oversampling populations most at risk (e.g., rural, Black, sexual, and gender minorities); (2) the employment of concept mapping with both community members and statewide organizational partners to prioritize the wide array of identified cancer needs in our catchment area; and (3) the combination of several quantitative and qualitative data sources to ultimately inform new strategic plans, both across our catchment area and for local healthcare systems and communities. The CNA results were subsequently used to inform a new Kentucky Cancer Action Plan. Our use of concept mapping provided an opportunity for resulting strategic plans to incorporate the practical considerations of local organizational partners and the lived experiences of those in the community. Although concept mapping has been utilized for program planning, evaluation, and community needs assessments previously, to our knowledge this is the first use of the approach by a cancer center in a cancer-focused needs assessment capturing needs in a statewide catchment area [ 31 , 40 ]. Similar to other research conducted during the COVID-19 pandemic, we successfully adapted to pandemic restrictions by using virtual data collection for both the focus group and concept mapping elements [ 41 , 42 ]. Our ability to capture perspectives from diverse lay community members in over 40 different Kentucky counties suggests that virtual qualitative data collection is a viable CNA method for cancer centers and other organizations seeking to broadly capture needs in diverse and/or large catchment areas.

Although this project makes important contributions to the literature, it contains certain limitations that we acknowledge. The available secondary quantitative data sources to assess cancer rates, social determinants of health, and behavioral risk factors have varying time delays. To moderate this impact, we used data from the most recent years available for each source. Additionally, our analysis is subject to all of the limitations that come with the available secondary sources—in particular, lack of information about sexual orientation, gender identity, and disabilities. Though collecting new (primary) survey data on individuals in our catchment area could have potentially contributed additional quantitative insights unavailable in the secondary sources, the large size of our catchment area and cost made doing so impractical. For focus groups and concept mapping, the steering committee expressed concerns regarding technological literacy among participants due to the use of virtual formats. We sought to address this issue by asking potential participants about their comfort level with the technologies used and, when requested, providing them trainings to increase participation. Our goal was to broadly capture diverse perspectives across the state, which we achieved. However, although we oversampled vulnerable populations, we did not include a sufficient number of participants from individual groups to form specific recommendations by demographic group. The qualitative methods used in this project were not intended to be generalizable, but were designed with the intent of gaining depth of understanding on needs and general consensus on priorities with respect to cancer specifically. Finally, we did not gather extensive process metrics beyond dissemination data to evaluate the process. However, the steering committee expressed satisfaction with the process throughout each phase. Future applications of this framework and process could gather additional process metrics to demonstrate efficacy.

Performing periodic CNAs should be adopted as standard practice for cancer center COE offices and other organizations wishing to improve impact within their catchment area. The CNA framework aids in understanding the multilevel determinants of cancer-related outcomes to guide CNA planning and interpretation of findings. The CNA process employs a mixed-methods design with a variety of community input throughout to complement the quantitative data with people’s lived experiences. The resulting KY CNA report and dissemination not only informed the cancer center’s institutional research and strategic efforts, but also the efforts of state-level and community-based cancer organizations across Kentucky. This CNA framework and process can be replicated by COE offices at NCI-Designated Cancer Centers, public health agencies, cancer coalitions, oncology programs and community organizations. In addition, they can be used by universities for institution-level efforts focused on community engagement, health equity, and cancer research that spans across multiple research projects.

Data availability

Inquiries about data availability should be directed to the authors.

Altschuld JW, Watkins R (2014) A primer on needs assessment: more than 40 years of research and practice: a primer on needs assessment: more than 40 years of research and practice. New Dir Eval 2014:5–18. https://doi.org/10.1002/ev.20099

Article   Google Scholar  

Public Health Accreditation Board Standards & Measures for Initial Accreditation, Version 2022

Community Health Needs Assessment for Charitable Hospital Organizations—Section 501(r)(3). https://www.irs.gov/charities-non-profits/community-health-needs-assessment-for-charitable-hospital-organizations-section-501r3

Commission on Cancer, American College of Surgeons. Cancer Programs Standards 2012: Ensuring Patient-Centered Care. Version 1.2.1. https://apos-society.org/wp-content/uploads/2016/06/CoCStandards-1.pdf . Published 2012. Accessed 14 Sep 2023.

The GW Cancer Center. Implementing the Commission on Cancer Standard 8: Addressing Barriers to Care. https://cancercontroltap.smhs.gwu.edu/sites/g/files/zaskib661/files/2021-09/30%20-%20CoC%20Standard%208.1%20Road%20Map%20FINAL%20508.pdf . Published 2021. Accessed 12 Sep 2023.

Cancer Center Support Grants (CCSGs) for NCI-designated Cancer Centers. https://grants.nih.gov/grants/guide/pa-files/PAR-21-321.html

Paskett ED, Hiatt RA (2018) Catchment areas and community outreach and engagement: the new mandate for NCI-designated cancer centers. Cancer Epidemiol Biomark Prev 27:517–519. https://doi.org/10.1158/1055-9965.EPI-17-1050

Tai CG, Hiatt RA (2017) The population burden of cancer: research driven by the catchment area of a cancer center. Epidemiol Rev 39:108–122. https://doi.org/10.1093/epirev/mxx001

Article   PubMed   Google Scholar  

Hiatt RA, Kobetz EN, Paskett ED (2022) Catchment areas, community outreach and engagement revisited: the 2021 guidelines for cancer center support grants from the National Cancer Institute. Cancer Prev Res 15:349–354. https://doi.org/10.1158/1940-6207.CAPR-22-0034

Vadaparampil ST, Tiro JA (2022) Catchment area: an opportunity for collective impact, strategic collaboration, and complementary focus. Cancer Epidemiol Biomark Prev 31:952–954. https://doi.org/10.1158/1055-9965.EPI-22-0173

Durch J, Bailey LA, Stoto MA (1997) Improving health in the community: a role for performance monitoring. National Academy Press, Washington, D.C.

Google Scholar  

Pennel CL, Burdine JN, Prochaska JD, McLeroy KR (2017) Common and critical components among community health assessment and community health improvement planning models. J Public Health Manag Pract 23:S14–S21. https://doi.org/10.1097/PHH.0000000000000588

Burdine JN, Felix MRJ (2017) A personal history of the evolution of health status assessment. J Public Health Manag Pract 23:S9–S13. https://doi.org/10.1097/PHH.0000000000000577

Pennel CL, McLeroy KR, Burdine JN, Matarrita-Cascante D, Wang J (2016) Community health needs assessment: potential for population health improvement. Popul Health Manag 19:178–186. https://doi.org/10.1089/pop.2015.0075

National Comprehensive Cancer Control Program. https://www.cdc.gov/cancer/ncccp/index.htm . Accessed 27 Sept 2023

Hager P, Given L, Redmond J, Rogers K (2010) Revision of comprehensive cancer control plans: experiences shared by three states. Cancer Causes Control 21:2005–2013. https://doi.org/10.1007/s10552-010-9662-y

Rodriguez SD, Vanderford NL, Huang B, Vanderpool RC (2018) A social-ecological review of cancer disparities in Kentucky. South Med J 111:213–219. https://doi.org/10.14423/SMJ.0000000000000794

Article   PubMed   PubMed Central   Google Scholar  

Hiatt RA, Breen N (2008) The social determinants of cancer. Am J Prev Med 35:S141–S150. https://doi.org/10.1016/j.amepre.2008.05.006

Institute of Medicine (2013) Delivering high-quality cancer care: charting a new course for a system in crisis. The National Academies Press, Washington, D.C.

Lippman SM, Abate-Shen C, Colbert Maresso KL, Colditz GA, Dannenberg AJ, Davidson NE, Disis ML, DuBois RN, Szabo E, Giuliano AR, Hait WN, Lee JJ, Kensler TW, Kramer BS, Limburg P, Maitra A, Martinez ME, Rebbeck TR, Schmitz KH, Vilar E, Hawk ET (2018) AACR white paper: shaping the future of cancer prevention—a roadmap for advancing science and public health. Cancer Prev Res 11:735–778. https://doi.org/10.1158/1940-6207.CAPR-18-0421

Alcaraz KI, Wiedt TL, Daniels EC, Yabroff KR, Guerra CE, Wender RC (2020) Understanding and addressing social determinants to advance cancer health equity in the United States: a blueprint for practice, research, and policy. CA Cancer J Clin 70:31–46. https://doi.org/10.3322/caac.21586

Warnecke RB, Oh A, Breen N, Gehlert S, Paskett E, Tucker KL, Lurie N, Rebbeck T, Goodwin J, Flack J, Srinivasan S, Kerner J, Heurtin-Roberts S, Abeles R, Tyson FL, Patmios G, Hiatt RA (2008) Approaching health disparities from a population perspective: the National Institutes of Health Centers for Population Health and Health Disparities. Am J Public Health 98:1608–1615. https://doi.org/10.2105/AJPH.2006.102525

Portrait of Promise: The California Statewide Plan to Promote Health and Mental Health Equity

Tariq S, Woodman J (2013) Using mixed methods in health research. JRSM Short Rep 4:204253331347919. https://doi.org/10.1177/2042533313479197

Ravaghi H, Guisset A-L, Elfeky S, Nasir N, Khani S, Ahmadnezhad E, Abdi Z (2023) A scoping review of community health needs and assets assessment: concepts, rationale, tools and uses. BMC Health Serv Res 23:44. https://doi.org/10.1186/s12913-022-08983-3

Burus JT, Park L, McAfee CR, Wilhite NP, Hull PC (2023) Cancer InFocus: tools for cancer center catchment area geographic data collection and visualization. Cancer Epidemiol Biomark Prev 32:889–893. https://doi.org/10.1158/1055-9965.EPI-22-1319

Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG (2009) Research electronic data capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform 42:377–381. https://doi.org/10.1016/j.jbi.2008.08.010

Harris PA, Scott KW, Lebo L, Hassan N, Lightner C, Pulley J (2012) ResearchMatch: a national registry to recruit volunteers for clinical research. Acad Med 87:66–73. https://doi.org/10.1097/ACM.0b013e31823ab7d2

Burke JG, O’Campo P, Peak GL, Gielen AC, McDonnell KA, Trochim WMK (2005) An introduction to concept mapping as a participatory public health research method. Qual Health Res 15:1392–1410. https://doi.org/10.1177/1049732305278876

Kan M, Trochim WMK (2007) Concept mapping for planning and evaluation. Sage Publications, Inc.

Thompson JR, Burke JG (2020) Increasing community participation in public health research: applications for concept mapping methodology. Prog Community Health Partnersh 14:243–250. https://doi.org/10.1353/cpr.2020.0025

The Concept System® groupwisdom™ (Build 2022.30.10) [Web-based Platform]. (2022). Ithaca, NY. Available from https://www.groupwisdom.tech .

IARC Working Group on the Evaluation of Carcinogenic Risks to Humans, World Health Organization, International Agency for Research on Cancer (2010) Alcohol consumption and ethyl carbamate. International Agency for Research on Cancer; Distributed by WHO Press, Lyon, France: Geneva

Viens LJ, Henley SJ, Watson M, Markowitz LE, Thomas CC, Thompson TD, Razzaghi H, Saraiya M (2016) Human papillomavirus-associated cancers—United States, 2008–2012. MMWR Morb Mortal Wkly Rep 65:661–666. https://doi.org/10.15585/mmwr.mm6526a1

Gallaway MS, Henley SJ, Steele CB, Momin B, Thomas CC, Jamal A, Trivers KF, Singh SD, Stewart SL (2018) Surveillance for Cancers Associated with Tobacco Use—United States, 2010–2014. MMWR Surveill Summ 67:1–42. https://doi.org/10.15585/mmwr.ss6712a1

Vanderpool RC, Huang B, Deng Y, Bear TM, Chen Q, Johnson MF, Paskett ED, Robertson LB, Young GS, Iachan R (2019) Cancer-related beliefs and perceptions in Appalachia: findings from 3 states. J Rural Health 35:176–188. https://doi.org/10.1111/jrh.12359

Rosenbaum S (2013) Principles to Consider for the Implementation of a Community Health Needs Assessment Process. Washington, DC : The George Washington University, School of Public Health and Health Services. Dep Health Policy

Franz BA, Skinner D, Murphy JW (2018) Defining “Community” in community health evaluation: perspectives from a sample of nonprofit Appalachian hospitals. Am J Eval 39:237–256. https://doi.org/10.1177/1098214017722857

Pennel CL, McLeroy KR, Burdine JN, Matarrita-Cascante D (2015) Nonprofit hospitals’ approach to community health needs assessment. Am J Public Health 105:e103–e113. https://doi.org/10.2105/AJPH.2014.302286

Velonis AJ, Molnar A, Lee-Foon N, Rahim A, Boushel M, O’Campo P (2018) “One program that could improve health in this neighbourhood is ____?” using concept mapping to engage communities as part of a health and human services needs assessment. BMC Health Serv Res 18:150. https://doi.org/10.1186/s12913-018-2936-x

Dos Santos Marques IC, Theiss LM, Johnson CY, McLin E, Ruf BA, Vickers SM, Fouad MN, Scarinci IC, Chu DI (2021) Implementation of virtual focus groups for qualitative data collection in a global pandemic. Am J Surg 221:918–922. https://doi.org/10.1016/j.amjsurg.2020.10.009

Keen S, Lomeli-Rodriguez M, Joffe H (2022) From challenge to opportunity: virtual qualitative research during COVID-19 and beyond. Int J Qual Methods 21:160940692211050. https://doi.org/10.1177/16094069221105075

Download references

Acknowledgments

This work as supported in part by the National Institutes of Health (NIH) (Grant Numbers P30CA177558 and UL1TR001998), the Center for Disease Control and Prevention (CDC) (Grant Numbers NU58DP006313-04-02, NU58DP007144-01-00, NU58DP006058-05-00), and the William Stamps Farish Endowed Chair in Cancer Research. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH or CDC. We wish to acknowledge the valuable input of the members of the Kentucky Cancer Needs Assessment Steering Committee, some of whom contributed to this article as co-authors, along with the additional members who represented Kentucky Cancer Registry (Jaclyn McDowell), Kentucky Cancer Program (Elizabeth Westbrook, Jamie Smith), Kentucky Cabinet for Health and Family Services (Janie Cambron, Vivian Lasley-Bibbs, Emily Messerli, Carissa Adams, Carrie Conia, Elizabeth Owen), American Cancer Society (Elizabeth Holtsclaw, Julie Waters), Foundation for a Healthy Kentucky (Allison Adams), and the University of Louisville (Stephanie Boone). We also wish to thank Dr. Eric Durbin, Director of the Kentucky Cancer Registry for his feedback and support of this effort.

P.C. Hull, B. Huang and C.R. McAfee were supported by the National Institutes of Health (NIH) grant P30CA177558. P.C. Hull and J.R. Thompson received support from the William Stamps Farish Endowed Chair in Cancer Research. P.C. Hull, E. Russell, J. Redmond Knight, and B. Huang were supported by the Center for Disease Control and Prevention (CDC) Grants NU58DP006313-04-02 and NU58DP007144-01-00. B, Huang received support from the NIH SEER program contract HHSN26100001. S. Kanotra was supported in part by funds from the CDC Cooperative Agreement Number 5 NU58DP006058-05-00. This work was supported by the Cancer Research Informatics Shared Resource Facility and the Biostatistics and Bioinformatics Shared Resource Facility of the University of Kentucky Markey Cancer Center (P30CA177558) and by the NIH National Center for Advancing Translational Sciences through Grant Number UL1TR001998. The content is solely the responsibility of the authors and does not necessarily represent the official views of the NIH or CDC.

Author information

Authors and affiliations.

Markey Cancer Center, University of Kentucky, Lexington, KY, USA

Todd Burus, Jessica R. Thompson, Caree R. McAfee, Lovoria B. Williams, Natalie P. Wilhite, Elaine Russell, Christine Stroebel & Pamela C. Hull

College of Nursing, University of Kentucky, Lexington, KY, USA

Lovoria B. Williams

Kentucky Cancer Consortium, Lexington, KY, USA

Jennifer Redmond Knight & Elaine Russell

Department of Health Management and Policy, College of Public Health, University of Kentucky, Lexington, KY, USA

Jennifer Redmond Knight

Division of Cancer Biostatistics, College of Medicine, University of Kentucky, Lexington, KY, USA

Kentucky Cancer Registry, Markey Cancer Center, University of Kentucky, Lexington, KY, USA

Kentucky Department for Public Health, Frankfort, KY, USA

Sarojini Kanotra

Kentucky Cancer Program, Markey Cancer Center, University of Kentucky, Lexington, KY, USA

Melinda Rogers

Kentucky Cancer Program, Brown Cancer Center, University of Louisville, Louisville, KY, USA

Connie L. Sorrell

American Cancer Society, Kennesaw, GA, USA

Rachael King

Department of Behavioral Science, College of Medicine, University of Kentucky, Lexington, KY, USA

Pamela C. Hull

University of Kentucky Markey Cancer Center, 760 Press Avenue, Suite 460, Lexington, KY, 40536, USA

You can also search for this author in PubMed   Google Scholar

Contributions

All authors contributed to the study conception and design. Project administration was performed by TB, JT, CM, LW, and PH. Data collection, data curation and analysis were performed by TB, JT, BH, SK, and CS. Visualizations were prepared by TB and NW. The first draft of the manuscript was written by TB, JT, CM, and PH. All authors provided critical review and comments on drafts for revisions of the manuscript, and all authors read and approved the final manuscript.

Corresponding author

Correspondence to Todd Burus .

Ethics declarations

Competing interests.

The authors have no relevant financial or non-financial interests to disclose.

Ethical approval

The University of Kentucky Institutional Review Board (IRB) approved the research procedures under three protocols: Kentucky Behavioral Risk Factor Survey analysis (#69894), focus groups (#65451), and concept mapping (#73420). The IRB issued a non-human research determination for using the publicly available, aggregated secondary data and the CNA dissemination follow-up survey.

Consent to participate

Informed consent was obtained from all individual participants included in the study.

Additional information

Publisher's note.

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (PPTX 702 kb)

Rights and permissions.

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/ .

Reprints and permissions

About this article

Burus, T., Thompson, J.R., McAfee, C.R. et al. A framework and process for community-engaged, mixed-methods cancer needs assessments. Cancer Causes Control (2024). https://doi.org/10.1007/s10552-024-01892-2

Download citation

Received : 16 October 2023

Accepted : 17 May 2024

Published : 29 May 2024

DOI : https://doi.org/10.1007/s10552-024-01892-2

Share this article

Anyone you share the following link with will be able to read this content:

Sorry, a shareable link is not currently available for this article.

Provided by the Springer Nature SharedIt content-sharing initiative

• Catchment area
• Needs assessment
• Qualitative research
• Quantitative evaluation
• Community outreach

Advertisement

• Find a journal
• Publish with us
• Track your research

NTRS - NASA Technical Reports Server

Available downloads, related records.

An official website of the United States government

The .gov means it’s official. Federal government websites often end in .gov or .mil. Before sharing sensitive information, make sure you’re on a federal government site.

The site is secure. The https:// ensures that you are connecting to the official website and that any information you provide is encrypted and transmitted securely.

• Publications
• Account settings

Preview improvements coming to the PMC website in October 2024. Learn More or Try it out now .

• Advanced Search
• Journal List
• Can J Hosp Pharm
• v.67(5); Sep-Oct 2014

An Introduction to the Fundamentals of Cohort and Case–Control Studies

Associated data, introduction.

As pharmacotherapy experts, pharmacists are continually updating their knowledge about drug effects. In addition to being knowledge users of research findings, pharmacists increasingly play a larger role in observational studies of drug effects. Observational studies are inherently nonexperimental and, unlike randomized clinical trials (RCTs), do not involve any manipulation (such as randomization) of the treatment and control groups by the investigator.

This article reviews for the practising pharmacist the fundamental design elements and foundational methodologic knowledge for conducting cohort and case–control studies, 2 common and robust observational study designs for elucidating drug–outcome associations. Readers interested in learning about other observational study designs, such as cross-sectional studies, ecological studies, case series, case reports, within-person studies, and quasi-experimental designs, or the critical appraisal of such designs, are referred elsewhere. 1 – 6

WHY WE NEED COHORT AND CASE–CONTROL STUDIES

We need well-designed and rigorous cohort and case– control studies because their findings provide knowledge complementary to that garnered from RCTs ( Table 1 ). The design properties of RCTs maximize their ability to estimate the potential causal effects of drugs under ideal circumstances and thereby to estimate the efficacy of those drugs. However, many RCTs involve a relatively limited number of highly selected patients and a limited duration. Indeed, RCTs typically follow patients for only a small fraction of the time that the drug would be used in clinical practice, especially when the medications are for chronic diseases. Moreover, RCTs typically exclude complex patients, they often use irrelevant comparators (e.g., placebo), and they frequently measure outcomes that are not patient-centred (i.e., surrogate end points). 7 Although many of these limitations may be overcome by designing more pragmatic RCTs that do indeed measure effectiveness, 8 cohort and case–control studies are 2 feasible study design alternatives that address the limitations of RCTs ( Table 1 ) without the considerable financial and human resource costs of pragmatic RCTs.

Limitations of Randomized Clinical Trials (RCTs) Potentially Addressed by Cohort and Case–Control Studies

COHORT STUDIES

A cohort is a group of people who share a common experience or characteristic. The term “cohort” first appeared in the medical literature in the 1930s in an article by epidemiologist W H Frost. 9 Interestingly, the word “cohort” has military roots, originating from the Latin word “cohors”. 10 The term was first used in the Roman military, where a group of 300 to 600 soldiers constituted a cohort. 11

A cohort study compares the experience of 2 or more groups of patients who are followed concurrently forward in time ( Figure 1 ). This prospective tracking, from exposure to outcome, is in fact one of the defining features of a cohort study. 11 The temporal sequence involved in following a group of patients who are exposed to a certain factor (the treatment group) and a group of patients who are not exposed to that factor (the control group) is akin to that of a clinical trial, where instead of chance determining a patient’s exposure status (as occurs in an RCT), choice or happenstance determines exposure status.

Schematic for the design of cohort and case–control studies.

Selecting the Study Cohort

For any cohort study, a source population must be defined, from which the eligible study cohort is derived through application of various inclusion and exclusion criteria. At a minimum, patients entering the study cohort must be free of the outcome of interest. For example, in a cohort study designed to measure the association between atypical antipsychotics and diabetes mellitus, patients with diabetes would have to be excluded from the study cohort because they are not at risk of the outcome. Often, other restrictions are put in place to minimize the risk of bias. For example, restriction to new users of a medication will ensure avoidance of multiple biases. 12 Inclusion of prevalent or current drug users can lead to significant bias because patients who experience early intolerance or adverse effects of a drug may discontinue the drug, and the remaining cohort will consist of a healthier and usually more adherent group. 13 Risk that varies over time, whereby new users have a higher risk of an adverse event, has been observed for numerous associations, including those between nonsteroidal anti-inflammatory drugs and upper gastrointestinal bleeding, 14 oral contraceptives and venous thromboembolism, 15 benzodiazepines and falls, 16 and angiotensin-converting enzyme inhibitors and angioedema. 17

Defining Drug Exposure Groups

Once the study cohort has been created, 2 or more exposure groups must be clearly defined, 1 of which must serve as the control or reference group. The reference group should be clinically relevant. For example, in a comparative safety or effectiveness study, patients taking a drug within the same therapeutic class or receiving usual care may serve as the reference group. If clinically and scientifically relevant, a group with no therapeutic exposure may be the reference group. Drug exposure may be measured in terms of persons or person-time (the time for which a person is exposed to a particular drug). Drug exposure is often categorized in a binary fashion (i.e., yes or no), based on either a minimum number of prescription records (e.g., at least 3 records) or a specified duration of exposure (e.g., at least 90 days’ exposure), or a combination of these 2 factors (i.e., cumulative exposure). Irrespective of how exposure is defined, it is essential that follow-up time be properly categorized following entry into the cohort to avoid time-related bias. 18 Furthermore, the definition of exposure should be coherent with the study hypothesis. For example, a certain amount of time or a certain dose of drug may be required to elicit an effect, or a drug may continue to have an effect once discontinued (e.g., bisphosphonates). Moreover, decisions about when to discontinue drug exposure must be made. There are 2 common approaches: “as treated”, whereby drug exposure is recorded as being stopped when a person no longer meets the definition of exposure; and “intention-to-treat”, whereby a person is considered exposed from the time of first meeting the study’s exposure definition until experiencing the outcome of interest or the end of the study, irrespective of changes in actual exposure status. There is no consensus on how to best define drug exposures, and hence the definitions of exposure often vary considerably among cohort studies assessing identical drug–outcome associations.

Measuring Occurrence of Outcomes

Complete and accurate measurement of the outcome of interest is essential to ensure the validity of study results. When subjective outcome data (e.g., diagnosis of pneumonia) are being collected during the study period, exposure status should be blinded for the outcome assessors and adjudicators, to prevent responder bias. When previously collected data (i.e., secondary data) are being used, investigators should ideally use outcome definitions that have been validated in previous studies. For example, Hux and others 19 validated definitions of diabetes by comparing International Classification of Diseases codes obtained from administrative health care databases in Ontario with diagnostic data from primary care charts.

Quantifying the Drug–Outcome Association

For cohort studies, the drug–outcome association is usually expressed as a relative risk, a relative rate, or a hazard ratio. Advanced statistical techniques are used to account for factors other than the drug exposure of interest that might distort the drug–outcome association. These factors or potential confounders are often handled simultaneously with multivariable regression models. Because these statistical models account for measured variables, it is crucial that the data source capture as many potential confounding variables (or proxies of confounders) as possible. Potential confounders should usually be measured before entry into the cohort, to avoid adjustment for factors in the causal pathway.

Strengths and Weaknesses

One of the major strengths of a cohort study is that the temporal sequence—drug exposure preceding outcome—is explicit in the study design. The incidence of a particular outcome among persons exposed to a certain drug can be directly calculated using a cohort design. Cohort studies are also relatively efficient for studying rare exposures, and multiple outcomes may be assessed for a single exposure. However, cohort studies with long observation periods may be more susceptible to losses to follow-up and to inaccurate measurement of exposures and outcomes. Large numbers of patients may be required to precisely estimate meaningful drug–outcome associations, especially for rare outcomes or outcomes that take a long time to occur.

CASE-CONTROL STUDIES

The first case–control study using the design with which we are familiar today was published in 1926. However, the concept of case–control studies has its origins in the investigation of disease etiologies through detailed histories and examination of patients. 20

In a case–control study, a number of cases and noncases (controls) are identified, and the occurrence of one or more prior exposures is compared between groups to evaluate drug–outcome associations ( Figure 1 ). A case–control study runs in reverse relative to a cohort study. 21 As such, study inception occurs when a patient experiences an outcome and is thus designated a “case”. A modern conceptual view holds that the case–control study can be thought of as an efficient cohort design. Essentially, patients who would have experienced the outcome of interest in a cohort study are the cases in a case–control study. Similarly, patients who were at risk but did not experience the outcome of interest in a cohort study are the controls in a case–control study. The potential data sources for a case–control study are identical with those for a cohort study, and the investigator may collect data after study inception or may use previously collected data. An extension of the case–control study is the nested case–control study, which is a case–control study conducted within a cohort. Details regarding this design are beyond the scope of this article and are reviewed elsewhere. 22 , 23

Selection of Cases

The first step in a case–control study is to identify the cases through application of explicitly defined inclusion and exclusion criteria. Ideally, cases should be directly sampled from the source population in a manner that is unrelated to the drug exposures of interest; however, the source population that gave rise to the cases is often unknown and difficult to identify (except in a nested-case control study, where the source population is known). The case-selection process and the data sources from which cases were selected should be described in detail, especially if cases are from a variety of sources, such as hospital and community-based sources. Selecting only hospital-based cases may lead to systematic error related to hospital admission practices, whereby exposed cases may be more likely to be admitted and therefore selected into the study (a phenomenon known as Berksonian bias). Furthermore, only new (incident) cases should be selected, as nonincident cases usually over-represent long-term survivors, and diagnostic practices may change over time, introducing potential bias. When cases are selected from a secondary data source, the case definitions should be supported by previous validation studies.

Selection of Controls

The selection of controls in a case–control study is fraught with difficulty and is often the source of significant bias. Essentially, the controls should be selected from the same source population as the cases. 24 In other words, controls should be at risk of becoming cases and should come from a population with the same exposure distribution as the cases. Multiple controls are usually selected for each case, to increase the statistical efficiency of the study; however, the gains are minimal beyond 3 or 4 controls per case. Nonetheless, modern case–control studies involving large databases often use much higher control–case ratios to maximize study precision. To control for potential confounding, cases and controls are often matched on one or more patient characteristics, such as age or sex (although it may not always be appropriate to match on these variables). The study investigator must be careful not to match on too many factors or on factors that are not confounders, as doing so might lead to overmatching and bias. Furthermore, matching should not involve variables that the investigator is interested in examining in association with an outcome. The selection of controls is one of the most difficult aspects of epidemiologic research, and readers are encouraged to consult additional resources. 24 – 28

Similar to the situation for a cohort study, the drug exposures of interest and their definitions should be clearly specified in the methods. Because exposure in a case–control study is determined after the cases have been identified, a period before occurrence of the case, called the “look-back period” or “look-back window”, must be defined. A comparable look-back period must be defined for the control group. Look-back periods should consider the study hypothesis and thus may vary considerably from one study to another. For example, Abdelmoneim and others 29 specified a 120-day look-back period before the date of their cases (patients with acute coronary syndrome) to assess recent exposure to glyburide and gliclazide. Azoulay and others 30 specified an exposure window of any time prior to a year before the date of cases in their study evaluating the association between pioglitazone and bladder cancer. If the investigators are collecting exposure data themselves, then outcome status should be blinded to study personnel.

In a case–control study, the odds ratio is the usual measure of association reported. This measure is the ratio of the odds of an exposure between cases and controls and in most cases approximates the relative risk. As in a cohort study, the analytic plan for a case–control study typically involves advanced statistical methods to adjust for multiple potential confounders.

The major strengths of the case–control design are statistical efficiency (i.e., uses fewer data to quantify a drug–outcome association than would be required in a cohort study), efficiency for studying rare outcomes, efficiency for studying conditions with long latency periods, efficiency for handling the time-varying nature of drug exposures, and relatively low cost. The weaknesses of case–control studies include inefficiency for studying rare exposures, difficulty of selecting unbiased controls, and inability to directly calculate incidence rates of outcomes.

LIMITATIONS OF COHORT AND CASE–CONTROL STUDIES

Bias and confounding.

Observational studies are methodologically difficult, susceptible to bias and confounding, and difficult to interpret, given the many types of bias potentially at play. For these reasons, observational studies are limited to studying drug–outcome associations and cannot be used to measure the causal effects of drugs. Recent methodologic advances in design and analytic techniques in pharmacoepidemiology have helped to combat the various types of selection bias, information bias, and confounding at play in cohort and case–control studies (see Appendix 1 , available online at www.cjhp-online.ca/index.php/cjhp/issue/view/104/showToc ). Many of these techniques can account for multiple potential confounders simultaneously. A comprehensive review of these techniques is beyond the scope of this article, but such reviews may be found elsewhere, 25 , 31 – 33 Bias and confounding result in spurious drug–outcome associations and are introduced at both the design and analysis stages. Appendix 2 (available online at www.cjhp-online.ca/index.php/cjhp/issue/view/104/showToc ) illustrates the origin of bias in relation to the cohort design, and Appendix 3 (available online at www.cjhp-online.ca/index.php/cjhp/issue/view/104/showToc ) lists common types of bias that occur in cohort and case–control studies of drug effects.

Study of Intended Drug Effects

Cohort and case–control studies are powerful approaches for estimating the association between drugs and unintended outcomes 34 ; however, their use for studying the intended effects of drugs has spurred debate in the past and remains controversial today. 35 – 37 This controversy has arisen because the propensity for bias and confounding is much higher when estimating the intended effects of drugs (i.e., benefits). 37 This higher propensity for bias is in turn due to the nonrandom nature of prescribing practices and is often referred to as “confounding by the reason for the prescription” or simply “confounding by indication”. Confounding by indication is expected with these types of studies, as it is good medical practice to prescribe intentionally and rationally, as opposed to prescribing according to a random process. 38 Some authors strongly recommend against using observational studies to study intended effects, suggesting instead that we consider restricting our research questions to those of unintended effects because confounding by indication introduces uncontrollable bias. 31 , 34 , 39 , 40 The literature contains numerous examples of confounding by indication. A most striking example is the distorted 27-fold increased risk of thrombotic events associated with use of warfarin, when in fact warfarin prevents thrombotic events. 39 Another example of confounding by indication is the observed relationship between short-acting ß-receptor agonists (e.g., salbutamol) and increased risk of death from asthma. 41 Of course confounding by indication is not verifiable, but it must be considered when studying the intended effects of drugs.

GENERAL CONSIDERATIONS IN CONDUCTING A COHORT OR CASE–CONTROL STUDY

Protocol and study team.

Cohort and case–control studies aim to quantitatively estimate the association between a drug exposure and outcome. Before embarking on a cohort or case–control study, the investigators must develop a well-articulated and focused research question. 42 Furthermore, the study protocol, including a detailed methodologic and analytic plan, should be consistent with international guidelines. 43 , 44 The study team should have appropriate clinical and methodologic expertise. Clinical expertise is essential for developing exposure and outcome definitions, as well as for understanding the overall clinical context of how the research question fits into the current body of knowledge. Methodologic expertise is critical for ensuring that robust methods are used, to minimize bias and confounding.

Data Sources

To estimate a drug–outcome association in a cohort or case– control study, accurate and comprehensive data must be collected on the drug exposures and outcomes of interest. Study investigators may collect data after study inception or may use previously collected data. The major advantage of prospectively collecting the data (primary data collection) is that the investigators have control over what information is collected; in contrast, when previously collected data are used (secondary data collection), the investigators are limited to the information already collected. Data may often be missing from or inaccurately recorded in secondary data sources, which creates challenges when the data are used for research purposes. Although previously collected data are considered retrospective to study inception, the data themselves are often collected prospectively; therefore, use of the terms “retrospective” and “prospective” may be misleading and usually does not provide any clarity in terms of important design characteristics. 25 There are 3 main sources of existing data: administrative data, medical records, and surveys. Special considerations and the advantages and disadvantages of these secondary data sources are discussed elsewhere. 45 , 46 For studying drug effects, secondary data sources are more commonly used than primary data collection, primarily because of gains in time, cost, and statistical efficiency. Furthermore, use of secondary data sources avoids the Hawthorne effect, whereby knowledge of participation in a study changes the behaviour of study participants and may lead to bias.

CONCLUSIONS

Pharmacists use knowledge from cohort and case–control studies to inform patients, clinicians, and the general public about drug effects. At a basic level, cohort and case–control studies quantitatively estimate the relation between exposures and outcomes. They represent rigorous study designs for answering drug safety and effectiveness questions, with case–control studies being more prone to bias. The methodologic rigour of cohort and case–control studies evaluating drug–outcome associations is advancing, and approaches are being developed and refined that limit the generation of misleading study results. Indeed, both RCTs and observational studies are necessary, and neither is sufficient to learn about the totality of drug effects in the population.

Acknowledgments

John-Michael Gamble is supported by a New Investigator Award in drug safety and effectiveness from the Canadian Institutes of Health Research and a Clinician Scientist Award from the Canadian Diabetes Association.

This article is the sixth in the CJHP Research Primer Series, an initiative of the CJHP Editorial Board and the CSHP Research Committee. The planned 2-year series is intended to appeal to relatively inexperienced researchers, with the goal of building research capacity among practising pharmacists. The articles, presenting simple but rigorous guidance to encourage and support novice researchers, are being solicited from authors with appropriate expertise.

Previous article in this series:

Bond CM. The research jigsaw: how to get started. Can J Hosp Pharm . 2014;67(1):28–30.

Tully MP. Research: articulating questions, generating hypotheses, and choosing study designs. Can J Hosp Pharm . 2014;67(1):31–4.

Loewen P. Ethical issues in pharmacy practice research: an introductory guide. Can J Hosp Pharm. 2014;67(2):133–7.

Tsuyuki RT. Designing pharmacy practice research trials. Can J Hosp Pharm . 2014;67(3):226–9.

Bresee LC. An introduction to developing surveys for pharmacy practice research. Can J Hosp Pharm . 2014;67(4):286–91.

Competing interests: None declared.