in conducting a literature review the researcher mcq

Research guidance, Research Journals, Top Universities

Top 20 MCQs on literature review with answers

MCQs on literature review : The primary purpose of literature review is to facilitate detailed background of the previous studies to the readers on the topic of research.

In this blog post, we have published 20 MCQs on Literature Review (Literature Review in Research) with answers.

20 Multiple Choice Questions on Literature Review

1. Literature is a 

Written Record

Published Record

Unpublished Record

All of these

2. Which method of literature review involves a non-statistical method to present data having the feature of systematic Method too?

Narrative Method

Systematic Method

Meta-Analysis Method of Literature Review

Meta-Synthesis Method of Literature Review

3. Comparisons of non-statistical variables are performed under which method of literature review?

4. Literature review is not similar to

Annotated Bibliography 

5. APA Style, MLA Style, Chicago Manual, Blue Book, OSCOLA are famously known as

Citation Manuals

Directories

Abbreviation Manuals

6. Literature collected is reviewed and preferably arranged 

Alphabetically

Chronologically

None of these

7. Literature collected for review includes

Primary and Secondary Sources

Secondary and Tertiary Sources

Primary and Tertiary Sources

8. Literature includes

Previous Studies

Scholarly publications

Research Findings

9. No time frame is set to collect literature in which of the following method of compiling reviews?

Traditional Method

10. Which method of the literature review is more reliable for drawing conclusions of each individual researcher for new conceptualizations and interpretations?

11. The main purpose of finalization of research topics and sub-topics is

Collection of Literature

Collection of Questions

Collection of Statistics

Collection of Responses

12. Literature review is basically to bridge the gap between

Newly established facts

Previously established facts

Facts established time to time

Previous to current established facts

13. The last step in writing the literature review is 

Developing a Final Essay

Developing a Coherent Essay

Developing a Collaborated Essay

Developing a Coordinated Essay

14. The primary purpose of literature review is to facilitate detailed background of 

Present Studies

Previous studies

Future Studies

15. Narrative Literature Review method is also known as 

Advanced Method

Scientific Method

16. Which method of literature review starts with formulating research questions?

17. Which method of literature review involves application of clinical approach based on a specific subject.

18. Which literature review involves timeline based collection of literature for review

19. Which method of literature review involves application of statistical approach?

20. Which literature review method involves conclusions in numeric/statistical form?

More MCQs Related to MCQs on Literature Review

• MCQs on Qualitative Research with answers
• Research Proposal MCQs with answers PDF
• Solved MCQ on legal Reasoning in Research
• MCQ on data analysis in research methodology
• Research Report writing MCQs with answers
• All Solved MCQs on Research Methodology
• MCQs on Legal Research with answers
• MCQs on sampling in research methodology with answers
• MCQs with answers on plagiarism
• MCQ on Citation and Referencing in Research
• Research Ethics MCQs with answers
• Solved MCQs on Sampling in research methodology
• Solved MCQs on Basic Research

MCQs  on literature review  with answers PDF | Research methods multiple choice questions | Literature review  questions and answers

Share this:

1 thought on “top 20 mcqs on literature review with answers”.

Very nice questions for revision

Leave a Comment Cancel reply

Save my name, email, and website in this browser for the next time I comment.

Notify me of follow-up comments by email.

Notify me of new posts by email.

• Request new password
• Create a new account

Research Methods in Early Childhood: An Introductory Guide

Student resources, multiple choice quiz.

Test your understanding of each chapter by taking the quiz below. Click anywhere on the question to reveal the answer. Good luck!

1. What is an abstract?

• A short footnote in the text of your work
• A chance to acknowledge those who have helped you in your research project, placed at the beginning of the written-up project
• A brief resume of the study to entice the reader to read more
• An introduction to the research

2. What should always be included in an introduction?

• Detail of your research findings
• Your interview schedule (or detail of any other research instruments)
• A rationale for conducting the study and key research questions
• Recommendations for early childhood practice in detail

3. What is a literature review?

• A place to provide autobiographic detail
• A place to outline all the reading you have done in your studies
• A place to link your reading to your research findings
• A place to discuss literature which helps to frame your study

4. Where is a literature review typically found in written-up research?

• Before the methodology chapter
• After the findings chapter
• In the appendices
• Integrated into the data analysis

5. What would you expect to find in a methodology chapter in a piece of empirical research?

• Detail of the methodological approach taken
• Detail of the methods employed
• Detail of ethical considerations linked to the research project
• All of the above

6. Should data and discussion of data be presented as two separate chapters?

• Never. They are inappropriate for early childhood research
• Always. Students tend to undertake qualitative research projects
• Possibly, depending on the kind of research undertaken and tutor advice
• Possibly, depending if there is time to complete two chapters as opposed to one larger chapter

7. What should a conclusion chapter contain?

• A sense of the research story
• A summary of the key findings
• Reflection on what these findings mean
• Discussion of possible implications for practice or future research

8. Having read the chapter, what do Mukherji and Albon advise in relation to using ‘I’ when writing up research?

• You should always use the personal pronoun when writing up research as it reflects that it is a personal piece of work
• Writing seems far too ‘chatty’ if you use ‘I’ when writing. In order to write academically use of ‘I’ should always be avoided
• If talking about the decisions you made in research and your own viewpoint, it is hard to avoid use of ‘I’ and it does not necessarily mean lack of academic tone if used judiciously (but do ask your tutor!)
• You could be in danger of failing a module if you use the personal pronoun in your work. You will never see academic journal articles using ‘I’

Harvey Cushing/John Hay Whitney Medical Library

• Collections
• Research Help

YSN Doctoral Programs: Steps in Conducting a Literature Review

• Biomedical Databases
• Global (Public Health) Databases
• Soc. Sci., History, and Law Databases
• Grey Literature
• Trials Registers
• Data and Statistics
• Public Policy
• Google Tips
• Recommended Books
• Steps in Conducting a Literature Review

What is a literature review?

A literature review is an integrated analysis -- not just a summary-- of scholarly writings and other relevant evidence related directly to your research question.  That is, it represents a synthesis of the evidence that provides background information on your topic and shows a association between the evidence and your research question.

A literature review may be a stand alone work or the introduction to a larger research paper, depending on the assignment.  Rely heavily on the guidelines your instructor has given you.

Why is it important?

A literature review is important because it:

• Explains the background of research on a topic.
• Demonstrates why a topic is significant to a subject area.
• Discovers relationships between research studies/ideas.
• Identifies major themes, concepts, and researchers on a topic.
• Identifies critical gaps and points of disagreement.
• Discusses further research questions that logically come out of the previous studies.

APA7 Style resources

APA Style Blog - for those harder to find answers

1. Choose a topic. Define your research question.

Your literature review should be guided by your central research question.  The literature represents background and research developments related to a specific research question, interpreted and analyzed by you in a synthesized way.

• Make sure your research question is not too broad or too narrow.  Is it manageable?
• Begin writing down terms that are related to your question. These will be useful for searches later.
• If you have the opportunity, discuss your topic with your professor and your class mates.

2. Decide on the scope of your review

How many studies do you need to look at? How comprehensive should it be? How many years should it cover? 

• This may depend on your assignment.  How many sources does the assignment require?

3. Select the databases you will use to conduct your searches.

Make a list of the databases you will search. 

Where to find databases:

• use the tabs on this guide
• Find other databases in the Nursing Information Resources web page
• More on the Medical Library web page
• ... and more on the Yale University Library web page

4. Conduct your searches to find the evidence. Keep track of your searches.

• Use the key words in your question, as well as synonyms for those words, as terms in your search. Use the database tutorials for help.
• Save the searches in the databases. This saves time when you want to redo, or modify, the searches. It is also helpful to use as a guide is the searches are not finding any useful results.
• Review the abstracts of research studies carefully. This will save you time.
• Use the bibliographies and references of research studies you find to locate others.
• Check with your professor, or a subject expert in the field, if you are missing any key works in the field.
• Ask your librarian for help at any time.
• Use a citation manager, such as EndNote as the repository for your citations. See the EndNote tutorials for help.

Review the literature

Some questions to help you analyze the research:

• What was the research question of the study you are reviewing? What were the authors trying to discover?
• Was the research funded by a source that could influence the findings?
• What were the research methodologies? Analyze its literature review, the samples and variables used, the results, and the conclusions.
• Does the research seem to be complete? Could it have been conducted more soundly? What further questions does it raise?
• If there are conflicting studies, why do you think that is?
• How are the authors viewed in the field? Has this study been cited? If so, how has it been analyzed?

Tips: 

• Review the abstracts carefully.  
• Keep careful notes so that you may track your thought processes during the research process.
• Create a matrix of the studies for easy analysis, and synthesis, across all of the studies.
• << Previous: Recommended Books
• Last Updated: Jan 4, 2024 10:52 AM
• URL: https://guides.library.yale.edu/YSNDoctoral

• UConn Library
• Literature Review: The What, Why and How-to Guide
• Introduction

Literature Review: The What, Why and How-to Guide — Introduction

• Getting Started
• How to Pick a Topic
• Strategies to Find Sources
• Evaluating Sources & Lit. Reviews
• Tips for Writing Literature Reviews
• Writing Literature Review: Useful Sites
• Citation Resources
• Other Academic Writings

What are Literature Reviews?

So, what is a literature review? "A literature review is an account of what has been published on a topic by accredited scholars and researchers. In writing the literature review, your purpose is to convey to your reader what knowledge and ideas have been established on a topic, and what their strengths and weaknesses are. As a piece of writing, the literature review must be defined by a guiding concept (e.g., your research objective, the problem or issue you are discussing, or your argumentative thesis). It is not just a descriptive list of the material available, or a set of summaries." Taylor, D.  The literature review: A few tips on conducting it . University of Toronto Health Sciences Writing Centre.

Goals of Literature Reviews

What are the goals of creating a Literature Review?  A literature could be written to accomplish different aims:

• To develop a theory or evaluate an existing theory
• To summarize the historical or existing state of a research topic
• Identify a problem in a field of research 

Baumeister, R. F., & Leary, M. R. (1997). Writing narrative literature reviews .  Review of General Psychology , 1 (3), 311-320.

What kinds of sources require a Literature Review?

• A research paper assigned in a course
• A thesis or dissertation
• A grant proposal
• An article intended for publication in a journal

All these instances require you to collect what has been written about your research topic so that you can demonstrate how your own research sheds new light on the topic.

Types of Literature Reviews

What kinds of literature reviews are written?

Narrative review: The purpose of this type of review is to describe the current state of the research on a specific topic/research and to offer a critical analysis of the literature reviewed. Studies are grouped by research/theoretical categories, and themes and trends, strengths and weakness, and gaps are identified. The review ends with a conclusion section which summarizes the findings regarding the state of the research of the specific study, the gaps identify and if applicable, explains how the author's research will address gaps identify in the review and expand the knowledge on the topic reviewed.

• Example : Predictors and Outcomes of U.S. Quality Maternity Leave: A Review and Conceptual Framework:  10.1177/08948453211037398  

Systematic review : "The authors of a systematic review use a specific procedure to search the research literature, select the studies to include in their review, and critically evaluate the studies they find." (p. 139). Nelson, L. K. (2013). Research in Communication Sciences and Disorders . Plural Publishing.

• Example : The effect of leave policies on increasing fertility: a systematic review:  10.1057/s41599-022-01270-w

Meta-analysis : "Meta-analysis is a method of reviewing research findings in a quantitative fashion by transforming the data from individual studies into what is called an effect size and then pooling and analyzing this information. The basic goal in meta-analysis is to explain why different outcomes have occurred in different studies." (p. 197). Roberts, M. C., & Ilardi, S. S. (2003). Handbook of Research Methods in Clinical Psychology . Blackwell Publishing.

• Example : Employment Instability and Fertility in Europe: A Meta-Analysis:  10.1215/00703370-9164737

Meta-synthesis : "Qualitative meta-synthesis is a type of qualitative study that uses as data the findings from other qualitative studies linked by the same or related topic." (p.312). Zimmer, L. (2006). Qualitative meta-synthesis: A question of dialoguing with texts .  Journal of Advanced Nursing , 53 (3), 311-318.

• Example : Women’s perspectives on career successes and barriers: A qualitative meta-synthesis:  10.1177/05390184221113735

Literature Reviews in the Health Sciences

• UConn Health subject guide on systematic reviews Explanation of the different review types used in health sciences literature as well as tools to help you find the right review type
• << Previous: Getting Started
• Next: How to Pick a Topic >>
• Last Updated: Sep 21, 2022 2:16 PM
• URL: https://guides.lib.uconn.edu/literaturereview

Review Literature in Research MCQs

Research design (qualitative and quantitative) mcqs topics.

General MCQs

Selection of a Research Approach MCQs

Use of Research Design Theory MCQs

Writing Strategies and Ethical Consideration MCQs

Introduction Research Design MCQs

Purpose Statement for Research Design MCQs

Research Questions and Hypotheses MCQs

Quantitative Methods of Research Design MCQs

Qualitative Methods of Research Design MCQs

Mixed Methods Procedures of Research Design MCQs

Welcome to the page of MCQs on Reviewing Literature in Research. Here, you will find a collection of multiple-choice questions that focus on the process and significance of conducting a literature review in research.

The process of conducting a literature review involves several steps, such as identifying relevant sources, critically evaluating the literature, extracting key information, and synthesizing the findings. Researchers should follow best practices to ensure the reliability and validity of their literature review.

MCQss.com offers a diverse range of MCQs on reviewing literature in research, covering various aspects of the literature review process. By engaging with these MCQs, you can assess your knowledge, enhance your understanding of literature review methods and techniques, and improve your research skills.

Engaging with MCQs on reviewing literature in research equips researchers, students, and professionals with the necessary knowledge and skills to conduct a comprehensive and effective literature review. It enhances their ability to identify and synthesize relevant information, critically evaluate sources, and establish a strong foundation for their research.

1: Definitions of terms appear in qualitative, quantitative, and mixed methods proposals to ensure the researcher’s precision of meaning.

A.   True

B.   False

2: When abstracting a study, you should identify the methodological and technical strengths and weaknesses.

3: when abstracting a study, you should identify the methodological and technical strengths and weaknesses., 4: which type of literature varies the most in quality and should be lower in priority.

A.   Conference papers

B.   Research monographs

C.   Encyclopedias

D.   Dissertations

5: The Chicago Manual of Style is the most popular style manual used in field of education and psychology.

6: what is the primary purpose of reviewing literature in research.

A.   To demonstrate the researcher's knowledge on the topic

B.   To find supporting evidence for the research hypothesis

C.   To identify gaps, trends, and existing knowledge on the topic

D.   To provide citations for the research paper

7: Which of the following statements best describes a literature review in research?

A.   A summary of the research findings from the current study

B.   An evaluation of the quality of research articles

C.   A critical analysis and synthesis of existing literature on a specific topic

D.   A description of the research methodology used in previous studies

8: The main goal of conducting a literature review is to:

A.   Prove the researcher's originality

B.   Validate the research question

C.   Determine the research design for the study

D.   Inform the development of the research hypothesis and research design

9: Which type of sources should be included in a literature review?

A.   Only peer-reviewed journal articles

B.   Any sources that are relevant to the research topic, including books, conference papers, and grey literature

C.   Only sources published in the last year

D.   Only primary research studies

10: What is the importance of critically analyzing literature in a review?

A.   To ensure that the literature review is of sufficient length

B.   To identify the most popular sources on the topic

C.   To assess the strengths and weaknesses of previous research and its relevance to the current study

D.   To exclude any sources that contradict the research hypothesis

11: When conducting a literature review, it is essential to:

A.   Only focus on recent research studies

B.   Ignore studies that have conflicting results

C.   Provide a summary of each source without any analysis

D.   Synthesize and integrate information from different sources to identify themes and trends

12: What is the purpose of organizing the literature review thematically?

A.   To alphabetize the list of sources

B.   To group sources based on their publication dates

C.   To identify common ideas and concepts in the literature

D.   To separate primary research studies from secondary sources

13: Which of the following is a benefit of conducting a comprehensive literature review?

A.   It ensures that the researcher's hypothesis is supported by all the sources.

B.   It saves time by focusing only on the most recent studies.

C.   It helps the researcher avoid duplication of previous research and build upon existing knowledge

D.   It allows the researcher to use the literature review as the main body of the research paper.

14: What should a researcher do if they encounter conflicting results in the literature review?

A.   Include only sources that align with their research hypothesis

B.   Disregard any sources that present conflicting results

C.   Address and discuss the conflicting findings to provide a balanced view

D.   Exclude the conflicting results from the literature review entirely

15: Which section of a research paper is most appropriate for including the literature review?

A.   Abstract

B.   Introduction

C.   Methodology

D.   Results

List of Review Literature in R...

Related review literature in research mcqs:, available in:, latest mcqs:.

Politics of the United States MCQs

Geographic Regions MCQs

Physical Geography MCQs

Atmospheric Energy MCQs

Human Geography MCQs

Popular MCQs:

Privacy Policy | Terms and Conditions | Contact Us

© copyright 2024 by mcqss.com

• UWF Libraries

Literature Review: Conducting & Writing

• Steps for Conducting a Lit Review

1. Choose a topic. Define your research question.

2. decide on the scope of your review., 3. select the databases you will use to conduct your searches., 4. conduct your searches and find the literature. keep track of your searches, 5. review the literature..

• Finding "The Literature"
• Organizing/Writing
• APA Style This link opens in a new window
• Chicago: Notes Bibliography This link opens in a new window
• MLA Style This link opens in a new window
• Sample Literature Reviews

Disclaimer!!

Conducting a literature review is usually recursive, meaning that somewhere along the way, you'll find yourself repeating steps out-of-order.

That is actually a good sign.  

Reviewing the research should lead to more research questions and those questions will likely lead you to either revise your initial research question or go back and find more literature related to a more specific aspect of your research question.

Your literature review should be guided by a central research question.  Remember, it is not a collection of loosely related studies in a field but instead represents background and research developments related to a specific research question, interpreted and analyzed by you in a synthesized way.

• Make sure your research question is not too broad or too narrow.  Is it manageable?
• Begin writing down terms that are related to your question. These will be useful for searches later.
• If you have the opportunity, discuss your topic with your professor.

How many studies do you need to look at? How comprehensive should it be? How many years should it cover? 

Tip: This may depend on your assignment.  How many sources does the assignment require?

Make a list of the databases you will search.  Remember to include comprehensive databases such as WorldCat and Dissertations & Theses, if you need to.

Where to find databases:

• Find Databases by Subject UWF Databases categorized by discipline
• Find Databases via Research Guides Librarians create research guides for all of the disciplines on campus! Take advantage of their expertise and see what discipline-specific search strategies they recommend!
• Review the abstracts of research studies carefully. This will save you time.
• Write down the searches you conduct in each database so that you may duplicate them if you need to later (or avoid dead-end searches   that you'd forgotten you'd already tried).
• Use the bibliographies and references of research studies you find to locate others.
• Ask your professor or a scholar in the field if you are missing any key works in the field.
• Use RefWorks to keep track of your research citations. See the RefWorks Tutorial if you need help.

Some questions to help you analyze the research:

• What was the research question of the study you are reviewing? What were the authors trying to discover?
• Was the research funded by a source that could influence the findings?
• What were the research methodologies? Analyze its literature review, the samples and variables used, the results, and the conclusions. Does the research seem to be complete? Could it have been conducted more soundly? What further questions does it raise?
• If there are conflicting studies, why do you think that is?
• How are the authors viewed in the field? Has this study been cited?; if so, how has it been analyzed?

Tips: 

• Again, review the abstracts carefully.  
• Keep careful notes so that you may track your thought processes during the research process.
• << Previous: Home
• Next: Finding "The Literature" >>
• Last Updated: Mar 22, 2024 9:37 AM
• URL: https://libguides.uwf.edu/litreview

Top 10 Questions for a Complete Literature Review

An excellent literature review integrates information in such a way that it provides a new framework to build upon. It is a way of contextualizing your work and showcasing a bigger picture before you pin down to your research problem. It not only highlights principle issues in your field but also provides new perspectives on the research topic. Careful skimming of literature introduces the readers to relevant terminologies frequently used in context of their work. Literature review assists in recognizing related research findings and relevant theories. Furthermore, it aids in pinpointing the methodologies that one may adopt for research.

5 Steps to Begin the Literature Review

There are five steps that one should follow before preparing to conduct the literature review :

• Identify all the literature relevant to your topic of interest. Explore all the different types of literature including theoretical literature, applied literature, literature that talks about research methods, or a combination thereof.
• Using multiple keywords and strategies capture the most accurate and relevant data. Conduct an extensive search in multi-disciplinary databases.
• Group your findings into a detailed summary of what is known and what needs to be explored.
• Identify existing gaps or any unresolved issues
• Formulate broad questions that warrant further research

How to Best Critique a Research paper

For extracting maximum information from a research paper , researchers must ask the following questions!

• Has the author formulated an appropriate research question based on the problem/issue?
• Is the research question clearly defined in terms of its scope and relevance?
• Was there an alternative or better perspective to approach the research question?
• What is the author’s orientation towards the research problem – is it a critical analysis or interpretation based?
• Has the author extensively evaluated the literature considering both latest and relevant articles?
• How has the author defined the basic components (population, interventions, outcomes) of the study? Are the measurements valid, accurate and statistically significant? Are the conclusions based accurate interpretations of the data?
• Is there an objective based, unbiased reasoning provided for the problem statement or is the author merely attempting to prove his/her preconceived beliefs and opinions?
• How does this article contribute to your understanding of the research problem?
• What are the strengths, limitations and shortcomings of the study?

10 Questions for a Comprehensive Literature Review

1. Do I have clearly defined research aims prior to commencing the review?

It is important to choose a focused question that can efficiently direct your search. It can assist you to create a list of keywords related to your research problem. Furthermore, it helps in identifying relevant databases to search for related journals and articles.

2. Have I correctly identified all the sources that will help me define my problem statement or research question?

Literature is not limited to journal articles, thesis, and dissertations. One should also refer to credible internet sources, conference proceedings that provide latest unpublished papers, as well as government and corporate reports. Books, although do not have latest information, can serve as a good starting point to read background information.

3. Have I considered all kinds of literature – including both qualitative and quantitative research articles?

An exhaustive literature survey helps you position your research within the context of existing literature effectively creating a case as to why further study is necessary. Your search has to be robust enough to ensure that you have browsed through all the relevant and latest articles. Rather than reading everything, researchers must refer and follow the most relevant work!

4. Do I have enough empirical or theoretical evidence to support my hypothesis?

Discovering new patterns and trends becomes easy if you gather credible evidence from earlier works. Furthermore, it helps in rationalizing the significance of your study.

5. Have I identified all the major inconsistencies or other shortcomings related to my research topic?

Researchers should not only refer to articles that present supporting evidence but also focus on those that provide inconclusive or contradictory information. It helps to identify any open questions left by researchers in previous studies.

6. Is my relationship diagram ready?

A relationship diagram is an effective way of recognizing links between different elements of a complex research topic. It is an immensely important tool that helps in clarifying and structuring research specific findings and interpretations at various stages of the project. It is an effective way of representing your current understanding of the research topic. In addition, a good relationship diagram can help you find new insights owing to a clear picture of all the probable relationships between key concepts, variables and key factors.

7. Have I gathered sufficient evidence from the literature about the accuracy and validity of the designs or methods that I plan to use in my experiments?

It is paramount to use methodologies and research techniques that have scientific reliability. Moreover, since methods especially used in qualitative research are often more subjective, it becomes crucial for researchers to reflect on the approach and explain the criteria for selecting a particular method.

8. Have I identified the purpose for which articles have been shortlisted for literature review?

You can expedite your literature writing process if you tag your articles based on its purpose of inclusion in the review report. Following are the tags that can be added to articles:

• Show how latest developments or develop a theoretical base to your study
• Demonstrate limitations, inconsistencies or shortcomings of previous studies
• Critique or support certain methods or findings
• Replicate the study in a different setting (region/population)
• Indicate how the study supports or contradicts your findings
• Use it as a reference to further build your research
• Provide a general understanding of concerns relevant to your research topic

9. Have I recorded all the bibliographic information regarding my information sources?

Recording and cataloguing your bibliographical details and references is absolutely crucial for every researcher. You may use commercial software such as Reference manager, End Note, and Pro Cite to manage your references. Furthermore, you may also keep a record of keyword searches that you have performed.

10. Will my literature review reflect a report that is created after a through critical analysis of the literature?

An excellent literature review must be structured, logical, and coherent. It is a great opportunity to demonstrate that you have critically analyzed and understood the relevant body of literature underpinning your research. It is important to structure your literature into appropriate sections that discuss themes or presents trends. Grouping your literature helps in indicating relationships and making comparisons.

Still have more queries related to literature review and synthesis? Post your queries here and our experts will be happy to answer them! You can also visit our Q&A forum for frequently asked questions related to research writing and publishing answered by our team that comprises subject-matter experts, eminent researchers, and publication experts.

Rate this article Cancel Reply

Your email address will not be published.

Enago Academy's Most Popular Articles

• Reporting Research

AI Assistance in Academia for Searching Credible Scholarly Sources

The journey of academia is a grand quest for knowledge, more specifically an adventure to…

Beyond Spellcheck: How copyediting guarantees error-free submission

Submitting a manuscript is a complex and often an emotional experience for researchers. Whether it’s…

• Old Webinars
• Webinar Mobile App

How to Find the Right Journal and Fix Your Manuscript Before Submission

Selection of right journal Meets journal standards Plagiarism free manuscripts Rated from reviewer's POV

• Manuscripts & Grants

Research Aims and Objectives: The dynamic duo for successful research

Picture yourself on a road trip without a destination in mind — driving aimlessly, not…

How Academic Editors Can Enhance the Quality of Your Manuscript

Avoiding desk rejection Detecting language errors Conveying your ideas clearly Following technical requirements

Writing a Research Literature Review? — Here are tips to guide you through!

Sign-up to read more

Subscribe for free to get unrestricted access to all our resources on research writing and academic publishing including:

• 2000+ blog articles
• 50+ Webinars
• 10+ Expert podcasts
• 50+ Infographics
• 10+ Checklists
• Research Guides

We hate spam too. We promise to protect your privacy and never spam you.

I am looking for Editing/ Proofreading services for my manuscript Tentative date of next journal submission:

As a researcher, what do you consider most when choosing an image manipulation detector?

Question(s) similar to the following:

Question 1: in conducting a literature review; the researcher:, similar questions:, question 2: while conducting a literature review, the researcher engages in:, question 3: in conducting a literature review; the researcher:.

Our VULMS adds features of MDBs and lets your populate VU subjects automatically.

Try our 3 days free demo now! Online online holy quran tajweed classes are useful to learn reading holy quran with tajweed.

Gegasoft Point of Sale/Customer Relationship Management software is an accounting software to fulfill your business needs.

Gaussion Texture

Gradient Background

• Definitions

Verbal Ability

• Interview Q

• Send your Feedback to [email protected]

Help Others, Please Share

Learn Latest Tutorials

Transact-SQL

Reinforcement Learning

R Programming

React Native

Python Design Patterns

Python Pillow

Python Turtle

Preparation

Interview Questions

Company Questions

Trending Technologies

Artificial Intelligence

Cloud Computing

Data Science

Machine Learning

B.Tech / MCA

Data Structures

Operating System

Computer Network

Compiler Design

Computer Organization

Discrete Mathematics

Ethical Hacking

Computer Graphics

Software Engineering

Web Technology

Cyber Security

C Programming

Control System

Data Mining

Data Warehouse

• - Google Chrome

Intended for healthcare professionals

• Access provided by Google Indexer
• My email alerts
• BMA member login
• Username * Password * Forgot your log in details? Need to activate BMA Member Log In Log in via OpenAthens Log in via your institution

Search form

• Advanced search
• Search responses
• Search blogs
• Guidance on...

Guidance on terminology, application, and reporting of citation searching: the TARCiS statement

• Related content
• Peer review
• Julian Hirt , research fellow and lecturer 1 2 3 ,
• Thomas Nordhausen , research fellow 4 ,
• Thomas Fuerst , medical information specialist 5 ,
• Hannah Ewald , medical information specialist 5 ,
• Christian Appenzeller-Herzog , medical information specialist 5
• on behalf of the TARCiS study group
• 1 Pragmatic Evidence Lab, Research Centre for Clinical Neuroimmunology and Neuroscience Basel, University Hospital Basel and University of Basel, Basel, Switzerland
• 2 Department of Health, Eastern Switzerland University of Applied Sciences, St Gallen, Switzerland
• 3 Department of Clinical Research, University Hospital Basel and University of Basel, Basel, Switzerland
• 4 Institute of Health and Nursing Science, Medical Faculty, Martin Luther University Halle-Wittenberg, Halle (Saale), Germany
• 5 University Medical Library, University of Basel, 4051 Basel, Switzerland
• Correspondence to: C Appenzeller-Herzog christian.appenzeller{at}unibas.ch
• Accepted 19 March 2024

Evidence syntheses adhering to systematic literature searching techniques are a cornerstone of evidence based healthcare. Beyond term based searching in electronic databases, citation searching is a prevalent search technique to identify relevant sources of evidence. However, for decades, citation searching methodology and terminology has not been standardised. An evidence guided, four round Delphi consensus study was conducted with 27 international methodological experts in order to develop the Terminology, Application, and Reporting of Citation Searching (TARCiS) statement. TARCiS comprises 10 specific recommendations, each with a rationale and explanation on when and how to conduct and report citation searching in the context of systematic literature searches. The statement also presents four research priorities, and it is hoped that systematic review teams are encouraged to incorporate TARCiS into standardised workflows.

Synthesising scientific evidence by looking at the citation relationships of a scientific record (ie, citation searching) was the underlying objective when the Science Citation Index, the antecedent of Web of Science, was introduced in 1963. 1 Although the availability of electronic citation indexes has increased, evidence syntheses in systematic reviews do not primarily rely on citation searching for literature retrieval but rather on search methods based on text and keywords. 2 When used in systematic review workflows, citation searching traditionally constitutes a supplementary search technique that builds on an initial set of references from the primary database search (seed references). 3

Citation searching is an umbrella term that entails various methods of citation based literature retrieval ( fig 1 ). Checking references cited by seed references, also known as backward citation searching, is the most prevalent and a mandatory step when conducting Cochrane reviews. 4 In forward citation searching, systematic reviewers can also assess the eligibility of articles that cite the seed references. Backward and forward citation searching are known as direct citation searching ( fig 1 ). They can be supplemented by indirect retrieval methods—namely, by co-citing citation searching (retrieving articles that share cited references with a seed reference) and co-cited citation searching (retrieving articles that share citing references with a seed reference).

Overview of citation searching methods. Direct (dark blue boxes) and indirect (light blue boxes) citation relationships of references are shown, relative to a seed reference; arrows denote the direction of citation (ie, source A citing target B); horizontal axis denotes time (ie, the chronology in which references were published relative to the seed reference). Visual examples of cited references (accessible via backward citation searching), citing references (accessible via forward citation searching), co-citing references (accessible via co-citing citation searching), and co-cited references (accessible via co-cited citation searching) are shown. Note that the total number of the co-citing and co-cited references of a seed reference far exceeds the number shown in the light blue boxes

• Download figure
• Open in new tab
• Download powerpoint

Citation searching can contribute substantially to evidence retrieval and can show similar or even superior effectiveness and efficiency compared with text and keyword based searches. An audit of the different search methods used in a systematic review of complex evidence, for instance, revealed that 44% of all included studies were identified by backward citation searching, and 7% by forward citation searching. In comparison, initial text and keyword searches accounted for only 25% of included studies. 5 For the scoping review that collected methodological studies as a foundation for the present work, these figures were 28% and 12% for backward and forward citation searching, respectively, compared with 52% for extensive primary database searching. 6

The conduct of systematic reviews is prominently guided by standard recommendations such as those in the Cochrane handbook, 4 whereas their reporting is standardised by the Preferred Reporting Items for Systematic reviews and Meta-Analyses (PRISMA) statement. 7 In contrast and despite its application by systematic reviewers for decades, standardised methodology and terminology for citation searching is not available. Of the three aspects on when to do citation searching, how to conduct citation searching, and how to report citation searching, limited guidance exists only for the third aspect in the PRISMA extension for reporting literature searches (PRISMA-S). 8 Unsurprisingly, methodological studies show considerable heterogeneity in terms of citation searching terminology and recommended best practices. 6 Even in a sample of Cochrane reviews, 13% did not use backward citation searching despite this being a mandatory step. 9 The lack of standardisation not only impairs the transparency, reproducibility, and comparability of systematic reviews, but might also reduce article recall that could affect pooled effect estimates, guidance, and clinical decision making. On the other hand, uninformed use of citation searching in contexts where it is less useful might cause undue workloads.

We systematically collected evidence on the use, benefit, and reporting of citation searching 6 and put it through a four round, online Delphi study. Together with the Terminology, Application, and Reporting of Citation Searching (TARCiS) study group, an international panel of methodological experts, we aimed to develop consensus for recommendations on when and how to conduct citation searching, and on how to report it, including a consensus set of citation searching terms. Furthermore, we framed research priorities for future methodological development of citation searching in the context of systematic literature searches.

Summary points

The TARCiS (Terminology, Application, and Reporting of Citation Searching) statement provides guidance in which contexts citation searching is likely to be beneficial for systematic reviewers

TARCiS comprises 10 specific recommendations on when and how to conduct citation searching and how to report it in the context of systematic literature searches, and also frames four research priorities

The statement will contribute to a unified terminology, systematic application, and transparent reporting of citation searching and support those who are conducting or assessing citation searching methods

To develop the TARCiS statement, a stepwise approach comprising a scoping review of the methodological literature (step 1; reported in detail in a separate publication 6 ) and a Delphi study (step 2; reported in this publication) was chosen. The methods were prespecified in two study protocols. 10 11 The complete process is shown in figure 2 .

Flow diagram of the development process of the TARCiS (Terminology, Application, and Reporting of Citation Searching) statement. Actions and outcomes of the development phases of the TARCiS statement are shown. Appendix 1 shows more detailed reporting of consensus scores

Step 1: Scoping review

We conducted a scoping review on the terminology that describes citation searching, the methods and tools used for citation searching, and its benefit. We considered methodological studies of any design that aimed to assess the role of citation searching, compared multiple citation searching methods, or compared technical uses of citation searching within health related topics. We searched five bibliographic databases, conducted backward and forward citation searches of eligible studies and pertinent reviews, and consulted librarians and information specialists for further eligible studies. The results were summarised by descriptive statistics and narratively. The detailed methods of the scoping review have been published elsewhere. 6 10

Step 2: Delphi study

To develop consensus on recommendations and research priorities as tentatively derived from the results of step 1, 6 we performed a multistage online Delphi study. Delphi refers to a structured process where collective knowledge from an expert panel is synthesised using a series of questionnaires, each one questionnaire adapted on the basis of the responses to a previous version. 12 13 14 We recruited an international panel of individuals experienced in conducting or reporting citation searching methods. For this, we invited authors of methodological studies, as identified in step 1, 6 and methodological experts from international systematic review organisations or from our professional networks by email to participate in the Delphi study.

The Delphi study comprised four prespecified rounds. 10 11 The first round was pretested by four non-study related academic affiliates. Each round covered four to five thematic parts (appendix 2; table 1 ). Briefly, part A dealt with the terminology framework to describe citation searching methods in eight domains (for details, refer to table 4 in Hirt et al 6 ). Part B contained pre-formulated recommendations on conduct and reporting of citation searching. Each recommendation was supported by a rationale and explanation text that were also subjected to collective consensus finding. Part C covered research priorities that were also derived from the scoping review. 6 Part D contained a free text field to collect general comments from the panellists. Part E was designed to collect sociodemographic information and was limited to Delphi round 1.

Data collection through four rounds of Delphi study to develop consensus on recommendations and research priorities of the TARCiS statement

• View inline

Non-participating panellists were recorded as non-participators for a given round. Panellists who missed all rounds were recorded as non-responders. Recommendations and research priorities that had not yet reached the prespecified consensus of at least 75% were refined for the subsequent Delphi round. These refinements were based on the panellists’ comments. In rare cases, when additional valid suggestions from panellists for reformulation of rationale or explanation texts were submitted, recommendations that already reached the agreement threshold were also adapted and forwarded to the next Delphi round. For more methodological details on the Delphi study, see table 1 and the published protocols. 10 11

Deviations from the Delphi study protocol

For round 3 of the Delphi, we had originally planned to formulate one recommendation for each of the eight terminology domains ( table 1 , see also description to part A above). Depending on the votes, however, this approach might have led to the selection of inconsistent terms (eg, backward citation searching v forward citation tracking). Hence, we decided to use the terms that received the most votes in Delphi round 2 to formulate four term sets, which were consistent across all eight domains. Secondly, instead of using SosciSurvey 15 as a survey tool, 8 we switched to the Unipark/Enterprise Feedback Suite survey, 16 which provided enhanced design and functional features. Thirdly, in addition to personalised emails (person based approach), we originally intended to recruit panellists using professional mailing lists and central requests to systematic review organisations (organisation based approach). 8 However, because we had already recruited sufficient panellists using the person based approach (including individuals who were affiliated with various systematic review organisations), we waived the organisation based approach.

We identified 47 methodological studies that assessed the use, benefit, and reporting of citation searching. In 45 studies (96%), the use of citation searching showed an added value. Thirty two studies (68%) analysed the impact of citation searching in one or more previous systematic reviews. Application, terminology, and reporting of citation searching were heterogeneous. Details on the results of the scoping review can be found elsewhere. 6

Recruitment and characteristics of panellists

Of 35 experts identified and contacted, 30 declared an interest in participating and were invited to Delphi round 1. Three (10%) of the 30 panellists were non-responders. Table 2 summarises the personal and professional characteristics of the 27 participating panellists.

Characteristics of 27 panellists* participating in the Delphi study to develop consensus on recommendations and research priorities of the TARCiS statement

TARCiS statement: final recommendations, rationale and explanations, and research priorities

Items for data collection through the four Delphi rounds in parts A-E are summarised in table 1 . The Delphi study started with 41 terms describing different aspects of citation searching, eight draft recommendations with rationale texts on the conduct and reporting of citation searching, and one research priority (appendix 1). After Delphi round 4, the finalised TARCiS statement comprised 10 recommendations with rationale and explanation texts and four research priorities that reached consensus scores between 83% and 100%. Figure 2 and appendix 1 show details on content and consensus scores in rounds 1-4. An overview of all 14 TARCiS items omitting rationale and explanation texts is presented in box 1 . A terminology and reporting item checklist based on TARCiS recommendations 1 and 10 is available in appendix 3 and on the TARCiS website. 17

TARCiS statement

Recommendations on terminology, conduct, and reporting of citation searching.

The following terminology should be used to describe search methods that exploit citation relationships:

“Citation searching” as an umbrella term.

“Backward citation searching” to describe the sub-method retrieving and screening cited references.

“Reference list checking” to describe the sub-method retrieving and screening cited references by manually reviewing reference lists.

“Forward citation searching” to describe the sub-method retrieving and screening citing references.

“Co-cited citation searching” to describe the sub-method retrieving and screening co-cited references.

“Co-citing citation searching” to describe the sub-method retrieving and screening co-citing references.

“Iterative citation searching” to describe one or more repetition(s) of a search method that exploits citation relationships.

“Seed references” to describe relevant articles that are known beforehand and used as a starting point for any citation search.

For systematic search topics that are difficult to search for, backward and forward citation searching should be seriously considered as supplementary search techniques.

For systematic search topics that are easier to search for and addressed by a highly sensitive search, backward and forward citation searching are not explicitly recommended as supplementary search techniques. Reference list checking of included records can be used to confirm the sensitivity of the search strategy.

Backward and forward citation searching as supplementary search techniques should be based on all included records of the primary search (ie, all records that meet the inclusion criteria of the review after full text screening of the primary search results). Occasionally, it can be justified to deviate from this recommendation and either use further pertinent records as additional seed references or only a defined sample of the included records.

Backward citation searching should ideally be conducted by screening the titles and abstracts of the seed references as provided by a citation index. Screening titles as provided when checking reference lists of the seed references can still be performed.

Using the combined coverage of two citation indexes for citation searching to achieve more extensive coverage should be considered if access is available. This combination is especially meaningful if seed references cannot be found in one index and reference lists were not checked.

Before screening, the results of supplementary backward and forward citation searching should be deduplicated.

If citation searching finds additional eligible records, another iteration of citation searching should be considered using these records as new seed references.

Standalone citation searching should not be used for literature searches that aim at completeness of recall.

Reporting of citation searching should clearly state:

the seed references (along with a justification should the seed references differ from the set of included records from the results of the primary database search),

the directionality of searching (backward, forward, co-cited, co-citing),

the date(s) of searching (which might differ between rounds of iterative citation searching) (not applicable for reference list checking),

the number of citation searching iterations (and possibly the reason for stopping if the last iteration still retrieved additional eligible records),

all citation indexes searched (eg, Lens.org, Google Scholar, Scopus, citation indexes in Web of Science) and, if applicable, the tools that were used to access them (eg, Publish or Perish, citationchaser),

if applicable, information about the deduplication process (eg, manual/automated, the software or tool used),

the method of screening (ie, state whether the records were screened in the same way as the primary search results or, if not, describe the alternative method used), and

the number of citation searching results in the right column box of the PRISMA 2020 flow diagram for new or updated systematic reviews that included searches of databases, registers, and other sources .

Research priorities

The effectiveness, applicability, and conduct of indirect citation searching methods as supplementary search methods in systematic reviewing require further research (including retrieval of additional unique references, their relevance for the review and prioritisation of results).

Further research is needed to assess the value of citation searching. Potential research topics could be:

influence of citation searching on results and conclusions of systematic evidence syntheses,

topics or at least determinants of topics where citation searching likely/not likely has additional value, or

economic evaluation of citation searching to assess the cost and time of conducting citation searching in relation to its benefit.

Further research is needed to assess the best way to perform citation searching. Potential research topics could be:

optimal selection of seed references,

optimal use of indexes and tools and their combination to conduct citation searching,

methods and tools for deduplication of citation searching results,

subjective influences on citation searching (eg, experience of researcher, prevention of mistakes), or

reproducibility of citation searching.

Further research is needed to reproduce existing studies: Any recommendations in this Delphi that are based on only 1-2 studies require reproduction of these studies in the form of larger, prospectively planned studies that grade the evidence for each recommendation and propose additional research where the grade of evidence is weak.

The TARCiS checklist for terminology and reporting of citation searching is available for download. 17

PRISMA=Preferred Reporting Items for Systematic reviews and Meta-Analyses; TARCiS=Terminology, Application, and Reporting of Citation Searching.

Recommendation 1

Rationale and explanation supporting recommendation 1.

As compiled in a recent scoping review, 6 the reporting of citation searching methods is frequently unclear and far from being standardised. For example, “citation searching,” “snowballing,” or “co-citation searching” are sometimes used as methodological umbrella terms but also to denote a specific method such as backward or forward citation searching. 6 For clarity, standardised vocabulary is needed. The set of terms brought forward in this recommendation is consistent in itself as well as with the terminology used in PRISMA-S and PRISMA 2020 guidelines 8 18 and hence well suited for uniform reporting of citation searching.

Recommendation 2

Rationale and explanation supporting recommendation 2:.

Evidence indicates that the ability of citation searching as a supplementary search technique to find additional unique records in a systematic literature search varies between reviews. 6 Searches for particular study designs (qualitative, mixed method, observational, prognostic, or diagnostic test studies) or health science topics such as non-pharmacological, non-clinical, public health, policy making, service delivery, or alternative medicine have been linked with effective supplementary citation searching. 19 20 21 22 The underlying reasons include poor transferability to text based searching owing to poor conceptual clarity, inconsistent terminology, or vocabulary overlaps with unrelated topics. 5 The ability of citation searching to find any publication type including unpublished or grey literature or literature that is not indexed in major databases (eg, concerning a developing country) might also be relevant. 23 However, a clear categorisation of topics that are difficult to search for is currently not possible and it remains for the review authors themselves to judge whether their review topic is likely to fall into this category.

For people conducting the search who have difficulty assessing whether the topic is difficult or easier to search for, we recommend that they opt for citation searching or consult an experienced information specialist. 24 If the search strategy does not exhaustively capture the topic, backward and forward citation searching might compensate for some of the potential loss of information.

Recommendation 3

Rationale and explanation supporting recommendation 3.

Evidence indicates that the ability of citation searching as a supplementary search technique to find additional unique references in a systematic literature search varies between reviews. 6 Searches for clearly defined clinical interventions as part of PICO (participant, intervention, comparison, outcome) questions have been linked with less effective supplementary citation searching, especially when the search strategies are sensitive and conducted in several databases. However, a clear categorisation of topics that are easier to search for is currently not possible, and it remains for the review authors themselves to judge whether their review topic is likely to fall into this category.

By checking reference lists within the full texts of seed references, review authors can test the sensitivity of their primary search strategy (ie, electronic database search). 25 If no additional relevant, unique studies are found, the primary search might have been sensitive enough. If additional relevant, unique studies are found, these could indicate that the primary search was not sensitive enough.

For individuals conducting the search who have difficulty assessing whether the topic is difficult or easier to search for, we recommend that they opt for citation searching or consult an experienced information specialist. 24 If, for whatever reason, the search strategy does not exhaustively capture the topic, backward and forward citation searching could compensate for some of the potential loss of information.

Recommendation 4

Rationale and explanation supporting recommendation 4.

The more seed references used, the better the chance that citation searching finds additional relevant unique records. While using only a sample of the included records as seed references might be enough, there is currently no evidence that could help decide how many seeds are needed or how to decide which might perform better. Hence, we recommend using all the records that meet the inclusion criteria of the review after full text screening of the primary database search results.

However, review authors could deviate from this recommendation if they deal with a very small or large number of included records. A very small number of included records might not yield additional relevant records or only have limited value. In this case, review authors could use further records as seed references for citation searching (eg, systematic reviews on the topic that were flagged during the screening phase). 26 A very large number of included records could lead to too many records to screen. In this case, review authors might use a selected sample of included records as seed references for citation searching. In the event of such deviation, authors should describe their rationale and sampling method (eg, random sample).

Recommendation 5

Rationale and explanation supporting recommendation 5.

Citation searching workflows encompass two consecutive steps: retrieval of records and screening of retrieved records for eligibility. When using an electronic citation index for citation searching, retrieval and screening are usually separated. While forward citation searching requires a citation index, backward citation searching can also be performed by manually checking the reference lists of the seed references. Reference list checking is sometimes part of an established workflow, for example, during the eligibility assessment of the full text record or during data extraction. 25 Merging these two steps allows researchers to know the context in which a reference was used and that all references can be screened. However, reference list checking has three disadvantages:

The retrieval and screening phases are no longer separated, which makes reporting of the methods or results difficult and unclear

Citations from reference list checking cannot be deduplicated against each other or against the primary search results, which could add an unnecessarily high workload (see recommendation 7)

Eligibility assessments are restricted to the titles (instead of titles and abstracts) which could lead to relevant records being overlooked due to uninformative titles mentioned in vague contexts.

In recent years, online citation searching options via citation indexes or free to access citation searching tools have become more readily available leading to faster and easier procedures. 27 28 29 30 More and even better tools to facilitate this workflow are expected in the future. Combining citation searching via citation indexes with automated deduplication (free online tools available) 31 32 33 makes this recommendation feasible. A caveat is that a search in one citation index will in most cases fail to retrieve all the cited references. 34 35 Thus, references to some documents (such as websites, registry entries, or grey literature) that are less likely to be indexed in databases might only be retrievable by checking reference lists or only in some citation indexes. 3

Recommendation 6

Rationale and explanation supporting recommendation 6.

A single citation index or citation analysis tool might not cover all seed references and is likely to not find all the citing and cited literature. Citation indexes do not offer 100% coverage because some references are currently not indexed in one or several citation index(es) 36 and because of data quality problems. 37 Evidence indicates that when using more than one citation index for citation searching, the results of the different indexes can complement each other. 38 39 40 Thus, retrieval of backward and forward citation searching results from more than one citation index or citation analysis tool (eg, Lens.org via citationchaser, Scopus, citation indexes in Web of Science) followed by deduplication (see recommendation 7) can increase the sensitivity of citation searching. It is similar to the complementary effect of using multiple electronic databases for the primary database search, which is the preferred method in systematic search workflows. 4 In recent years, online citation searching options have increased and many open access tools make rapid electronic citation searching universally accessible. 27 28 29 30

Recommendation 7

Rationale and explanation supporting recommendation 7.

The concept of citation searching as a supplementary search method relies on the notion that reference lists and cited-by lists of eligible references are topically related to these references. 6 This topical relation implies a considerable degree of overlap within these lists leading to several duplicates. Furthermore, the overlap likely also extends to the results of the primary database search that was performed on the same topic. Based on these considerations and on the fact that the results of the primary database search have already been screened for eligibility, the screening load of citation searching results can be substantially cut by removing those references that have already been screened for eligibility (deduplication against the primary database search) and those references that appear as duplicates during citation searching. 34 Depending on the method of deduplication, this procedure can be done in one go.

While deduplication can be conducted manually, standard bibliographic management software and specialised tools currently provide automated deduplication solutions, allowing for easier and faster processing. 34 41 42 If citation searching leads to only very few results, omission of the deduplication step can be considered to save time and administrative effort.

Recommendation 8

Rationale and explanation supporting recommendation 8.

Citation searching methods can be conducted over one or more iterations, a process that we refer to as iterative citation searching. 43 The first iteration is based on the original seed references (see recommendation 4). If eligibility screening of the results of this first iteration leads to the inclusion of further eligible records, these records serve as new seed references for the second iteration, and so forth. Evidence indicates that conducting iterative citation searching can contribute to the identification of more eligible records. 6 43 44 45

Iterations beyond the first round of citation searching require additional time and effort and could interrupt the ongoing review process, so the decision in favour of or against further iterations should be guided by an informal cost-benefit assessment. Relevant factors to be assessed include the review topic (difficult or easier to search for), sensitivity of the primary search, aim for completeness of the literature search, and the estimated potential benefit of the iteration(s) (eg, based on the number or percentage of included records found with the previous citation searching iteration).

Review authors should report the number of iterations and possibly the reason for stopping if the last iteration still retrieved additional eligible records. Furthermore, stating “citation searching was done on all included records” can lead to confusion. Most authors might mean all records were included after full text screening of the primary search results. But strictly speaking, “all included records” also includes those records retrieved via citation searching. The second interpretation implies that iterative citation searching is required until the last iteration leads to no further identification of eligible records.

As outlined in the rationale of recommendation 7, results of citation searching iterations can be deduplicated against all previously retrieved records to reduce the screening load.

Recommendation 9

Rationale and explanation supporting recommendation 9.

We refer to standalone citation searching when any form of citation searching is used as the primary search method without extensive prior database searching. 6 This is contrary to citation searching as a supplementary search method to a primary database search. Seed references for standalone citation searching could, for example, be records from researchers’ personal collections or retrieved from less sensitive literature searches. Standalone citation searching can be based on a broad set of seed references. It can comprise backward and forward citation searching as well as indirect methods that collect co-citing and co-cited references.

When study authors have replicated published systematic reviews with standalone citation searching, they have mostly missed literature that was included in the systematic review. 27 46 47 48 Since search methods for systematic reviews and scoping reviews should aim at completeness of recall, standalone citation searching is not a suitable method for these types of literature review.

Recommendation 10

Rationale and explanation supporting recommendation 10.

Relevant guidance for researchers conducting citation searching in systematic literature searching can be found in item 5 of PRISMA-S. 8 Accordingly, required reporting items are the directionality of citation searching (examination of cited or citing references), methods and resources used for citation searching (bibliographies in full text articles or citation indexes), and the seed references that citation searching was performed on. 8 Additional information for the reporting of citation searching can be found in PRISMA-S items 1 (database name), 13 (dates of searches), and 16 (deduplication). 8 Although PRISMA-S can be seen as the minimum reporting standard for citation searching as a supplementary search technique, other important elements that emerged from our scoping review 6 need to be reported to achieve full transparency or reproducibility. These elements are listed in recommendation 10 as a supplement to PRISMA-S to comprehensively guide the reporting of supplementary citation searching in systematic literature searching.

Concerning reporting of citation searching results in the PRISMA 2020 flow diagram, 49 two variants are possible: reporting only those records that are additional to the primary search results after deduplication, or reporting all retrieved records followed by insertion of an additional box where the number of deduplicated records is reported.

Researchers should be aware that the detail of the citation searching methods do not have to be reported in the main methods of a study. Detailed search information can be provided in an appendix or an online public data repository.

Examples of good reporting

“As supplementary search methods, we performed . . . direct forward and backward CT [citation searching] of included studies and pertinent review articles that were flagged during the screening of search results (on February 10, 2021). For forward CT, we used Scopus, Web of Science [core collection as provided by the University of Basel; Editions = SCI-EXPANDED, SSCI, A&HCI, CPCI-S, CPCI-SSH, BKCI-S, BKCI-SSH, ESCI, CCR-EXPANDED, IC], and Google Scholar. For backward CT, we used Scopus and, if seed references were not indexed in Scopus, we manually extracted the seed references’ reference list. We iteratively repeated forward and backward CT on newly identified eligible references until no further eligible references or pertinent reviews could be identified (three iterations; the last iteration on May 5, 2021).” 6

“To supplement the database searches, we performed a forward (citing) and backwards (cited) citation analysis on 2 August 2022 using SpiderCite ( https://sr-accelerator.com/#/spidercite ).” 50

“Reference lists of any included studies and retrieved relevant SRs [systematic reviews] published in the last five years were checked for any eligible studies that might have been missed by the database searches.” 51

Research priority 1

Rationale and explanation supporting research priority 1.

Indirect citation searching involves the collection and screening for eligibility of records that share references in their bibliography or citations with one of the seed references (ie, co-citing or co-cited references). 10 Indirect citation searching typically retrieves a large volume of records to be screened. 46 48 Therefore, prioritisation algorithms for the screening of records and cut-off thresholds that might discriminate between potentially relevant and non-relevant records have been proposed with the aim to reduce the workload of eligibility screening. 27 47 The methodological studies that have pioneered indirect citation searching methods for health related topics have so far exclusively focused on standalone citation searching. 6 It is currently unclear whether the added workload and resources for searching and screening warrant indirect citation searching methods as supplementary search techniques in systematic reviews of any type (qualitative or quantitative studies, difficult or easier topics to search for).

Research priority 2

Research priority 3, research priority 4, tarcis recommendations and research priorities.

In keeping with our study aims, the TARCiS recommendations cover three aspects of citation searching in the context of systematic literature searches. They offer guidance regarding when to conduct, how to conduct, and how to report citation searching. The strength of each recommendation reflects the panellists’ assessment of the strength of evidence to support them.

In systematic evidence syntheses, citation searching techniques can be used to fill gaps in the results of primary database searches, but their application is not universally indicated. TARCiS recommendations 2 and 3 provide critical assistance in cost-benefit considerations (ie, whether a systematic search is likely to benefit from the use of citation searching). Systematic searchers of defined pharmaceutical interventions, for instance, might take from this guidance that they can skip citation searching because their primary database search might already allow for high sensitivity at reasonable specificity and expedite other supplementary search techniques, such as clinical trial registry searching. 52 Accordingly, TARCiS does not recommend the use of citation searching in easier-to-search-for topics, but—as formulated in research priority 2—more research is needed to more reliably discriminate between topics that are easier to search for and those that are difficult to search for.

TARCiS recommendations 4-8 comprise guidance for technical aspects of citation searching. This guidance includes the selection of seed references, use of electronic citation indexes, deduplication, and iterative citation searching. While composing these recommendations, the TARCiS study group has considered that individual workflows must be framed in line with institutional licenses for subscription only databases and software. For illustration, one such workflow that is based on the licenses as provided by the University of Basel was deposited as an online video. 53

Concerning guidance for reporting of citation searching, we developed a consensus terminology set for citation searching methods (TARCiS recommendation 1) as well as a recommendation for preferred reporting items for citation searching (TARCiS recommendation 10), along with a downloadable checklist. 17 TARCiS recommendation 10 increases the reporting standards provided by PRISMA-S 8 by dealing with the reporting of citation searching iterations, software tools that facilitate citation searching via a citation index, and the method of eligibility screening. Furthermore, TARCiS recommendation 10 standardises the reporting of citation searching results in the PRISMA 2020 flow diagram. We suggest that systematic reviewers, methodologists, journal reviewers, and editors use the TARCiS statement terminology and reporting checklist 17 (appendix 3) as an additional checklist until future work by the PRISMA-S study group produces an updated reporting guideline that renders the TARCiS checklist obsolete.

Dissemination

TARCiS is intended to be used by researchers, systematic reviewers, information specialists, librarians, editors, peer reviewers, and others who are conducting citation searching or assessing citation searching methods. To enhance dissemination among these stakeholders, we aim to provide additional open access publications in scientific and non-scientific journals relevant in the field of information retrieval and evidence syntheses.

We have launched a TARCiS website ( https://tarcis.unibas.ch/ ) and plan to disseminate the TARCiS terminology and reporting checklist 17 on various platforms, including EQUATOR. We aim to make the TARCiS statement available via the Library of Guidance for Health Scientists (LIGHTS), a living database for methods guidance 54 ; the Systematic Review Toolbox, an online catalogue of tools for evidence syntheses 55 ; and ResearchGate, a social scientific network to share and discuss publications.

We will also share the TARCiS terminology and reporting checklist 17 with editors of journals relevant in the field of information retrieval and evidence syntheses to request for inclusion in their instructions for authors and raise awareness of this topic. We hope that this effort will guide authors and peer reviewers to use TARCiS and assist their conduct, reporting, and evaluation of citation searching. We will also share the TARCiS statement with primary teaching stakeholders in evidence syntheses and systematic literature searching (eg, York Health Economics Consortium, RefHunter, Cochrane, Joanna Briggs Institute, and the Campbell Collaboration) and suggest its inclusion in future editions of their handbooks. We will present and discuss the TARCiS statement on international conferences and share our publications and presentations via relevant mailing lists and newsletters, X (formerly Twitter), and LinkedIn.

Limitations

A limitation of the TARCiS statement is that, despite the expectation and intent to recruit panellists from all parts of the world, their locations were limited to Australia, Europe, and North America. In addition, only a few panellists were recruited from countries where English was not the dominant language. Furthermore, both the evidence collected in our scoping review and the participating panellists are primarily involved with health related research. These considerations might reduce the generalisability of our recommendations and research priorities in other countries, languages, and research areas.

Conclusions

TARCiS comprises 10 specific recommendations on when and how to conduct citation searching and how to report it in the context of systematic literature searches. Furthermore, TARCiS frames four research priorities. It will contribute to a unified terminology, systematic application, and transparent reporting of citation searching and support researchers, systematic reviewers, information specialists, librarians, editors, peer reviewers, and others who are conducting or assessing citation searching methods. In addition, TARCiS might inform future methodological research on the topic. We encourage systematic review teams to incorporate TARCiS into their standardised workflows.

Ethics statements

Ethical approval.

This study is based on published information and uses surveys of topical experts and therefore did not fall under the regulations of the Swiss Human Research Act, and we did not need to apply for ethical approval according to Swiss law. Data protection and privacy issues for the survey are outlined in the main text.

Data availability statement

The survey sheets and questionnaires used for this study are included in the supplementary content. Data generated and analysed during this study (except for sociodemographic information) are available on the Open Science Framework ( https://osf.io/y7kh3 ).

Acknowledgments

We would like to dedicate this work to Cecile Janssens, who died soon after agreeing to join our Delphi panel. We thank Jill Hayden (Dalhousie University) and Claire Duddy (UK) for participating in our Delphi panel; and Christian Buhtz (Martin Luther University Halle-Wittenberg), Jasmin Eppel-Meichlinger (Karl Landsteiner University of Health Sciences), Tania Rivero (University of Berne), and Monika Wechsler (University of Basel) for participating in the pretest of the Delphi survey.

TARCiS study group: Alison Avenell (University of Aberdeen, UK), Alison Bethel (University of Exeter, UK), Andrew Booth (University of Sheffield, UK; and University of Limerick, Ireland), Christopher Carroll (University of Sheffield, UK), Justin Clark (Bond University, Australia), Julie Glanville (Glanville.info, UK ), Su Golder (University of York, UK), Elke Hausner (Institute for Quality and Efficiency in Health Care, Germany), Tanya Horsley (Royal College of Physicians and Surgeons of Canada, Canada), David Kaunelis (Canadian Agency for Drugs and Technologies in Health, Canada), Shona Kirtley (University of Oxford, UK), Irma Klerings (Donau University, Austria), Jonathan Koffel (USA), Paul Levay (National Institute for Health and Care Excellence, UK), Kathrine McCain (Drexel University, USA), Maria-Inti Metzendorf (Heinrich-Heine University Duesseldorf, Germany), David Moher (University of Ottawa, Canada), Linda Murphy (University of California at Irvine, USA), Melissa Rethlefsen (University of New Mexico, USA), Amy Riegelman (University of Minnesota, USA), Morwenna Rogers (University of Exeter, UK), Margaret J Sampson (Children’s Hospital of Eastern Ontario, Canada), Jodi Schneider (University of Illinois at Urbana-Champaign, USA), Terena Solomons (Curtin University, Australia), Alison Weightman (Cardiff University, UK)

Contributors: All authors made substantial contributions to conception and design, or acquisition of data, or analysis and interpretation of data; drafted the article or revised it critically for important intellectual content; and approved the final version to be published. JH, TN, TF, HE, and CA-H had full access to all of the data in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. JH, TN, TF, HE, and CA-H contributed to the study concept and methodology; acquisition, analysis, interpretation, validation, and visualisation of data; and critical revision of the manuscript for important intellectual content. JH, HE, and CA-H conducted the statistical analysis. JH and CA-H drafted the manuscript; provided administrative, technical, and material support; and supervised the study. The TARCiS study group authors are the Delphi panellists who were involved in Delphi rounds 1-4; they received the final manuscript draft for critical revision, important intellectual input, and approval for publication. CA-H is the guarantor for the study. The corresponding author attests that all listed authors meet authorship criteria and that no others meeting the criteria have been omitted.

Funding: The authors did not receive a specific grant for this study.

Competing interests: All authors have completed the ICMJE uniform disclosure form at www.icmje.org/disclosure-of-interest/ and declare: no specific support for the submitted work. CA-H received payments to his institution for a citation searching workshop by the University of Applied Sciences Northwestern Switzerland. JH received consulting fees from Medical University Brandenburg and payments for lecturing from the University of Applied Sciences Northwestern Switzerland, Catholic University of Applied Sciences, and Netzwerk Fachbibliotheken Gesundheit. From the TARCiS study group: JS received support from Alfred P Sloan Foundation; was funded by the US National Institutes of Health, National Science Foundation, US Office of Research Integrity, United States Institute of Museum and Library Services, and University of Illinois Urbana-Champaign; received book royalties from Morgan and Claypool; received consulting fees or honorariums from the European Commission, Jump ARCHES, NSF, and the Medical Library Association; received travel support by UIUC; contributes to the CREC (Communication of Retractions, Removals, and Expressions of Concern) Working Group; has non-financial associations with Crossref, COPE (Committee on Publication Ethics), the International Association of Scientific, Technical and Medical Publishers, the National Information Standards Organisation, and the Center for Scientific Integrity (parent organisation of Retraction Watch); and declares the National Information Standards Organisation as a subawardee on her Alfred P Sloan Foundation grant G-2022-19409. JG received payments for lecturing by York Health Economics Consortium. MJS received consulting fees at the Canadian Agency for Drugs and Technologies in Health and National Academy of Medicine (formerly Institute of Medicine) and for lecturing and support for attending a meeting at Institute for Quality and Efficiency in Health Care; and has a leadership role as secretary of the Ottawa Valley Health Library Association. AW received payments to her institution for a citation analysis workshop run via York Health Economics Consortium. SK declares non-financial interests as a member of the UK EQUATOR Centre and a coauthor of the PRISMA-S reporting guideline and was funded by Cancer Research UK (grant C49297/A27294); the current work was unrelated to this funding. PL is an employee of the National Institute for Health and Care Excellence. MR received payments by the Medical Library Association and declares non-financial interests as a member of the PhD programme affiliated with BMJ Publishing Group. ABo is a co-convenor of the Cochrane Qualitative and Implementation Methods Group and has authored methodological guidance on literature searching. All the other authors have no competing interests to disclose.

Provenance and peer review: Not commissioned; externally peer reviewed.

• Varley-Campbell J ,
• Britten N ,
• ↵ Higgins J, Thomas J, Chandler J, et al. Cochrane Handbook for Systematic Reviews of Interventions version 6.3 (updated February 2022): Cochrane; 2022. www.training.cochrane.org/handbook .
• Greenhalgh T ,
• Nordhausen T ,
• Appenzeller-Herzog C ,
• Bossuyt PM ,
• Rethlefsen ML ,
• Kirtley S ,
• Waffenschmidt S ,
• PRISMA-S Group
• Briscoe S ,
• ↵ Hirt J, Nordhausen T, Fuerst T, et al. Internal DELPHI Protocol. 2023. https://osf.io/4nh25 .
• Radbruch L ,
• Brearley SG
• Pawlowski SD
• Schulz KF ,
• ↵ Leiner DJ. SoSci Survey (Version 3.1.06) [Computer software]. 2019. https://www.soscisurvey.de .
• ↵ Gmb HTX. Unipark: 2023. https://www.unipark.com/en .
• ↵ Hirt J, Nordhausen T, Fuerst T, et al. TARCiS terminology and reporting item checklist. 2023. https://bit.ly/tarcispdf .
• McKenzie JE ,
• Preston L ,
• Carroll C ,
• Gardois P ,
• Paisley S ,
• Kaltenthaler E
• Lasda Bergman EM
• Haddaway NR ,
• Collins AM ,
• Coughlin D ,
• Livingston EH
• Horsley T ,
• Dingwall O ,
• Westphal A ,
• Kriston L ,
• Hölzel LP ,
• von Wolff A
• Janssens ACJW ,
• Brockman JE ,
• Grainger MJ ,
• Smalheiser NR ,
• Schneider J ,
• Torvik VI ,
• Fragnito DP ,
• Pallath A ,
• Ouzzani M ,
• Hammady H ,
• Fedorowicz Z ,
• Elmagarmid A
• ↵ Zotero Version 6. 2023. https://www.zotero.org:443 .
• Glasziou P ,
• Del Mar C ,
• Bannach-Brown A ,
• Stehlik P ,
• ↵ ClarivateAnalytics. Cited Reference Search. 2023. https://webofscience.help.clarivate.com/en-us/Content/cited-reference-search.htm .
• Falagas ME ,
• Pitsouni EI ,
• Malietzis GA ,
• Franceschini F ,
• Maisano D ,
• Mastrogiacomo L
• Ainsworth N ,
• Rodriguez-Lopez R
• Borissov N ,
• Lines RLJ ,
• Gucciardi DF ,
• ↵ Janssens AC. Updating systematic reviews and meta-analyses, the easy way: 2021. https://cecilejanssens.medium.com/updating-systematic-reviews-and-meta-analyses-the-easy-way-cbb2e23b48b9 .
• Kleijnen J ,
• Knipschild P
• Janssens AC ,
• Nightingale R ,
• Fautrel B ,
• Patterson J ,
• Hunter KE ,
• Webster AC ,
• ↵ Hirt J, Nordhausen T, Fuerst T, et al. Citation searching in multiple citation indexes. 2023. https://osf.io/jaeu5 .
• Schönenberger CM ,
• ↵ Marshall C, Sutton A, O’Keefe H, et al. The Systematic Review Toolbox: 2022. http://www.systematicreviewtools.com/ .