Systematic Review and Meta-Analysis

Platform: Open Learning Initiative
Duration: Self-paced
Certification: No
Collaborators: Campbell Collaboration and Carnegie Mellon University 

Open Learning Initiative (OLI) at Carnegie Mellon creates research-based courses and tools that help students learn faster and support effective teaching. According to their findings, students achieved comparable learning outcomes with significantly less instructional time.

Systematic Review and meta-analysis are considered the highest level of scientific evidence; thus, they teach rigorous, internationally accepted standards and guidelines for formulating research questions, identifying and evaluating study quality, synthesising results, and finally reporting.

The Systematic Reviews and Meta-Analysis course provides an overview of the steps involved in conducting a systematic review of multiple quantitative research studies. The course focuses on imparting knowledge and guidance on conducting a proper systematic review and meta-analysis to ensure the quality and reliability of the work so that it can be of use to readers and/or other researchers in the future.

The primary objectives of this course are to:

• Provide practical skills to conduct transparent, rigorous and reproducible review
• Emphasis on various steps to be followed during a systematic review.

The course is asynchronous, conceptual and appropriate for graduates, postdoctoral fellows, faculty, and senior researchers who have already completed introductory training in research methodology and statistics. The course consists of 8 Units and a total of 40 modules:

1. Introduction
2. Problem Formulation
3. Searching the Literature
4. Screening Potentially Eligible Studies
5. Data Extraction and Coding
6. Introduction to Effect Sizes
7. Introduction to Meta-Analysis
8. Course Wrap-up

Each module contains a few chapters. The course is mainly textual, primarily consisting of reading material, with a few pre-recorded videos integrated. After each chapter, there is a “Learn by Doing” section that requires us to apply what we’ve learned through either multiple-choice questions or by typing out our understanding of the topic. For each question, a predefined correct answer, often including an explanation, will be provided. This correct answer will then be compared to the participant’s response.

The Problem formulation unit emphasised how to pose answerable questions, use logic models and theories of change. It also covered setting PICOS (P- Population, I- Intervention, C- Comparison, O- Outcomes, S- Study Design) Eligibility (inclusion and exclusion) Criteria and planning the review in advance as a team. I found the “Using logic models and theories of change” module somewhat confusing, as the practical construction process for each model is fundamentally different. This distinction made it a bit difficult to apply the concepts in practice.

The course discussed in detail different databases, such as ERIC (Education Resources Information Centre) and other platforms (ProQuest). It explained that a single database can be found across various platforms and how to conduct searches (with video recordings to provide a clear understanding). Also, the PRISMA-S (Preferred Reporting Items for Systematic Reviews) was introduced, providing a checklist of reporting items specifically for the literature search component of the systematic review. Information regarding various Database operators and how they need to be modified across multiple platforms was also discussed.

The course also presses upon the importance of creating a screening guide and conducting a pilot test during screening of the relevant studies, as well as during the Data Extraction and coding process, to ensure that favourable results are obtained; if not, revision of the process can be carried out, thus maintaining the reliability of the study.

The units “Introduction to Effect Sizes” and “Introduction to Meta-Analysis” explain how effect sizes are used in meta-analysis and introduce the families of effect sizes. This also covers Homogeneity testing, Heterogeneity testing, and fixed-effects & random-effects meta-analysis. The topics can feel overwhelming initially, but once you understand the basics, they become pretty interesting.

The course does not explicitly cover a Qualitative systematic review; instead, it only briefly touches on it by mentioning Narrative synthesis. Nevertheless, the skills learned from the course’s focus on quantitative analysis – formulating precise research questions, judging evidence quality, understanding what interventions actually work – make it easier to evaluate programs, analyse data and draw reliable conclusions regarding market behaviour or the impact of various extension activities.

Overall, the course is comprehensive, packed with rich, high-quality information. It provides everything needed to conduct an effective systematic review and meta-analysis. It encourages you to formulate your own problem statements and then evaluate the quality of your questions, including the input, output, mediator, and moderator variables. This process strengthens your conceptual clarity, tests your understanding and helps build a strong foundation for research.

Abhinavya T is a Research Intern at CRISP and an MBA student in Agribusiness Management at ICAR-IVRI. She holds a degree in Agriculture from Sher-e-Kashmir University of Agricultural Sciences and Technology, Kashmir. Her research interests include climate change, ESG reporting, and interdisciplinary sustainability approaches. She can be reached at abhinavyat.abm.ivri@gmail.com.