How To Understand Scientific Evidence
How Strong Is the Evidence?

5 stars, average of 14 ratings

By Emily Maguire, BSc, MSc, updated April 10 2022
This post may contain affiliate links, see our disclaimer.
Expert Article Evidence Based

Share Follow us 261.1k

Quick Summary tl;dr

Although some studies rank lower on the scientific rigor scale than others, that does not mean that you just discount them.

Whenever a new study is published or discussed online, make sure you do your homework and find out exactly what type of study they are talking about. Once you know the type of study, you will be able to understand the limitations that come with that study and what that means for the results.

As a general rule of thumb, any study based on RCT or systematic review principles will have robust results. However, there are still limitations and biases that occur in these studies, so again interpret these with caution.

When reading a science-based article, again take time to look at what type of study they are meaning. If the author has based all of their argument on an animal or a one-person case study — you can confidently say the article is not scientifically robust enough. However, if they have a mixture of studies including observational, RCTs and even one systematic review, their writing is scientifically robust enough.

No matter what type of evidence is being used — no study should ever have a definitive answer and should always acknowledge their limitations.

Table of Contents

Not All Research is Equal
1. Systematic Reviews
2. Randomized Controlled Trials (RCTs)
3. Cohort Studies
4. Case-Controlled Studies
5. Cross-Sectional Studies
6. Case Reports
7. Animal and Cell Studies
8. Expert Opinions and Expert Reviews
Strength of Scientific Evidence: Infographic to Pin for Later

Clinical research lays the groundwork for progress within the field of medicine and the prerequisite for ‘evidence based medicine.’ Evidence based medicine is what all researchers and clinicians should focus upon when talking about a treatment or disease.

Not All Research is Equal

By the time the media or an ‘expert’ gets a hold of a new study, they often forget about the scientific rigor. What we can see happening is a cherry picking of the data or the conclusion of a study only presented.

However, as with the world of science, not all research is equal. Even though a conclusion of a study may show promising results or validate a particular belief, without understanding the type of study, we do not actually know if these results have any particular meaning.

That's why you should always interpret statements such as "studies say..." with caution. Always ask yourself a few questions. What type of study is it? Who is the author? Are there any conflicts of interest? Are there any new and/or conflicting studies available?

This article aims to provide an overview of the different types of study designs that are available — looking at the strengths and limitations of each, how and when they should be used, including the strength of the study. In a follow up article we will focus on other aspects that affect the strength of scientific evidence.

Not all research is equal. That's why you should always interpret studies with caution. What type of study is it? Who is the author? Are there any conflicts of interest? Are there any new and/or conflicting studies available?

The #1 Keto Diet App

Intelligent Fasting Easily record or schedule fasting periods with our intelligent fasting tool. Auto Fill Automatically complete your day with meals and snacks that perfectly match your macro targets. Unlimited Meal Plans Create bespoke meal plans tailored to your preferences while hitting your macros. Trialed and Tested Recipes Enjoy delicious, foolproof recipes developed exclusively for KetoDetApp. AI Assistant Get instant answers to all your questions about the keto diet, the app, and more. Net vs Total Carbs Switch seamlessly between tracking net and total carbs at any time. Barcode Scanning Quickly scan product barcodes to track your food intake with ease. High Protein Keto Tailor your protein intake to support your fitness goals, from shedding stubborn pounds to building strong, lean muscle. iPhone & iPad Get Started for Free

1. Systematic Reviews

Strength of evidence: HIGH

These types of studies are the highest level of scientific evidence. They are classed as secondary research articles, including meta-analyses, and review papers. A systematic review is a critical assessment and evaluation of all research studies that address a particular treatment or disease area. The review should be based on a peer-reviewed protocol so that they can be replicated if necessary. A high quality systematic review will aim to:

Identify all published and unpublished evidence within a particular area
Select studies or reports that are of high quality evidence
Assess each study or report based on its quality

Once all of the studies are collected, the quality of the studies and the results undergoes rigorous examination. Interpretation of the findings within a systematic review should be balanced and impartial. Accurate presentation of all errors and bias contained within the studies must be displayed.

Many systematic reviews also include a quantitative pooling of data, which is termed a meta-analysis. A meta-analysis combines the results of many studies using statistical methods to calculate an overall or ‘absolute’ effect of a treatment or intervention. For example, a meta-analysis detailing all studies looking at low fat vs low carb diets — pooling the results to establish which has the better treatment effect.

Not all published systematic reviews are produced with proper care; therefore, the findings should still be interrogated before accepting the conclusion of a paper.

2. Randomized Controlled Trials (RCTs)

Strength of evidence: HIGH

These types of studies are the ‘gold standard’ of clinical research. The study randomly places participants in two or more groups to test the effect of a particular treatment vs placebo or a standard treatment.

Information from the control group allows the researchers to see whether the new treatment(s) are more or less effective than the current standard treatment. The randomization ensures that the results of the trials are not biased.

Blinding of subjects will also help to mitigate any potential bias. An RCT can be double blind, single blind or open. A double-blind study means neither patient nor study researcher know which treatment the patient has been assigned.

Although this is the gold standard of clinical research, they can carry certain limitations. Often the sample sizes of these studies are small (less than 20 people). This can skew the results of the study, making a treatment option appear better than it actually is. In order for the results of an RCT to be translated into the real world, a large sample size with a mix population group is required to show a meaningful clinically significant effect.

3. Cohort Studies

Strength of evidence: MEDIUM

These types of studies typically observe large groups of individuals over time. Their exposure to certain risk factors are recorded, helping to highlight possible causes of disease. They can be prospective (where they look forward and collect data) or retrospective (where they look back at already existing data).

Although similar to case-controlled studies, cohorts are considered as strong observational studies because they contain larger amounts of people and follow them for a longer period. However, these studies again cannot determine causation so interpretation must be with caution.

4. Case-Controlled Studies

Strength of evidence: LOW to MEDIUM

These types of studies are retrospective involving two groups of subjects, one with a particular condition (“cases”) and one without (“control”). The aim of the study is to look back and determine an attribute or exposure that caused the problem or made it worse/better, compared to the control group.

For example, researchers may take a group of people with lung disease and those without. An exposure such as smoking is then applied and the rate of smoking in each group is determined. Researchers use this to quantify the risk of smoking vs non-smoking based on the results of both of the groups.

Again, like case-reports, case-controlled studies can help to inform further studies and evidence. However, as they are observational, there can be no identification of confounding factors and causality.

5. Cross-Sectional Studies

Strength of evidence: LOW

This study is an example of an epidemiological study — designed to examine the relationship between a disease and a particular population and point in time. For example, a sample of males between the ages of 50-55 years have their prevalence of heart disease assessed.

Unlike cohort or case controlled studies, cross-sectional studies cannot determine any causality or effect. The design intention is to give prevalence insights into population groups. They often carry a lot of bias depending on how the data collection methods. The role of these types of studies is for descriptive statistics and hypothesis generating only.

6. Case Reports

Strength of evidence: LOW

This study usually features a written record on a particular subject. The report will focus on observations of one or two patients and the impact of a particular intervention on them.

This type of study can help with insights into a disease or condition, informing any other research that is required. However, it cannot determine causation, only correlation — meaning the conclusions are weak about why a certain intervention helped or did not help.

Since the reports only focus on 1-2 people, reproduction of the results have to be in a larger population before firm conclusions are drawn.

7. Animal and Cell Studies

Strength of evidence: LOW

These studies focus on the investigation of how a particular intervention behaves in animals or particular cells. This is basic primary research intended to be the starting point or rationale for carrying out work within humans.

Although often discounted, these studies can be useful at predicting effects also seen in humans. However, how an animal responds will be different to how a human will respond, therefore the results cannot be directly translated.

The intention should be to take the findings from these studies and use it to inform clinical studies in humans. Combining these types of studies with more robust studies will add more weight to a scientific argument or claim.

8. Expert Opinions and Expert Reviews

Strength of evidence: LOW

This type of information usually means one person’s opinion on a particular topic. This can be anything from an article through to an editorial piece.

Generally ranked as the lowest in terms of being scientifically robust. However, if an author of an opinion piece is an expert within a field, then their words may be of value.

The sign of a good expert review will contain multiple references within their writing, backed by credible science whilst pointing out the limitations of their own research/writing.

Not all research is equal. Here's a comprehensive guide to help you read, understand and trust studies. Learn all about the different grades of scientific evidence.