Keto Diet App - Free Download Keto Diet App - Free Download
Take the guesswork out of following a low-carb diet, lose body fat & feel great!
KetoDiet app is FREE to download!

How To Understand Scientific Evidence
How Strong Is the Evidence?

5 stars, average of 14 ratings

How To Understand Scientific Evidence: How Strong Is the Evidence?ShareFollow us 261.1k

Quick Summary tl;dr

Although some studies rank lower on the scientific rigor scale than others, that does not mean that you just discount them.

Whenever a new study is published or discussed online, make sure you do your homework and find out exactly what type of study they are talking about. Once you know the type of study, you will be able to understand the limitations that come with that study and what that means for the results.

As a general rule of thumb, any study based on RCT or systematic review principles will have robust results. However, there are still limitations and biases that occur in these studies, so again interpret these with caution.

When reading a science-based article, again take time to look at what type of study they are meaning. If the author has based all of their argument on an animal or a one-person case study — you can confidently say the article is not scientifically robust enough. However, if they have a mixture of studies including observational, RCTs and even one systematic review, their writing is scientifically robust enough.

No matter what type of evidence is being used — no study should ever have a definitive answer and should always acknowledge their limitations.

Clinical research lays the groundwork for progress within the field of medicine and the prerequisite for ‘evidence based medicine.’ Evidence based medicine is what all researchers and clinicians should focus upon when talking about a treatment or disease.

Not All Research is Equal

By the time the media or an ‘expert’ gets a hold of a new study, they often forget about the scientific rigor. What we can see happening is a cherry picking of the data or the conclusion of a study only presented.

However, as with the world of science, not all research is equal. Even though a conclusion of a study may show promising results or validate a particular belief, without understanding the type of study, we do not actually know if these results have any particular meaning.

That's why you should always interpret statements such as "studies say..." with caution. Always ask yourself a few questions. What type of study is it? Who is the author? Are there any conflicts of interest? Are there any new and/or conflicting studies available?

This article aims to provide an overview of the different types of study designs that are available — looking at the strengths and limitations of each, how and when they should be used, including the strength of the study. In a follow up article we will focus on other aspects that affect the strength of scientific evidence.

Not all research is equal. That's why you should always interpret studies with caution. What type of study is it? Who is the author? Are there any conflicts of interest? Are there any new and/or conflicting studies available?

The #1 Keto Diet App
Keto Diet App for iOS and Android
Free Download
Trialed & tested for best results
Optimized for nutrition
Never feel hungry
Track all macros including net carbs
Scan products
Create your own meals
Track ketones, blood glucose & lipids
Stay hydrated with water tracking
Track your mood & energy levels
Calculate your ideal fat, protein & carb intake
Set any goal: weight loss, maintenance or weight gain
Your macros update based on your progress
Monitor your macros, water intake, mood & energy
Body weight, body fat and body measurements
Ketones, blood glucose & lipids
Expert articles to help you make informed choices
Guides & free diet plans
New daily content
Complete Keto Diet guide
Integrated shopping basket
Restaurants & guide to eating out
Free Download

1. Systematic Reviews

Strength of evidence: HIGH

These types of studies are the highest level of scientific evidence. They are classed as secondary research articles, including meta-analyses, and review papers. A systematic review is a critical assessment and evaluation of all research studies that address a particular treatment or disease area. The review should be based on a peer-reviewed protocol so that they can be replicated if necessary. A high quality systematic review will aim to:

  • Identify all published and unpublished evidence within a particular area
  • Select studies or reports that are of high quality evidence
  • Assess each study or report based on its quality

Once all of the studies are collected, the quality of the studies and the results undergoes rigorous examination. Interpretation of the findings within a systematic review should be balanced and impartial. Accurate presentation of all errors and bias contained within the studies must be displayed.

Many systematic reviews also include a quantitative pooling of data, which is termed a meta-analysis. A meta-analysis combines the results of many studies using statistical methods to calculate an overall or ‘absolute’ effect of a treatment or intervention. For example, a meta-analysis detailing all studies looking at low fat vs low carb diets — pooling the results to establish which has the better treatment effect.

Not all published systematic reviews are produced with proper care; therefore, the findings should still be interrogated before accepting the conclusion of a paper.

2. Randomized Controlled Trials (RCTs)

Strength of evidence: HIGH

These types of studies are the ‘gold standard’ of clinical research. The study randomly places participants in two or more groups to test the effect of a particular treatment vs placebo or a standard treatment.

Information from the control group allows the researchers to see whether the new treatment(s) are more or less effective than the current standard treatment. The randomization ensures that the results of the trials are not biased.

Blinding of subjects will also help to mitigate any potential bias. An RCT can be double blind, single blind or open. A double-blind study means neither patient nor study researcher know which treatment the patient has been assigned.

Although this is the gold standard of clinical research, they can carry certain limitations. Often the sample sizes of these studies are small (less than 20 people). This can skew the results of the study, making a treatment option appear better than it actually is. In order for the results of an RCT to be translated into the real world, a large sample size with a mix population group is required to show a meaningful clinically significant effect.

3. Cohort Studies

Strength of evidence: MEDIUM

These types of studies typically observe large groups of individuals over time. Their exposure to certain risk factors are recorded, helping to highlight possible causes of disease. They can be prospective (where they look forward and collect data) or retrospective (where they look back at already existing data).

Although similar to case-controlled studies, cohorts are considered as strong observational studies because they contain larger amounts of people and follow them for a longer period. However, these studies again cannot determine causation so interpretation must be with caution.

4. Case-Controlled Studies

Strength of evidence: LOW to MEDIUM

These types of studies are retrospective involving two groups of subjects, one with a particular condition (“cases”) and one without (“control”). The aim of the study is to look back and determine an attribute or exposure that caused the problem or made it worse/better, compared to the control group.

For example, researchers may take a group of people with lung disease and those without. An exposure such as smoking is then applied and the rate of smoking in each group is determined. Researchers use this to quantify the risk of smoking vs non-smoking based on the results of both of the groups.

Again, like case-reports, case-controlled studies can help to inform further studies and evidence. However, as they are observational, there can be no identification of confounding factors and causality.

5. Cross-Sectional Studies

Strength of evidence: LOW

This study is an example of an epidemiological study — designed to examine the relationship between a disease and a particular population and point in time. For example, a sample of males between the ages of 50-55 years have their prevalence of heart disease assessed.

Unlike cohort or case controlled studies, cross-sectional studies cannot determine any causality or effect. The design intention is to give prevalence insights into population groups. They often carry a lot of bias depending on how the data collection methods. The role of these types of studies is for descriptive statistics and hypothesis generating only.

6. Case Reports

Strength of evidence: LOW

This study usually features a written record on a particular subject. The report will focus on observations of one or two patients and the impact of a particular intervention on them.

This type of study can help with insights into a disease or condition, informing any other research that is required. However, it cannot determine causation, only correlation — meaning the conclusions are weak about why a certain intervention helped or did not help.

Since the reports only focus on 1-2 people, reproduction of the results have to be in a larger population before firm conclusions are drawn.

7. Animal and Cell Studies

Strength of evidence: LOW

These studies focus on the investigation of how a particular intervention behaves in animals or particular cells. This is basic primary research intended to be the starting point or rationale for carrying out work within humans.

Although often discounted, these studies can be useful at predicting effects also seen in humans. However, how an animal responds will be different to how a human will respond, therefore the results cannot be directly translated.

The intention should be to take the findings from these studies and use it to inform clinical studies in humans. Combining these types of studies with more robust studies will add more weight to a scientific argument or claim.

8. Expert Opinions and Expert Reviews

Strength of evidence: LOW

This type of information usually means one person’s opinion on a particular topic. This can be anything from an article through to an editorial piece.

Generally ranked as the lowest in terms of being scientifically robust. However, if an author of an opinion piece is an expert within a field, then their words may be of value.

The sign of a good expert review will contain multiple references within their writing, backed by credible science whilst pointing out the limitations of their own research/writing.

Not all research is equal. Here's a comprehensive guide to help you read, understand and trust studies. Learn all about the different grades of scientific evidence.

Strength of Scientific Evidence: Infographic to Pin for Later

How To Understand Scientific Evidence: How Strong Is the Evidence?Pin itFollow us 148.4k

Bestselling Books
KetoDiet Cookbook by Martina Slajerova
Created in collaboration with doctors and scientists
100 delicious recipes starting at the intersection of Mediterranean and Keto & Evolved with Science
Garnished with recipe Fun Facts to feed the mind!
My latest keto cookbook!
75 Delicious Treats for Your Low-Carb Diet
Chaffles are the epitome of simplicity and versatility
Meal-prepping, planning, and shopping strategies
100+ delicious low-carb recipes
Quick & easy, no stress, no complication
Easy keto guide
100+ delicious low-carb recipes
Whole foods, optimized for nutrition
My best keto recipes
100+ delicious low-carb recipes
Allergy-free options
Quick and easy meals
100+ delicious low-carb recipes
Allergy-free options
Sweet treats & savory snacks
100+ delicious keto recipes
Allergy-free options
100+ delicious low-carb recipes
Ideal for batch cooking and meal prep
Allergy-free options
150+ delicious low-carb recipes
Quick guide to keto
Keto food list
100+ delicious low-carb recipes
Tasty paleo treats and snacks
Ultra low-carb fat bombs
Have any of my books?
  1. Blog
  2. Expert Articles
  3. How To Understand Scientific Evidence: How Strong Is the Evidence?
  1. Blog
  2. Guides
  3. How To Understand Scientific Evidence: How Strong Is the Evidence?
  1. Blog
  2. Nutrition
  3. How To Understand Scientific Evidence: How Strong Is the Evidence?
  1. Blog
  2. Advice
  3. How To Understand Scientific Evidence: How Strong Is the Evidence?
  1. Blog
  2. Diet & Nutrition
  3. How To Understand Scientific Evidence: How Strong Is the Evidence?
  1. Blog
  2. Emily Maguire, BSc, MSc
  3. How To Understand Scientific Evidence: How Strong Is the Evidence?

Do you like this post? Share it with your friends! 

Emily Maguire, BSc, MSc
Creator of lowcarbgenesis.com

Emily Maguire

After completion of her BSc in Nutrition, Emily went onto study for an MSc in Obesity Science and Management. Author at lowcarbgenesis.com, she is keen to share the myths and truths surrounding the ever confusing and interesting topic of nutrition.

With over 7 years of experience working within the commercial weight loss sector, she has unprecedented insight knowledge into the use of alternative nutritional therapies, particularly that of the ketogenic diet.

Expert Article

This article was written by Emily Maguire, BSc, MSc who is a qualified expert. At KetoDiet we work with a team of health professionals to ensure accurate and up-to-date information. You can find out more on the About us page.

Evidence Based

Evidence-Based articles are based on medical research, and scientific evidence. Our expert authors focus on hard evidence alone and include relevant research references from trusted sources to support their articles. We always aim to deliver relevant, trustworthy and up-to-date information based on trusted evidence and proven research.

Let us know what you think, rate this post!

Leave a comment

Note: Any links to products or affiliate links will not be approved.
Please note that we do not offer personalised advice. For any diet related questions, please join our Facebook community.