Track the facts

Which scientific studies are the most “solid”?

News - image_intro
Can we trust all the “scientific studies” making headlines? That depends on the “solidity” of the scientific evidence they provide, which depends in turn on the type of study conducted.

Studies can be ranked in ascending order according to the solidity of their evidence:

  • observational studies;
  • experimental studies;
  • systematic reviews.

Observational studies

In this type of study, the researchers observe the facts without seeking to intervene to modify their development.

In medicine, these studies describe the appearance of development of a symptom, without seeking to modify its progress with  medical treatment. This category primarily includes four types of studies:

 

  • Case studies report on observations of a single patient or a limited number of patients.

    These studies provide a detailed description of a new disease or shed light on a side effect of a drug. Their quality can be good, offering relevant research pathways. But they are essentially descriptive, based on a small number of observations.

    For example, this study reported how an 11-year-old boy developed Guillain-Barré syndrome three weeks after contracting COVID-19.

 

 

 

 


 

  • Case-control studies compare two groups of people with different health conditions. The studies compare the lifestyles, eating behaviours or medical histories of the two groups. The objective is to identify the risk factors associated with the health condition studied.

    This study compares patients hospitalized for COVID-19. Some have moderate symptoms and others exhibit respiratory distress. In this study, smoking appeared to be one of the risk factors associated with respiratory distress.

 

 

 

 


 

  • Cross-sectional studies are similar to a survey conducted within a population at a given time. In health, they use questionnaires or medical tests to determine the prevalence in the population of a risk factor, a behaviour or use of specific medication.

    They can reveal correlations between two factors. But they can’t necessarily conclude a cause-and-effect relationship between these factors.

    For example, in this study, the scientists surveyed the population about their mental health during the COVID-19 lockdown.

 

 

 


 

  • Longitudinal studies are also conducted within a population followed over a period of time. Instead of only taking one “snapshot” at a given time, they take several over a period. This allows researchers to track the progress of a health condition.

    This study followed a cohort of patients with COVID-19 who lost their sense of taste or smell. It observed that most of these patients recovered their taste and smell after one month.

 

 

 

 

 

Experimental studies

In these studies, the researchers don’t settle for passively observing a phenomenon. They introduce a measure and observe its consequences.

This may involve the effects of a drug or a diet on the development of disease. Or it could concern testing of a building material or a less polluting fuel. In medicine, experimental studies provide stronger evidence than observational studies. But you must check whether they were conducted on culture cells, animals or humans. Even if a cell or animal study is well conducted and solid,  it’s difficult to extrapolate the results to humans.

There are two types of experimental studies:


 

  • In randomized controlled studies, the participants are randomly assigned to one of the different groups in the study. For example, one group will receive a placebo, while another will receive an already validated treatment. A third group will receive a new treatment to find out its efficacy compared to the placebo and the usual treatment.

    Randomization constitutes groups with the same composition based on age, sex or other characteristics. If a difference is observed between the groups, it can be attributed to the treatment received. This is no longer just an association or a correlation, but a cause-and-effect relationship.

    One study of this kind was able to evaluate the efficacy of an anti-COVID-19 tritherapy combining the antivirals lopinavir, ritonavir and ribavirin.

 


 

  • Randomized double-blind trials are even stronger. That’s because the participants don’t know which treatment they received and the researcher doesn’t know who received which treatment.

    This study showed that the antiviral Remdesivir speeds up healing of patients hospitalized for COVID-19.

 

 

 

 

 

The strongest studies: systematic reviews (meta-analyses)

 

Even though it is solid, a randomized double-blind study is only one study among others.

Its conclusions could diverge from those of a study that is just as solid, but conducted under different conditions. Each study is only one piece of a puzzle. All the pieces must be put together  to see the full picture. That’s what systematic reviews do. They analyze all the studies produced on a subject. They examine their solidity and biases to extract the state of knowledge on the subject.

Unlike a jigsaw puzzle, which has a fixed number of pieces, research always generates new studies that provide a better resolution of the scientific knowledge picture. Even a systematic review must be updated, and you must make sure you have the most recent one.

Here’s a good example of a systematic review: this study identifies risk factors (age, sex, high blood pressure, diabetes…) associated with an increase in mortality from COVID-19.

 

  

Back to news list