Clinical trials play a vital role in medical research, helping to determine the safety and effectiveness of new treatments and interventions. But not all clinical trial evidence is created equal. Understanding the levels of evidence in clinical trials is crucial for healthcare professionals, researchers, and patients alike.
In this article, we will delve into the various levels of evidence in clinical trials. We will explore how these levels are determined and how they impact the interpretation and application of trial results. By gaining a better understanding of how the levels of evidence are decided, you will have a better understanding of how accurate the results of different studies are likely to be, and what needs to be done with that information. It will answer the question ‘How do we take this to the next level?’ or “Is this worth taking to the next level?’
Whether you are a healthcare professional looking to stay up-to-date with the latest research, a researcher wanting to design high-quality studies, or a patient seeking reliable information about treatment options, this article will provide useful insights into the hierarchy of evidence in clinical trials, cohort studies and case reports.
Everyone wants information. It is up to you to decide if this information is what it seems to be. It is up to you to decide how useful this information is. First understand levels of evidence and how each level is measured. Then apply the rules to each level to assess methodology, bias, etc
Join us as we unravel the complexities of evidence levels and discover why they are essential for evidence-based medicine.
Why are levels of evidence important in clinical trials?
When it comes to evaluating scientific research, not all studies are created equal. The quality and reliability of evidence can vary greatly, which is why a hierarchy of evidence levels has been developed. This hierarchy allows healthcare professionals, researchers, and policymakers to assess the strength and validity of different studies.
The levels of evidence provide a framework for understanding the design, methodology, and potential biases of a study. They help determine the level of confidence we can have in the results and the degree to which the findings can be applied to clinical practice. By understanding the levels of evidence, healthcare professionals can make more informed decisions about treatment options and interventions.
It is also important to note that the levels of evidence are not static; they can change over time as new studies are conducted and new evidence emerges. As research methods and technology continue to advance new information comes to light. This helps us to understand why what we used to think is no longer correct. We need to stay updated on the latest evidence levels to ensure the best possible patient care.
The hierarchy of evidence in clinical trials
In order to establish a hierarchy of evidence, different types of studies are assigned levels based on their design and methodology. The higher the level, the more rigorous and reliable the study is considered to be. Let’s explore the different levels of evidence in clinical trials:
Level 1: Systematic reviews and meta-analyses
Systematic reviews and meta-analyses are considered the highest level of evidence in clinical research. These studies involve a comprehensive review and analysis of all relevant research on a specific topic. By pooling together data from multiple studies, researchers can gain a more accurate and comprehensive understanding of the effectiveness of a treatment or intervention.
Systematic reviews and meta-analyses are conducted using a predefined protocol and rigorous methodology, making them highly reliable sources of evidence. They provide a quantitative summary of the available evidence, allowing healthcare professionals to make evidence-based decisions.
However, it is important to note that not all systematic reviews and meta-analyses are created equal. The quality of these studies can vary based on factors such as study design, inclusion criteria, and statistical methods. Before you can critically evaluate the methods and findings of these studies it will help to learn how reviews can be written. A well written review takes a team of experienced reviewers working together. With a careful design and a variety of experts to execute the design the results can be reliable. But even with all this effort you may not agree with the choices that were made at each level of the work. The results will still be valuable today. The conclusions will change as new studies are published. We need the best today to apply them to clinical practice now.
Level 2: Randomized controlled trials (RCTs)
Randomized controlled trials (RCTs) are widely considered the gold standard in clinical research. These studies involve randomly assigning participants to different treatment groups and comparing the outcomes. By using randomization, researchers can minimize bias and ensure that any observed differences between groups are due to the intervention being studied.
RCTs are designed to provide high-quality evidence about the efficacy and safety of interventions. They often involve large sample sizes and rigorous study designs, making them highly reliable sources of evidence. The results of RCTs are used to inform clinical guidelines and treatment recommendations.
RCTs are expensive to run. There is no such thing as a free trial. Doctors need to be paid, nurses need to be paid, technicians need to be paid, etc. RCTs consume a lot of time and resources. Your contribution is very important. Researchers put a lot of their life and reputation into running a clinical trial. Just as they treat you with respect you should only join a trial if you have the time and are willing to follow the protocols regards of your result. We don’t know if you will get the treatment or if you will be part of the placebo group. We want to know how much or how little change you get. Each trial can only reach its full strength if you follow the protocols as carefully as the researchers do. Losing data by not following the protocols makes a trial weaker.
Cost of Randomized Clinical Trials
In order to remove one important bias it will give more reliable results if participants (Patients) do not have to contribute to the cost of running a trial. It is felt that when you pay for a treatment you have an incentive for it to succeed. You want it to succeed because you paid hard earned cash for it. This is a bias. If patients don’t pay directly to participate they think it is “free”. It is not. You buy food and electricity to live. You pay tax (VAT) on everything you buy. The government collects this money and some of it is given to researchers to run important trials. You paid your tax. This is your tax money that the government is spending on you. Someone always has to pay.
However, it is important to consider the limitations of RCTs. In some cases, it may not be ethically or practically feasible to conduct an RCT. Surgery is particularly difficult. “Sham operations” have been done for knee arthroscopy. You go to hospital. You have an anaesthetic. They stick an arthroscope into your knee and look around. But they don’t touch anything. You go home having lost time off work and your body recovers from an anaesthetic and a hole in your knee. You improve. Why? This is the placebo effect of surgery. Prof Ian Harris has written a book ‘Surgery: the ultimate placebo’.
Additionally, RCTs may not always reflect real-world clinical practice, as they often involve strict inclusion and exclusion criteria. What if all the patients are in a nursing home. What if all the patients live in the Sahara. How does that help me? Despite these limitations, RCTs remain an essential tool in evidence-based medicine.
Level 3: Cohort studies
Cohort studies are observational studies that follow a group of individuals over a period of time. These studies can be prospective, where data is collected moving forward, or retrospective, where data is collected from past records. Cohort studies aim to identify associations between exposures (such as treatments or risk factors) and outcomes (such as disease incidence or mortality).
Cohort studies provide valuable evidence about the natural history of diseases and the long-term effects of interventions. They can help identify risk factors, determine the incidence of diseases, and assess the effectiveness of treatments in real-world settings. However, cohort studies are susceptible to biases, such as selection bias and confounding variables, which can impact the validity of the results. Public hospitals and universities can fund these with monies they have received from the government or from charitable institutions. These cohort studies will shed light on operations or other procedures that may be part of traditional treatments and need review.
Cost of Cohort Studies
Only 20% of research grant applications for government funding are successful. 80% of research cannot be funded by the government. Private Foundations provide some funding. When funding is not available these studies are often funded by doctors in the private sector who want to collect information about the treatments they are providing. Patients will contribute to the cost of the trial. This will introduce a bias that will need to be considered when evaluating the results. The results of these patient pays cohort studies will guide the design of a full Double Blinded Randomised controlled trial. This is the next level up.
An example of a observational cohort study that was confounded is the question ‘Is meat good for you?’ The initial results were not clear cut and leaned towards NO. We then realised that the correct question was in fact two questions. We then asked ‘How much red meat do you eat?’ and “How much processed meat o you eat?’ The answer was clear.
Unprocessed red meat = Good, Processed red meat = Bad.
Level 4: Case-control studies
Case-control studies are observational studies that start with the outcome of interest (cases) and compare them to a control group without the outcome. These studies are often used to investigate rare diseases or outcomes with long latency periods.
Case-control studies are useful for generating hypotheses and exploring potential associations between exposures and outcomes. However, they are prone to selection bias and recall bias, which can affect the accuracy of the results. It is important to interpret the findings of case-control studies cautiously and consider them in the context of other evidence. Their major benefit is that they tell us if we should invest time and money in a larger study and if we got the design right.
Level 5: Case series and case reports
Case series and case reports are considered the lowest level of evidence in clinical research. These studies involve the description of individual cases or a series of cases with similar characteristics. While case reports can provide valuable insights into rare diseases or adverse events, they do not provide strong evidence of causality or generalizability.
Case reports and case series are often used to generate hypotheses or highlight unusual clinical presentations. They can serve as a starting point for further research but should not be used as the sole basis for clinical decision-making. So yes they are very good because they help us to understand ‘what are the questions that we need to answer.’ This is the stepping stone to the next level.
Criticisms and limitations of levels of evidence in clinical trials
While the levels of evidence provide a useful framework for evaluating scientific research, they are not without criticism and limitations. One criticism is that the hierarchy assumes that study design alone determines the reliability and validity of evidence, overlooking other important factors such as sample size and statistical power.
Additionally, the levels of evidence by themselves do not take into account the quality of individual studies within each level. For example, not all RCTs are conducted with the same rigor, and not all systematic reviews are free from bias. It is important to critically evaluate the methods and findings of each study, regardless of its assigned level of evidence.
This is an example of a review by Ossendorff et al that has carefully followed the recommended guidelines.
The Preferred Reporting Items for Systematic reviews and Meta-Analysis statement (PRISMA) was followed as a guideline for the study. The review was registered on the PROSPERO database (Registration number: CRD42022339795). The search string was constructed with the aid of an experienced librarian. Two reviewers independently applied the predefined eligibility and inclusion criteria to the articles. The references of all the fully assessed articles and relevant review papers were also hand-searched to identify additional articles. All the studies that met the final inclusion criteria were individually assessed for quality by two independent reviewers, first by assigning a level of evidence according to the recommendations by Marx et al. The quality of all studies was evaluated using the modified Coleman Methodology Score (mCMS). The Cochrane risk-of-bias tool for randomized trials (Version 2) was used to assess the risk of bias of Level I and II randomized controlled trials.
As you can see this is a lot of work but we can be confident that the correct methodology has been applied.
Cochrane
When evaluating RCTs the Cochrane Risk of Bias Tool (RoB 2) has been updated. It remains the tool for assessing RCT’s both during design and after trial publishing.
Modified Coleman Method (mCMS)
Coleman and his group, after comparing studies on the results of tendon repair, said ‘we suggest practical guidelines for improving study design in this area of clinical research’. These guidelines have been refined and are a useful tool to help us evaluate the study methodology.
How to use levels of evidence in clinical practice
The levels of evidence may not always align with the needs of individual patients or clinical scenarios. Evidence-based medicine takes the best research evidence we have and decides whether it fits with clinical expertise (experience) and patient preferences to reach the best possible care.
Understanding the levels of evidence is essential as part of evidence-based medicine. Healthcare professionals can use the hierarchy of evidence to critically appraise research studies and make informed decisions about patient care. Here are some tips for using levels of evidence in clinical practice:
1. Start with systematic reviews and meta-analyses: These studies provide the highest level of evidence and offer a comprehensive summary of the available research. They can help identify the most effective treatments and interventions.
2. Consider the quality of individual studies: Even within each level of evidence, the quality of studies can vary. It is important to critically evaluate the design, methodology, and potential biases of each study before applying the findings to clinical practice.
3. Consider the applicability to your patient population: The results of a study may not always be applicable to your specific patient population. Where was the trial conducted (Inpatient, Outpatient, etc) and is this patient population representitive of your patient? Consider factors such as age, gender, comorbidities, and treatment preferences when interpreting the evidence.
4. Stay updated with the latest research: The levels of evidence are not static and can change over time as new studies are conducted. Doctors work hard to stay updated with the latest research in their field. This helps to ensure the best possible patient care.
5. Combine research results with clinical expertise and patient preferences. Doctors consider the individual needs and values of their patients when making treatment decisions.
Conclusion
Levels of evidence play a crucial role in evaluating the quality and reliability of clinical trial evidence. By understanding the hierarchy of evidence, healthcare professionals, researchers, and patients can make more informed decisions about treatment options and interventions. Systematic reviews and meta-analyses provide the highest level of evidence, followed by randomized controlled trials, cohort studies, case-control studies, and case series/case reports. While the levels of evidence have their limitations, they provide a valuable framework for evidence-based medicine. By critically appraising the design and findings of research studies and considering the applicability to individual patients, healthcare professionals can provide the best possible care based on the available evidence.
As the field of clinical research continues to evolve, it is important to stay updated with the latest evidence levels and research findings. By doing so, healthcare professionals can ensure that they are providing the most accurate and effective care to their patients. So, whether you are a healthcare professional, a researcher, or a patient, understanding the levels of evidence in clinical trials is essential for evidence-based medicine.