Evidence-Based Medicine and its discontents (with Gordon Guyatt)
Gordon Guyatt, a McMaster University Distinguished Professor, discusses Evidence-Based Medicine (EBM), emphasizing its evolution, the importance of patient preferences and values, and the need for better trial methods and study quality. He highlights how EBM transformed medical decision-making from reliance on clinical experience to systematic evidence.
Deep Dive Analysis
14 Topic Outline
Medical Decision-Making Before Evidence-Based Medicine (EBM)
Limitations of Physiologic Reasoning and Observational Studies
Initial Resistance and Rapid Adoption of EBM
Evolution of EBM and Modern Information Resources
Addressing Conflicts of Interest in Medical Guidelines
The GRADE Approach to Evidence Quality and Recommendations
Incorporating Patient Values and Preferences in Medical Decisions
Understanding Study Quality: Randomized vs. Observational Trials
The Problem with Low-Quality Studies and Overstated Claims
Absolute vs. Relative Risk in Medical Communication
Evidence for Mental Health and Alzheimer's Treatments
The Role of Surrogate Outcomes in Research
Understanding the Placebo Effect
Future Directions and Challenges for Evidence-Based Medicine
5 Key Concepts
Evidence-Based Medicine (EBM)
EBM is a movement that shifted medical decision-making from relying on clinical experience and physiologic reasoning to systematically reviewing and applying the best available research evidence. It emphasizes that medical decisions should be grounded in high-quality studies, primarily randomized controlled trials, and integrate patient values and preferences.
Confounding
Confounding is a bias in non-randomized (observational) studies where people exposed to an intervention are inherently different from those not exposed. This difference, rather than the intervention itself, can explain observed outcomes, making it difficult to determine true cause-and-effect relationships.
GRADE Approach
GRADE (Grading of Recommendations Assessment, Development and Evaluation) is a widely adopted system for assessing the quality of evidence and developing clinical practice guidelines. It provides a structured method for determining whether evidence is high, moderate, low, or very low quality, and for translating that evidence into recommendations that consider patient values.
Shared Decision-Making
Shared decision-making is a process where clinicians and patients collaborate to make healthcare decisions, ensuring that choices align with the patient's values and preferences. It acknowledges that while clinicians are experts in evidence, patients are experts in their own lives and priorities, especially when trade-offs are involved.
Surrogate Outcomes
Surrogate outcomes (or substitute outcomes) are intermediate variables used in studies as proxies for patient-important outcomes like mortality or long-term morbidity. While appealing for efficiency, changes in surrogate markers (e.g., blood glucose levels, amyloid plaque reduction) often do not reliably translate into improvements in the actual health outcomes patients care about.
12 Questions Answered
Before EBM, medical decisions were primarily based on a clinician's personal experience, the experience of their mentors, published clinical observations, and physiologic reasoning, rather than systematic review of the clinical literature.
Biological arguments are often weak because the body's systems are far more complicated and counterintuitive than initially understood, leading to many instances where predicted biological mechanisms do not translate to actual patient benefits or even cause harm.
While things can go wrong and complications can occur in hospitals, the net benefit of being in a hospital when truly sick outweighs the risks. If not sick, one should avoid hospitals, and if feeling better, one should leave.
Despite initial hostility from some practitioners, EBM gained rapid uptake among medical leadership, particularly in North America, with major journals and residency programs adopting its principles and curriculum, leading to widespread nominal support.
Clinicians now have access to electronic resources like 'UpToDate' and trustworthy guidelines from leading organizations that integrate new information quickly, often based on systematic reviews summarizing the best available evidence.
Pharmaceutical marketing aims to make drugs look better than they are, often underplaying adverse effects or suppressing evidence, which can lead physicians to receive a biased impression compared to unconflicted, evidence-based sources.
Randomized trials minimize confounding bias by ensuring that groups receiving an intervention and those not receiving it are comparable, meaning any observed differences in outcomes are more likely due to the intervention itself rather than other differences between the groups.
Observational studies can yield high-certainty evidence when there are huge, obvious treatment effects, such as with dialysis for terminal renal failure or insulin for diabetic ketoacidosis, where the benefit is so clear that randomized trials are not necessary.
Relative risks are fundamentally unhelpful because they do not convey the actual magnitude of benefit or harm without knowing the baseline risk. A 50% relative risk reduction can be trivial if the baseline risk is very low, but highly significant if the baseline risk is high.
The DSM is still developed by committees and involves significant value judgments in defining what constitutes a disease, as exemplified by the historical classification of homosexuality as a disease, indicating that it's not purely evidence-based.
Evidence suggests that while amyloid plaque reduction is a surrogate outcome for Alzheimer's, most anti-Alzheimer's drugs targeting amyloid have shown little to no impressive effect on patient-important outcomes, despite high-quality testing.
While the existence of placebo effects has long been known, understanding has refined to recognize variability across conditions (e.g., injections and surgery often have larger effects than pills) and the need to distinguish true placebo effects from natural disease progression or regression to the mean in studies.
18 Actionable Insights
1. Practice Shared Decision-Making
Engage in shared decision-making with your doctor, understanding that you are the expert in your values and preferences, while the clinician is the expert in the evidence.
2. Understand Evidence Quality
Recognize that not all evidence is equal; randomized trials generally start as high-quality evidence, while non-randomized studies start as low-quality, but both can be downgraded or upgraded based on specific factors.
3. Prefer Randomized Trials
When possible, prioritize evidence from well-conducted randomized controlled trials over observational studies, as randomization helps avoid confounding bias where differences in people, not the intervention, cause observed effects.
4. Focus on Absolute Risk
When evaluating treatment benefits or harms, always consider the absolute risk reduction or increase, as relative risks can sound impressive but be trivial if the baseline risk is very low.
5. Beware Surrogate Outcomes
Be skeptical of treatments that only show improvement in surrogate markers (like HDL levels or blood glucose) without direct evidence of improving patient-important outcomes like mortality or major disease events.
6. Question Conflicted Sources
Be wary of medical information or recommendations from sources with conflicts of interest, particularly financial ties to the pharmaceutical industry or strong intellectual attachments to specific work.
7. Demand Honest Uncertainty
Expect authorities and clinicians to acknowledge uncertainty when it exists, as pretending to know when they don’t erodes trust and can lead to poor decisions.
8. Distrust Biological Arguments Alone
Do not rely solely on biological arguments or physiological reasoning for treatment efficacy, as history shows these can be misleading and even harmful; always seek evidence from randomized trials.
9. Assess Study Limitations
When evaluating studies, consider factors that can lower evidence quality, such as risk of bias, inconsistent results, small sample sizes, indirect applicability to your situation, and potential publication bias.
10. Prioritize Patient-Important Outcomes
When evaluating treatments, focus on those that demonstrably improve long-term patient-important outcomes (like cardiovascular and renal risk) rather than just surrogate markers.
11. Evaluate Screening Test Risk
If you are at low risk for a condition, be cautious about screening tests, as they can lead to false positives and unnecessary interventions, and may be oversold to low-risk populations.
12. Minimize Hospital Stays
Avoid going to the hospital if not sick, and if in the hospital and feeling better, get out as soon as medically advised, as there are inherent risks and complications.
13. Skepticism on Moderate Alcohol
Approach claims about the health effects of moderate alcohol consumption with skepticism, as the evidence is often low-quality observational data and may be overplayed.
14. Address Severe Obesity
Recognize that severe obesity is unequivocally a major health problem with clear negative health outcomes, while the health implications of modest obesity are less certain.
15. Recognize Placebo Power
Be aware that placebo effects are real and can be substantial, especially in conditions with chronic symptoms that don’t spontaneously improve, and can significantly influence perceived treatment efficacy.
16. Question Disease Definitions
Be aware that definitions of diseases, especially in mental health (e.g., DSM), involve value judgments and are not purely evidence-based, and can change over time.
17. Demand High-Quality Research
Advocate for resources to be concentrated on a smaller number of large, well-done, high-quality studies rather than numerous low-quality studies that often yield misleading or unhelpful results.
18. Seek Evidence-Based Resources
When seeking medical information, consult up-to-date, evidence-based electronic resources and guidelines from reputable organizations that adhere to EBM principles.
6 Key Quotes
Physiologic reasoning makes sense, but things turn out to be more complicated than we knew.
Gordon Guyatt
Defensiveness is one word. Hostility is another.
Gordon Guyatt
Evidence never tells you what to do. It's evidence in the context of people's values and preferences.
Gordon Guyatt
We are the experts in the evidence. The patient is the experts in their values and preferences.
Gordon Guyatt
Relative risks are fundamentally unhelpful in making decisions.
Gordon Guyatt
When we are uncertain, we need to acknowledge that we are uncertain. And you alluded to the fact that people might not like that. People might be uncomfortable with uncertainty, but man, the strategy of dealing with that discomfort by pretending you don't when you, pretending you know, when you don't know, that is a mistake.
Gordon Guyatt