Reading the evidence

The evidence hierarchy#

The key question about any study is whether researchers assigned the treatment (an experiment) or observed what people chose (an observational study). Only assignment by chance reliably separates cause from coincidence.

A good trial has a question and main outcome registered before it starts, a representative sample, a placebo control, random assignment, blinding of participants and researchers, and results others can replicate. Registration matters more than it sounds: among large US heart trials, 57% published before 2000 reported a benefit, versus 8% afterward, when outcomes had to be declared in advance.

Trials and observational studies#

Observational studies identified smoking as a cause of lung cancer, but they cannot by themselves prove a treatment works. Three problems explain most of their failures.

Confounding and the healthy-user effect#

A confounder is a third factor that drives both the exposure and the outcome. People who take supplements and stick with their pills also tend to exercise more, smoke less, and see doctors sooner, so almost any "good" habit looks protective. This is the healthy-user effect, and it is large. In the Coronary Drug Project, men who reliably took their placebo had a five-year death rate of 15.1%, versus 28.3% for those who did not. In a large study of statin users, faithful pill-takers had 25% fewer car crashes, an outcome no statin can affect.

Trials have overturned observational findings repeatedly. Observational studies suggested estrogen protected women's hearts; the Women's Health Initiative trial of 16,608 women found estrogen plus progestin raised coronary disease (hazard ratio 1.29), stroke (1.41), and breast cancer (1.26), though timing and age turned out to matter (see Hormones). Diets rich in beta-carotene were linked to less cancer; in a trial in Finnish male smokers, beta-carotene pills raised lung cancer by 18% and deaths by 8%.

Reverse causation and sick quitters#

Reverse causation is when the outcome causes the exposure: people stop drinking, lose weight, or slow down because they are already getting sick. In alcohol research this is the sick-quitter effect. Count former drinkers who quit for health reasons as "nondrinkers," and moderate drinkers look healthier by comparison. When a 2023 meta-analysis of 107 studies and 4.8 million people corrected for this and for study quality, low-volume drinkers no longer had a significantly lower death rate than lifetime abstainers. The full alcohol story is in Avoiding harm.

Mendelian randomization#

Mendelian randomization uses genes as a natural experiment. Because inherited variants are shuffled at conception, a variant that raises a risk factor acts like a lifelong, randomly assigned dose of it. People born with genes that raise HDL ("good") cholesterol had no fewer heart attacks, even though high HDL tracks with fewer heart attacks observationally; genes that raise LDL raised risk exactly as predicted. In 500,000 adults in China, genetically predicted drinking raised stroke risk steadily, showing that the apparent protection of moderate drinking was not causal. Its main pitfall is genes that affect several traits at once (BMJ guide).

Absolute and relative risk#

Relative risk says how much a treatment changes the odds. Absolute risk says how many people it actually helps. Headlines quote the first because it is bigger; decisions need the second. The number needed to treat (NNT) is how many people must take a treatment for one to benefit.

Harms have a mirror image, the number needed to harm. In the ASPREE trial of healthy older adults, daily aspirin did not significantly reduce cardiovascular disease but raised major bleeding from 6.2 to 8.6 events per 1,000 people per year: about one extra serious bleed for every 400 people treated for a year. Because the same relative benefit helps a high-risk person far more than a low-risk one, guidelines start with your baseline risk, as described in Heart and blood vessels.

Surrogate endpoints#

A surrogate endpoint is a measurement that stands in for what you care about: a lab value or scan in place of heart attacks, dementia, or death. Surrogates make trials faster but can mislead, because a drug can move the number through a pathway that does nothing for the patient, or harms them.

Some surrogates have earned trust: LDL cholesterol and blood pressure have tracked hard outcomes across many drug classes and decades of trials. Aging research is full of surrogates that have not, including epigenetic clocks, NAD+ levels, and senescence markers (see measuring biological age). A product that "lowers your biological age" has shown that it moves a number, nothing more.

Animal and mouse studies#

Animal studies are where ideas start, and they are often right about biology. They are a poor guide to whether something will help you. An umbrella review of 122 systematic reviews covering 367 therapies found that 50% of therapies tested in animals reached any human study, 40% a randomized trial, and 5% approval, a median of 10 years later. Mouse aging studies add their own problems:

• Lab mice are not like people. They are usually young, sedentary, overfed, and protected from most infections, and doses by body weight often exceed what people could take.
• Sex and strain matter. Most lifespan gains in the rigorous Interventions Testing Program appear in males only, a bias its authors acknowledge. Single-lab results in a single inbred strain are weaker still.
• Results do not always repeat. In 2026 the program reported that astaxanthin, meclizine, and mitoglitazone, which had extended male lifespan earlier, showed no benefit at other doses or starting ages.

Press releases often skip these caveats: in one analysis of university releases, 36% exaggerated how far animal results applied to humans, and 33% made causal claims from correlational data. News stories usually repeated the exaggeration.

Statistics that mislead#

Most bad claims are not fraud but the predictable result of small studies, many comparisons, and pressure to publish.

• P-hacking. A p-value below 0.05 is the usual bar for "significant." Test enough outcomes or subgroups and something will cross it by chance, and the literature shows this is widespread.
• Small samples. Small studies miss real effects and exaggerate the ones they find.
• Failed replication. When 100 psychology studies were repeated, 36% produced significant results again. In preclinical cancer biology, replicated effects were 85% smaller than the originals.
• Early findings fade. Of 45 highly cited clinical studies claiming a treatment worked, 16% were later contradicted and 16% had overstated the effect, mostly the non-randomized ones.
• Nutrition is especially noisy. Of 50 ingredients picked from a cookbook, 40 had been studied for cancer risk, and 72% of single studies claimed a higher or lower risk, mostly on weak statistics.

Look at the size of the effect and its confidence interval (the range of effects compatible with the data). A 5% reduction in an observational study is easily explained by confounding; a halving of risk in a large blinded trial is not. "20% lower risk, 95% CI 1% to 35%" is consistent with almost no benefit.

Funding, retractions, and hype#

Industry funding#

Company money does not make a study wrong; many of the most important trials in this guide were industry-funded. But it shifts the odds. A Cochrane review found industry-sponsored studies more often reported favorable results (risk ratio 1.27) and favorable conclusions (1.34) than independent ones. Funders also choose the questions: in the 1960s the sugar industry paid for research that steered blame toward fat. Company funding plus a surrogate endpoint plus no independent replication is a common pattern in longevity products.

Retractions and corrections#

Even prominent studies get corrected or withdrawn:

• PREDIMED (2013, republished 2018). The Mediterranean diet trial was retracted after some participants turned out not to have been properly randomized. The reanalysis of 7,447 people still found about 30% fewer cardiovascular events with olive oil or nuts.
• Apple cider vinegar (2024, retracted 2025). A weight-loss trial was retracted after statisticians could not reproduce its results and it proved not to have been registered in advance.
• KETO-CTA (2025, retracted 2026). A widely shared study of 100 people on ketogenic diets that concluded "plaque predicts plaque, ApoB does not" was retracted at the request of its authors and editors over methodological errors too large to correct.

Retraction Watch tracks retractions, and journal pages mark retracted papers.

Preprints, abstracts, and press releases#

Much health news arrives before peer review. Preprints (posted on servers such as medRxiv) and conference abstracts are early signals that no independent reviewer has checked. In 2024 an American Heart Association abstract reported a 91% higher risk of cardiovascular death among people who ate within an 8-hour window. It went viral, yet it was unreviewed, observational, and based on two days of self-reported eating in about 20,000 people. A later peer-reviewed analysis of the same national survey found a U-shaped pattern, lowest at 11 to 12 hours, which is still only an association (see Nutrition).

Experimenting on yourself#

An n-of-1 trial is a randomized trial in one person: you alternate a treatment with placebo or nothing, in random order and ideally blinded, and compare. It is a legitimate research design for the right questions, and it addresses a real problem: people respond differently to the same treatment, which is the case for one-person trials.

Self-experiments work for treatments that act and wear off quickly, measured on outcomes you can track, such as a sleep aid and sleep quality, or a medication and home blood pressure. They cannot show whether something prevents heart attacks or dementia or extends life. Wearables add noise as well as data: heart rate variability and sleep scores swing night to night, and a bad week followed by a normal one looks like improvement (regression to the mean). See Wearables and home monitoring.

Reading a health headline#

Six questions sort most claims in a few minutes.

A claim that fails most of these is a hypothesis, not a reason to change what you do.

How this guide grades evidence#

Interventions and headline claims in this guide carry a badge. A badge grades the evidence for a specific claim in a specific group, so the same drug can be strong for one use and weak for another.

When guidelines and prevention-minded practice differ, the guide leads with the optimal plan and labels the guideline minimum; the next section explains how it decides.

Guidelines and optimal practice#

Guidelines are the floor of good care, not the ceiling. This guide is written for a motivated adult with access to care, so it leads with what a well-informed, prevention-focused physician would advise and shows the guideline minimum second, labeled, so you know the standard of care and what insurance covers.

Guidelines run conservative for structural reasons:

• They are written for everyone. The USPSTF covers people without signs or symptoms seen in primary care, and its grades decide what insurers must cover for free. Its verdicts weigh false alarms and harms across millions of people, not what is best for one motivated person.
• They require a high bar. A "B" grade needs at least moderate certainty of a moderate net benefit. When evidence is thin, the Task Force issues an "I" statement, which means insufficient evidence, not a recommendation against.
• They lag. A commonly quoted estimate is that research takes 17 years to reach routine practice, though the measures behind it vary. The American Cancer Society lowered the age to start colorectal screening to 45 in 2018; the USPSTF followed in 2021.

The guide goes beyond a guideline under three rules:

1. The evidence must point the same way. Trials, strong observational data, and mechanism, plus prevention-focused expert practice. The badge stays honest, so an optimal recommendation can carry Moderate or Weak, with the reason it is still the best bet.
2. Cheap, safe, informative tests are repeated often enough to act on, typically yearly and every three to six months while adjusting treatment. A lipid panel "at least every five years" is the guideline minimum; this guide recommends one with ApoB every year, because LDL's damage builds with time.
3. Optimal is not maximal. When more testing or treatment does net harm, the guide says not to do it: daily aspirin for healthy people, whole-body CT, routine thyroid ultrasound, megadose vitamins, and decisions driven by unvalidated biological-age tests.

Action plan#