Two Reasons Good News From Medical Trials is Often Overturned Later

Medical trials that claim large effects from treatment are often followed by much less impressive results the more they are replicated or tried out in the real world. A new study reports two factors behind this, one of which is obvious, the other less so.

Medical trials that claim large effects from treatment are often followed by much less impressive results the more they are replicated or tried out in the real world. A new study reports two factors behind this, one of which is obvious, the other less so.

The first factor is that trials showing big effects are often performed with few trial participants. The small sizes make it a lot more likely that the results are based on chance. Follow-up studies tend to show a regression to the mean as the effect of chance is reduced. That’s the obvious one.

The other factor is that many of the large-effect studies are based on intermediate endpoints, such as lowering cholesterol, rather than more meaningful outcomes such as extension of life or prevention of heart attack.

In general patients and doctors should be skeptical when they hear about a trial showing great results. Look to see if it’s a big or small trial and what endpoint is being measured.