Image credit: Robyn Twomey/Redux |
Disturbing, revealing and even somewhat unsurprising, The Atlantic has published a piece on how medical researchers are consistently coming up with conclusions that are misleading, exaggerated and even flat-out long. In the article, titled Lies, Damned Lies, and Medical Science, the work of meta-researcher Dr. John Ioannidis is explored; he has spent his career challenging his peers by exposing their questionable science. And just as importantly, he asks the question: Why are so many doctors still drawing upon misinformation in their everyday practice?
One of the researchers, a biostatistician named Georgia Salanti, fired up a laptop and projector and started to take the group through a study she and a few colleagues were completing that asked this question: were drug companies manipulating published research to make their drugs look good? Salanti ticked off data that seemed to indicate they were, but the other team members almost immediately started interrupting. One noted that Salanti’s study didn’t address the fact that drug-company research wasn’t measuring critically important “hard” outcomes for patients, such as survival versus death, and instead tended to measure “softer” outcomes, such as self-reported symptoms (“my chest doesn’t hurt as much today”). Another pointed out that Salanti’s study ignored the fact that when drug-company data seemed to show patients’ health improving, the data often failed to show that the drug was responsible, or that the improvement was more than marginal.
Salanti remained poised, as if the grilling were par for the course, and gamely acknowledged that the suggestions were all good—but a single study can’t prove everything, she said. Just as I was getting the sense that the data in drug studies were endlessly malleable, Ioannidis, who had mostly been listening, delivered what felt like a coup de grâce: wasn’t it possible, he asked, that drug companies were carefully selecting the topics of their studies—for example, comparing their new drugs against those already known to be inferior to others on the market—so that they were ahead of the game even before the data juggling began? “Maybe sometimes it’s the questions that are biased, not the answers,” he said, flashing a friendly smile. Everyone nodded. Though the results of drug studies often make newspaper headlines, you have to wonder whether they prove anything at all. Indeed, given the breadth of the potential problems raised at the meeting, can any medical-research studies be trusted?
That question has been central to Ioannidis’s career. He’s what’s known as a meta-researcher, and he’s become one of the world’s foremost experts on the credibility of medical research. He and his team have shown, again and again, and in many different ways, that much of what biomedical researchers conclude in published studies—conclusions that doctors keep in mind when they prescribe antibiotics or blood-pressure medication, or when they advise us to consume more fiber or less meat, or when they recommend surgery for heart disease or back pain—is misleading, exaggerated, and often flat-out wrong. He charges that as much as 90 percent of the published medical information that doctors rely on is flawed. His work has been widely accepted by the medical community; it has been published in the field’s top journals, where it is heavily cited; and he is a big draw at conferences. Given this exposure, and the fact that his work broadly targets everyone else’s work in medicine, as well as everything that physicians do and all the health advice we get, Ioannidis may be one of the most influential scientists alive. Yet for all his influence, he worries that the field of medical research is so pervasively flawed, and so riddled with conflicts of interest, that it might be chronically resistant to change—or even to publicly admitting that there’s a problem.
- Neurodiversity vs. Cognitive Liberty - November 8th, 2009 [November 8th, 2009]
- Link dump: 2009.10.13 - November 8th, 2009 [November 8th, 2009]
- Limits to the biolibertarian impulse - November 8th, 2009 [November 8th, 2009]
- Link dump: 2009.10.15 - November 8th, 2009 [November 8th, 2009]
- Neurodiversity vs. Cognitive Liberty, Round II - November 8th, 2009 [November 8th, 2009]
- Link dump: 2009.10.17 - November 8th, 2009 [November 8th, 2009]
- Cognitive liberty and right to one's mind - November 8th, 2009 [November 8th, 2009]
- TED Talks: Henry Markram builds a brain in a supercomputer - November 8th, 2009 [November 8th, 2009]
- And Now, for Something Completely Different: Doomsday! - November 8th, 2009 [November 8th, 2009]
- Link dump: 2009.10.19 - November 8th, 2009 [November 8th, 2009]
- Oklahoma and abortion - some fittingly harsh reflections - November 8th, 2009 [November 8th, 2009]
- Pigliucci on science and the scope of skeptical inquiry - November 8th, 2009 [November 8th, 2009]
- Remembering Mac Tonnies - November 8th, 2009 [November 8th, 2009]
- Link dump: 2009.10.24 - November 8th, 2009 [November 8th, 2009]
- Link dump: 2009.10.26 - November 8th, 2009 [November 8th, 2009]
- The Bright Side of Nuclear Armament - November 8th, 2009 [November 8th, 2009]
- Grieving chimps - November 8th, 2009 [November 8th, 2009]
- Elephant prosthetic - November 8th, 2009 [November 8th, 2009]
- Mass produced artificial skin to replace animal testing - November 8th, 2009 [November 8th, 2009]
- Dog gets osseointegrated prosthetic - November 8th, 2009 [November 8th, 2009]
- NASA Shuttle-derived Sidemount Heavy Launch Vehicle Concept - November 8th, 2009 [November 8th, 2009]
- Link dump for 2009.02.02 - November 8th, 2009 [November 8th, 2009]
- Link dump for 2009.11.04 - November 8th, 2009 [November 8th, 2009]
- Link dump for 2009.11.05 - November 8th, 2009 [November 8th, 2009]
- IEET's Biopolitics of Popular Culture Seminar - November 8th, 2009 [November 8th, 2009]
- Einstein and Millikan should have done a Kurzweil - November 8th, 2009 [November 8th, 2009]
- Affective Death Spirals - November 8th, 2009 [November 8th, 2009]
- Cure aging or give a small number of disabled people jobs as janitors? - November 8th, 2009 [November 8th, 2009]
- Would unary notation prevent scope insensitivity? - November 8th, 2009 [November 8th, 2009]
- Cure aging or give a small number of disabled people jobs as janitors - unary version. - November 8th, 2009 [November 8th, 2009]
- At SENS4, Cambridge, UK - November 8th, 2009 [November 8th, 2009]
- SENS4 overview and review - how evolution complicates SENS, and why we must try harder - November 8th, 2009 [November 8th, 2009]
- SENS4 top 10 photos - November 8th, 2009 [November 8th, 2009]
- My AI research for this year - November 8th, 2009 [November 8th, 2009]
- My AI research: Formal Logic - November 8th, 2009 [November 8th, 2009]
- My AI research: Category theory and institution theory - November 8th, 2009 [November 8th, 2009]
- My AI research: The Semantic Web - November 8th, 2009 [November 8th, 2009]
- My AI research: Features and Flaws of Logical representation - November 8th, 2009 [November 8th, 2009]
- My AI research: Graphical models and probabilistic logics - November 8th, 2009 [November 8th, 2009]
- Hughes and More engage Italian Catholicism: Image caption competition - November 8th, 2009 [November 8th, 2009]
- Surprisingly good solutions, falling in love and life in a materialistic universe - November 8th, 2009 [November 8th, 2009]
- What do you get when you cross slightly evolved, status seeking monkeys with the scientific method? - November 8th, 2009 [November 8th, 2009]
- Seeking the optimal philanthropic strategy: Global Warming or AI risk? - November 8th, 2009 [November 8th, 2009]
- Machine Learning - harbinger of the future of AI? - November 8th, 2009 [November 8th, 2009]
- At the Singularity Summit in NYC - November 8th, 2009 [November 8th, 2009]
- Katja Grace: world-dominating superintelligence is "unlikely" - November 8th, 2009 [November 8th, 2009]
- Normal Human Heroes on "Nightmare futures" - November 8th, 2009 [November 8th, 2009]
- Anissimov on Intelligence Enhancement - November 8th, 2009 [November 8th, 2009]
- Yudkowsky on "Value is fragile" - November 8th, 2009 [November 8th, 2009]
- Response to Pearce - November 8th, 2009 [November 8th, 2009]
- Creative thinking lets you believe whatever you want - December 13th, 2009 [December 13th, 2009]
- Let’s get metaphysical: How our ongoing existence could appear increasingly absurd - December 13th, 2009 [December 13th, 2009]
- Linda MacDonald Glenn guest blogging in November and December - December 13th, 2009 [December 13th, 2009]
- Link dump for 2009.11.15 - December 13th, 2009 [December 13th, 2009]
- Call 1-800-New-Organ, by 2020? - December 13th, 2009 [December 13th, 2009]
- IBM's claim to have simulated a cat's brain grossly overstated - December 13th, 2009 [December 13th, 2009]
- John Hodgman pulls off Fermi Paradox schtick - December 13th, 2009 [December 13th, 2009]
- Deus Sex Machina - December 13th, 2009 [December 13th, 2009]
- How Americans spent themselves into ruin... but saved the world - December 13th, 2009 [December 13th, 2009]
- I am my own grandpa (or grandma)? - December 13th, 2009 [December 13th, 2009]
- Link dump for 2009.11.29 - December 13th, 2009 [December 13th, 2009]
- The art of Tomas Saraceno - December 13th, 2009 [December 13th, 2009]
- Link dump: 2009.12.05 - December 13th, 2009 [December 13th, 2009]
- The Harmonic Convergence of Science, Sight, & Sound - December 13th, 2009 [December 13th, 2009]
- Working on my website - December 13th, 2009 [December 13th, 2009]
- Transhumanism, personal immortality and the prospect of technologically enabled utopia - December 13th, 2009 [December 13th, 2009]
- RokoMijic.com is up - December 13th, 2009 [December 13th, 2009]
- Why the Fuss About Intelligence? - December 13th, 2009 [December 13th, 2009]
- Initiation ceremony - December 13th, 2009 [December 13th, 2009]
- Birthing Gods - December 13th, 2009 [December 13th, 2009]
- 11 core rationalist skills - from LessWrong - December 13th, 2009 [December 13th, 2009]
- The best of the guests - December 13th, 2009 [December 13th, 2009]
- The best of Sentient Developments: 2009 - December 13th, 2009 [December 13th, 2009]
- Link dump: 2009.12.15 - December 15th, 2009 [December 15th, 2009]
- The Utopia Force - December 22nd, 2009 [December 22nd, 2009]
- Avatar: The good, the bad and ugly - December 23rd, 2009 [December 23rd, 2009]
- Singularity Institute launches "2010 Singularity Research Challenge" - December 24th, 2009 [December 24th, 2009]
- Transhumanism as a "nonissue" - December 24th, 2009 [December 24th, 2009]
- Hanson on "Meh, Transhumanism" - December 25th, 2009 [December 25th, 2009]
- Merry Newtonmas from Transhuman Goodness - December 25th, 2009 [December 25th, 2009]