Mick Wiggins/Ikon Images/Getty Images
Mick Wiggins/Ikon Images/Getty Images
A surprising medical finding caught the eye of NPR's veteran science correspondent Richard Harris in 2014. A scientist from the drug company Amgen had reviewed the results of 53 studies that were originally thought to be highly promising findings likely to lead to important new drugs. But when the Amgen scientist tried to replicate those promising results, in most cases he couldn't.
"He tried to reproduce them all," Harris tells Morning Edition host David Greene. "And of those 53, he found he could only reproduce six."
That was "a real eye-opener," says Harris, whose new book Rigor Mortis: How Sloppy Science Creates Worthless Cures, Crushes Hope, and Wastes Billions explores the ways even some talented scientists go wrong pushed by tight funding, competition and other constraints to move too quickly and sloppily to produce useful results.
"A lot of what everybody has reported about medical research in the last few years is actually wrong," Harris says. "It seemed right at the time but has not stood up to the test of time."
The impact of weak biomedical research can be especially devastating, Harris learned, as he talked to doctors and patients. And some prominent scientists he interviewed told him they agree that it's time to recognize the dysfunction in the system and fix it.
"If it's not operating at full steam ... and not doing everything right," Harris says, "it's worth pointing that out and saying, 'No. Think about this. Let's make it better.' "
The following has been edited for clarity.
On the ways unreliable research results affect patients
Tom Murphy was a healthy rugby player diagnosed with ALS in his 50s. .... With his doctor's help he signs up for an experimental treatment with a drug called dexpramipexole, or "Dex." At first, he's very hopeful, and it seems to be helping him, but they run the tests and figure out that it actually doesn't work. In fact none of the ALS drugs work. I focus on Tom Murphy because he's a victim of the system here of these failures.
What happened in the case of ALS was there were at least a dozen drugs that had been tried in a handful of small studies way too small of animals. And they all seemed to have some sort of promise some of them went into very large clinical trials. We spent tens of millions of dollars developing these drugs, and they all failed. There's a group in Cambridge, Mass. the ALS Therapy Development Institute that went back and reviewed all these studies and realized all the initial studies were wrong. They used very few mice. They weren't thinking enough about the different genetics of the mice. And a lot of other problems. ... This therapy institute came away thinking none of these drug candidates were really realistic.
On the ways the scientific enterprise in Charles Darwin's time was very different
Darwin was very interesting. It took him decades to come up with his theory of evolution and he was not in a hurry he was studying barnacles, he was studying birds, all sorts of things. He felt no pressure to publish until somebody came up with a similar idea, and he decided, 'Hmmm ... maybe I do want to be first. ..." But we're not in that world anymore. Things are very competitive, very fast-paced. So the competitive world of biomedicine is shaping this problem of evidence that can't be replicated a lot.
On why the delight that's long been an intrinsic part of science can disappear over time and why that's bad
I think a lot of people go into science out of a sense of wonder. But ... as time goes on, people feel the career pressures, and they realize it isn't just about exploring and having big ideas. They have to have research that helps them progress toward their first job, toward tenure, then the next grant, and so on. Those pressures are different from just, sort of, exploring and understanding fundamental biology.... And the less you're focusing on delight, the less maybe you're aiming at the truth and the more you are, inadvertently, often aiming at other goals career goals, financial goals and so on. This may give you a fruitful life as an individual, but may produce less value to us as a society.
On how the public should respond when they hear of a big biomedical advance
I think it is good to question it. Every time you hear something like this, just remember, it's all contingent here is one study, and it may not stand the test of time. I think that's healthy. ... When scientists read the scientific literature, they realize, "Oh, probably half of this is wrong." It's just, not knowing which half that's the vexing part.
On the risk that pointing out flaws in science will make people question its value
It's always uncomfortable to point out problems, but it's also essential. I mean, we are taxpayers we are citizens, and we support this enterprise and we expect to reap its rewards. If it's not operating at full steam ... and not doing everything right, it's worth pointing that out and saying, "No. Think about this. Let's make it better." Many prominent scientists agree with me and are concerned about this and are thinking hard about how to make things better, from the top of NIH on down. There are solutions, and I talk about them in my book.
On why the Trump administration's proposed cuts to NIH funding wouldn't make things better
It's a very appealing idea, obviously, to say, "Oh, well, let's just identify the waste and root it out." But that's not the way science works. ... If you cut the [$30 billion] budget of the National Institutes of Health, you're going to shrink that already very small pool of money even smaller, and you're going to increase the competitive pressures. You're going to increase all these perverse incentives that put us in this position to begin with. So I think that would actually be devastating to biomedical research.
Read more here:
Pressure To Publish Leads To Shoddy Science And Bad Medicine ... - NPR
- Yes, But. The Annotated Atlantic. - November 7th, 2009 [November 7th, 2009]
- Health Insurance Benefit Costs by Region - November 8th, 2009 [November 8th, 2009]
- For an Operator, Please Press... - November 8th, 2009 [November 8th, 2009]
- Pollyanna With a Pen: Maine Governor Signs 18 New Health Care Bills into Law - November 8th, 2009 [November 8th, 2009]
- AMA Sounds the Alarm, Medicare Making Yet Another Attempt to Cut Reimbursement - November 8th, 2009 [November 8th, 2009]
- Mass Governor Asks Blue Cross to Keep Higher Employer Contribution - November 8th, 2009 [November 8th, 2009]
- Lifespan and Care New England Plan Monopoly (Again) - November 8th, 2009 [November 8th, 2009]
- Dirigo Health: Con Artists, Liars, and Thieves? - November 8th, 2009 [November 8th, 2009]
- New Orleans: Health Challenges - November 8th, 2009 [November 8th, 2009]
- August a Flurry of Activity - November 8th, 2009 [November 8th, 2009]
- Maine's Dirigo Health Savings One-Third of Original Estimate - November 8th, 2009 [November 8th, 2009]
- “Methodolatry”: My new favorite term for one of the shortcomings of evidence-based medicine - November 8th, 2009 [November 8th, 2009]
- Suzanne Somers’ Knockout: Dangerous misinformation about cancer (part 1) - November 8th, 2009 [November 8th, 2009]
- A science-based blog about GMO - November 8th, 2009 [November 8th, 2009]
- A Not-So-Split Decision - November 8th, 2009 [November 8th, 2009]
- Military Medicine in Iraq - November 8th, 2009 [November 8th, 2009]
- The effective wordsmithing of Amy Wallace - November 8th, 2009 [November 8th, 2009]
- A Science Lesson from a Homeopath and Behavioral Optometrist - November 8th, 2009 [November 8th, 2009]
- Join CFI in opposing funding mandates for quackery in health care reform - November 8th, 2009 [November 8th, 2009]
- Mainstreaming Science-Based Medicine: A Novel Approach - November 8th, 2009 [November 8th, 2009]
- Those who live in glass houses… - November 8th, 2009 [November 8th, 2009]
- J.B. Handley of the anti-vaccine group Generation Rescue: Misogynistic attacks on journalists who champion science - November 8th, 2009 [November 8th, 2009]
- When homeopaths attack medicine and physics - November 8th, 2009 [November 8th, 2009]
- The cancer screening kerfuffle erupts again: “Rethinking” screening for breast and prostate cancer - November 8th, 2009 [November 8th, 2009]
- All Medicines Are Poison! - November 8th, 2009 [November 8th, 2009]
- When Loud Wins: Will Your Tax Dollars Pay For Prayer? - November 8th, 2009 [November 8th, 2009]
- It’s All in Your Head - November 8th, 2009 [November 8th, 2009]
- The Skeptical O.B. joins the Science-Based Medicine crew - November 8th, 2009 [November 8th, 2009]
- The Tragic Death Toll of Homebirth - November 8th, 2009 [November 8th, 2009]
- What’s the right C-section rate? Higher than you think. - November 8th, 2009 [November 8th, 2009]
- Recombinant Human Antithrombin – Milking Nanny Goats for Big Bucks - December 13th, 2009 [December 13th, 2009]
- Does C-section increase the rate of neonatal death? - December 13th, 2009 [December 13th, 2009]
- Man in Coma 23 Years – Is He Really Conscious? - December 13th, 2009 [December 13th, 2009]
- Why Universal Hepatitis B Vaccination Isn’t Quite Universal - December 13th, 2009 [December 13th, 2009]
- Ontario naturopathic prescribing proposal is bad medicine - December 13th, 2009 [December 13th, 2009]
- Naturopaths and the anti-vaccine movement: Hijacking the law in service of pseudoscience - December 13th, 2009 [December 13th, 2009]
- The Institute for Science in Medicine enters the health care reform fray - December 13th, 2009 [December 13th, 2009]
- Neti pots – Ancient Ayurvedic Treatment Validated by Scientific Evidence - December 13th, 2009 [December 13th, 2009]
- Early Intervention for Autism - December 13th, 2009 [December 13th, 2009]
- A temporary reprieve from legislative madness - December 13th, 2009 [December 13th, 2009]
- A critique of the leading study of American homebirth - December 13th, 2009 [December 13th, 2009]
- Lose those holiday pounds - December 13th, 2009 [December 13th, 2009]
- Endocrine disruptors—the one true cause? - December 13th, 2009 [December 13th, 2009]
- Acupuncture for Chronic Prostatitis/Chronic Pelvic Pain Syndrome - December 13th, 2009 [December 13th, 2009]
- Evidence in Medicine: Experimental Studies - December 13th, 2009 [December 13th, 2009]
- Midwives and the assault on scientific evidence - December 13th, 2009 [December 13th, 2009]
- The Mammogram Post-Mortem - December 13th, 2009 [December 13th, 2009]
- An Influenza Recap: The End of the Second Wave - December 13th, 2009 [December 13th, 2009]
- The End of Chiropractic - December 13th, 2009 [December 13th, 2009]
- Cell phones and cancer again, or: Oh, no! My cell phone’s going to give me cancer! (revisited) - December 20th, 2009 [December 20th, 2009]
- Another wrinkle to the USPSTF mammogram guidelines kerfuffle: What about African-American women? - December 20th, 2009 [December 20th, 2009]
- Acupuncture, the P-Value Fallacy, and Honesty - December 20th, 2009 [December 20th, 2009]
- The One True Cause of All Disease - December 20th, 2009 [December 20th, 2009]
- Communicating with the Locked-In - December 20th, 2009 [December 20th, 2009]
- Are the benefits of breastfeeding oversold? - December 20th, 2009 [December 20th, 2009]
- Measles - December 20th, 2009 [December 20th, 2009]
- Radiation from medical imaging and cancer risk - December 21st, 2009 [December 21st, 2009]
- Multiple Sclerosis and Irrational Exuberance - December 21st, 2009 [December 21st, 2009]
- Medical Fun with Christmas Carols - December 22nd, 2009 [December 22nd, 2009]
- Lithium for ALS – Angioplasty for MS - December 23rd, 2009 [December 23rd, 2009]
- “Toxins”: the new evil humours - December 24th, 2009 [December 24th, 2009]
- 2009’s Top 5 Threats To Science In Medicine - December 24th, 2009 [December 24th, 2009]
- Buteyko Breathing Technique – Nothing to Hyperventilate About - December 26th, 2009 [December 26th, 2009]
- The Graston Technique – Inducing Microtrauma with Instruments - December 29th, 2009 [December 29th, 2009]
- The “pharma shill” gambit - December 29th, 2009 [December 29th, 2009]
- Ginkgo biloba – No Effect - December 30th, 2009 [December 30th, 2009]
- Oppose “Big Floss”; practice alternative dentistry - January 1st, 2010 [January 1st, 2010]
- Causation and Hill’s Criteria - January 3rd, 2010 [January 3rd, 2010]
- The life cycle of translational research - January 10th, 2010 [January 10th, 2010]
- The anti-vaccine movement strikes back against Dr. Paul Offit - January 10th, 2010 [January 10th, 2010]
- Osteoporosis Drugs: Good Medicine or Big Pharma Scam? - January 10th, 2010 [January 10th, 2010]
- Acupuncture for Hot Flashes - January 10th, 2010 [January 10th, 2010]
- The case for neonatal circumcision - January 10th, 2010 [January 10th, 2010]
- A victory for science-based medicine - January 10th, 2010 [January 10th, 2010]
- James Ray and testosterone replacement therapy (TRT) - January 10th, 2010 [January 10th, 2010]
- The Water Cure: Another Example of Self Deception and the “Lone Genius” - January 12th, 2010 [January 12th, 2010]
- Be careful what you wish for, Dr. Dossey, you just might get it - January 13th, 2010 [January 13th, 2010]
- You. You. Who are you calling a You You? - January 15th, 2010 [January 15th, 2010]
- The War on Salt - January 16th, 2010 [January 16th, 2010]
- Is breech vaginal delivery safe? - January 16th, 2010 [January 16th, 2010]