NB: This is a partial posting; I was up all night ‘on-call’ and too tired to continue. I’ll post the rest of the essay over the weekend, when this note will disappear.

Review

This is the fourth and final part of a series-within-a-series* inspired by statistician Steve Simon. Professor Simon had challenged the view, held by several bloggers here at SBM, that Evidence-Based Medicine (EBM) has been mostly inadequate to the task of reaching definitive conclusions about highly implausible medical claims. In Part I, I reiterated a fundamental problem with EBM, reflected in its Levels of Evidence scheme, that although it correctly recognizes basic science and other pre-clinical evidence as insufficient bases for introducing novel treatments into practice, it fails to acknowledge that they are necessary bases. I explained the difference between “plausibility” and “knowing the mechanism.”

I showed, with several examples, that in the EBM lexicon the word “evidence” refers almost exclusively to the results of clinical trials: thus, when faced with equivocal or no clinical trials of some highly implausible claim, EBM practitioners typically declare that there is “not enough evidence” to either accept or reject the claim and call for more trials—although in many cases there is abundant evidence, other than clinical trials, that conclusively refutes the claim. I rejected Prof. Simon’s assertion that we at SBM want to “give (EBM) a new label,” making the point that we only want it to live up to its current label by considering all the evidence. I doubted Prof. Simon’s contention that “people within EBM (are) working both formally and informally to replace the rigid hierarchy with something that places each research study in context.”

In Part II I responded to the widely held assertion, also held by Prof. Simon, that there is “societal value in testing (highly implausible) therapies that are in wide use.” I made it clear that I don’t oppose simple tests of basic claims, such as the Emily Rosa experiment, but I noted that EBM reviewers, including those employed by the Cochrane Collaboration, typically ignore such tests. I wrote that I oppose large efficacy trials and public funding of such trials. I argued that the popularity gambit has resulted in human subjects being exposed to dangerous and unethical trials, and I quoted language from ethics treatises specifically contradicting the assertion that popularity justifies such trials. Finally, I showed that the alleged popularity of most “CAM” methods—as irrelevant as it may be to the question of human studies ethics—has been greatly exaggerated.

In Part III I continued to argue against trials of implausible methods. I didn’t share Prof. Simon’s optimism, expressed in another post on his blog, that “research can help limit the fraction of CAM expenditures that are inappropriate.” I argued that whatever evidence there is suggests otherwise. I argued that if existing science is sufficient to reject a method, as is the case for much of “CAM,” then the research has already been done, and the task of EBM is to explain this kind of evidence—not to pretend that the jury is still out. I argued, furthermore, that efficacy trials of highly implausible, ineffective methods inevitably yield equivocal, rather than merely disconfirming results, and that this leads to an endless cycle of further (equivocal) trials. I offered parapsychology as a longstanding example of such an “an immortal field of fruitless inquiry: a pathological science.”

I promised that in this final part of the series I would mention a few more points about Cochrane Reviews of highly implausible methods, and even report some reasons for slight optimism. Finally, I promised to respond briefly to this comment by Prof. Simon:

…how can we invoke scientific plausibility in a world where intelligent people differ strongly on what is plausible and what is not? Finally, is there a legitimate Bayesian way to incorporate information about scientific plausibility into a Cochrane Collaboration systematic overview(?)

Cochrane Reviews 

The ongoing theme is that Cochrane Reviews ignore key ‘external’ evidence, by which I mean all evidence other than what might be found in randomized, controlled trials (RCTs). I’ve previously alluded to the 2002 Cochrane Review of “Chelation Therapy for Atherosclerotic Cardiovascular Disease,” which concluded,

At present, there is insufficient evidence to decide on the effectiveness or ineffectiveness of chelation therapy in improving clinical outcomes of people with atherosclerotic cardiovascular disease.

Elsewhere we have shown that the evidence against such effectiveness is substantial, far exceeding the evidence against Laetrile for cancer or against bilateral ligation of the internal mammary arteries for coronary disease, two long-since discredited methods that no biomedical researcher in his right mind would consider resurrecting for further trials. Oops, make that almost no biomedical researcher.

Quips aside, my reason for bringing up the chelation review again is as follows. The review acknowledges that there have been RCTs, involving about 250 subjects, which

…showed no significant difference in the following outcomes: direct or indirect measurement of disease severity and subjective measures of improvement.

That those findings weren’t sufficient reasons for the Cochrane reviewers to judge chelation ineffective was apparently due first to their having ignored the abundant non-RCT evidence, and second to their having been intrigued by a single, tiny RCT that reported a positive outcome:

One of the studies, which included only 10 participants, was interrupted prematurely, because of an apparent treatment effect. However, relevant data were not available in the report and have been requested from the authors.

I had to chuckle when I read that passage, because I know a lot about that study and its authors. If you look here and scroll down to “Olszewer (1988),” you will know, too. I wonder if the Cochrane Reviewers ever got a straight story from the authors. There hasn’t been an update of that review, so whether they did or not is anyone’s guess.

I’m sorry to say that I haven’t been able to get a copy of that review in its entirety, so my comments apply only to the abstract. I have recently obtained a few other complete reviews, of which two are worth mentioning. A 2008 review of “Touch therapies for pain relief in adults” looked at Healing Touch, Therapeutic Touch, and Reiki. It doesn’t mention the Emily Rosa experiment. Its conclusion is what we’ve come to expect:

Touch therapies may have a modest effect in pain relief. More studies on HT and Reiki in relieving pain are needed. More studies including children are also required to evaluate the effect of touch on children.

In this review we are told why it is that not touching is called “touch”:

Touch Therapies are so-called as it is believed that the practitioners have touched the clients’ energy ?eld.

For readers who are unfamiliar with such practices, which consist of waving one’s hands over a “client” (you really have to see it to believe it), the review continues:

It is believed this effect occurs by exerting energy to restore, energize, and balance the energy ?eld disturbances using hands-on or hands-off techniques (Eden 1993). The underlying concept is that sickness and disease arise from imbalances in the vital energy ?eld. However, the existence of the energy ?eld of the human body has not been proven scienti?cally and thus the effect of such therapies, which are believed to exert an effect on one’s energy ?eld, is controversial and lies in doubt.

Indeed. The following passages are not to be read in detail, other than by the masochistic. Their purpose is to demonstrate the elaborate wheel spinning that EBM treatments of such fanciful methods inevitably involve. The perseveration of statistics, as if they can support the house of cards that is the basis for the technique, will, I hope, ruffle Prof. Simon’s own energy field: 

Types of touch therapies

The effects of different kinds of Touch Therapies were examined (Comparison 3). It appears that all three types of Touch Therapy, HT, TT and Reiki, may decrease pain to a certain extent. Substantial heterogeneity exists among the HT group and TT group. The HT group (163 participants) had an I2 of 76% and a P value= 0.04 (Chi-square) and the TT group (686 participants) had an I2 of 70% and a P value < 0.00001 (Chi-square). The results for both the TT and the HT group indicate that there is signi?cant heterogeneity and that the effects were positive. There were two studies in HT included in the analyses (Cook 2004; Post-White 2003). The pooled results showed that participants exposed to HT had, on average, 0.71 units less pain, however, this was not statistically signi?cant (95% CI: -2.27 to 0.86). The pooled estimates of TT suggested a statistically signi?cant result of 0.81 units (95% CI: -1.19 to -0.43) less pain in the exposed group. Nonsigni?cant heterogeneity was detected in the Reiki group (116 participants) which consisted of three studies (Dressen 1998; Olson 2003; Tsang 2007) with an I2 of 7% and a P value = 0.34 (Chi-square). The pooled estimates of the results for Reiki reported a statistically signi?cant effect. Participants exposed to Reiki had an average of 1.24 (95% CI: -2.06 to -0.42) less pain.

Experience of practitioner

The experience of the practitioner on the effects of touch therapies was also analyzed. This helped to explore whether a less experienced touch practitioner would result in less effect and an experienced touch practitioner would result in an increased effect. Subgroup analyses were thus performed. There are usually four levels of training in HT, TT and Reiki, level I, II, III and a master level or teacher level. Studies were divided into subgroups according to the level of training. Due to a small number of studies having reported the experience of touch practitioners, studies were divided into two groups, level I and II , and level III or above, rather than four groups. Four studies were included in the subgroup of less experienced practitioners (212 participants) (Blank?eld 2001; Cook 2004; Frank 2003; Redner 1991). An I2 of 2% and a P value = 0.38 (Chi-square) was found. In the subgroup of more experienced practitioners (116 participants), three studies were included (Dressen 1998; Olson 2003; Tsang 2007) and an I2 of 7% and a P value = 0.34 (Chi-square) was found. No signi?cant heterogeneity was detected in the two subgroups. Participants in the subgroup of less experienced practitioners had, on average, 0.47 units (95% CI: -0.73 to -0.22) less pain. More experienced practitioners yielded higher contribution, having on average of 1.24 units reduction in pain intensity (95% CI: -2.63 to -0.23) (Comparison 4). Minor non-signi?cant heterogeneity existed in both the experienced and less-experienced group. This might suggest that any heterogeneity calculated in other subgroups was owing to the difference in the experience of the practitioners. However, only a small number of identi?ed studies were included in this subgroup and the apparent small heterogeneity may be owing to the low power of the chi-square test or due to having a small numbers of studies (Higgins 2005), it would not be appropriate to make conclusions about the existence of heterogeneity at this stage with the current results. 

Dose-response analyses

Dose-response analyses were also conducted to investigate if there was any difference in effect due to differences in duration of treatment. The study with the shortest session had the session lasting for ?ve minutes while studies with the longest session lasted for ninety minutes. The number of treatment sessions ranged from a single session to ten sessions. Data regarding the duration and number of treatment sessions were pooled. Data were analyzed in terms of total duration of treatment (duration of a single session multiplied by the number of treatment sessions). Three hundred and ninety six participants exposed to touch for less than an hour had an average of 1.16 units (95% CI: -1.85 to -0.47) less pain; 239 participants exposed to touch for more than one hour but less than two hours had an average of 0.75 units (95% CI: -1.81 to 0.31) less pain, but this was insigni?cant; 255 participants exposed to touch for between two to three hours had an average of 0.47 units (95% CI: -1.09, 0.14) less pain; 116 participants exposed for over three hours to touch therapies had an average of 1.57 units less pain (95% CI: -2.38 to -0.76). No dose-response relationship can be gained as yet from this information (Comparison 7).

Phew! (‘Dose-response analyses’ ?). At the end of the monograph are the first author’s acknowledgements:

I would like to thank Dr Yan-Kit Cheung, who is my Reiki teacher and an expert in complementary and alternative medicine, in giving me valuable advice in conducting this review…Last but not least, I would like to express my gratitude to Miss Wan-Choi Patsy Lee who brought me to Reiki. Without her, I would know nothing about Reiki and would not be enlightened by this precious gift.

I don’t know who should be more embarrassed: Cochrane, for having asked a devout believer to pass judgment on the sacred object of her belief, or me, for having stumbled upon her supplication.

…

*The Prior Probability, Bayesian vs. Frequentist Inference, and EBM Series:

1. Homeopathy and Evidence-Based Medicine: Back to the Future Part V

2. Prior Probability: The Dirty Little Secret of “Evidence-Based Alternative Medicine”

3. Prior Probability: the Dirty Little Secret of “Evidence-Based Alternative Medicine”—Continued

4. Prior Probability: the Dirty Little Secret of “Evidence-Based Alternative Medicine”—Continued Again

5. Yes, Jacqueline: EBM ought to be Synonymous with SBM

6. The 2nd Yale Research Symposium on Complementary and Integrative Medicine. Part II

7. H. Pylori, Plausibility, and Greek Tragedy: the Quirky Case of Dr. John Lykoudis

8. Evidence-Based Medicine, Human Studies Ethics, and the ‘Gonzalez Regimen’: a Disappointing Editorial in the Journal of Clinical Oncology Part 1

9. Evidence-Based Medicine, Human Studies Ethics, and the ‘Gonzalez Regimen’: a Disappointing Editorial in the Journal of Clinical Oncology Part 2

10. Of SBM and EBM Redux. Part I: Does EBM Undervalue Basic Science and Overvalue RCTs?

11. Of SBM and EBM Redux. Part II: Is it a Good Idea to test Highly Implausible Health Claims?

12. Of SBM and EBM Redux. Part III: Parapsychology is the Role Model for “CAM” Research

13. Of SBM and EBM Redux. Part IV: More Cochrane and a little Bayes

A number of news outlets (e.g. Bloomberg Business Week, MSN.Com, US News, etc) have recently reported that use of complementary and alternative therapies (CAT) is widespread in hospice care facilities. This is based on a report from the Centers for Disease Control, Complementary and Alternative Therapies in Hospice: The National Home and Hospice Care Survey, Untied States, 2007. According to most news reports, about 42% of hospice care providers offer some kind of CAT.

I was initially inclined to find this a little worrisome. In my own field of veterinary medicine, advocates of alternative therapies are prominent among the organizers of the nascent hospice care movement. And while I am strongly supportive of better and more available veterinary hospice care, the involvement of CAM advocates raises the concern that animals at the end of their life might receive ineffective palliative care, or be denied the benefits of conventional treatments by some CAM providers, who often characterize “allopathic” treatments as “unnatural” and harmful.

In practice, I have seen this happen to patients with terminal diseases. I will never forget a Rottweiler dog I diagnosed with osteosarcoma, a very painful bone cancer, whose owner was convinced that homeopathy was adequate to control his pain and refused to use NSAIDs because of her conviction they were “toxic.” I have also seen my patients denied euthanasia even in the face of great suffering because so-called “animal communicators” claimed the pet was “not ready to leave” and had expressed a desire to remain with their owner as long as possible.

Perhaps these experiences have made me overly sensitive on this subject, but I saw these recent news reports and pictured people at the end of their lives being similarly denied effective palliative care or subjected to pointless therapies like homeopathy and “energy medicine,” or even more worrisome treatments like chiropractic or herbal remedies with real risks. However, a little digging into the details suggests that the headlines are a bit misleading, and these fears are probably unfounded.

As always, when trying to assess how popular alternative medical therapies are, the tricky issue arises of defining “alternative.” In this study, the authors referenced the MedlinePlus definition:

Complementary and alternative medicine (CAM) is the term for medical products and practices that are not part of standard care. Standard care is what medical doctors, doctors of osteopathy and allied health professionals, such as registered nurses and physical therapists, practice. Alternative medicine means treatments that you use instead of standard ones. Complementary medicine means nonstandard treatments that you use along with standard ones. Examples of CAM therapies are acupuncture, chiropractic and herbal medicines.

Personally, I prefer Dr. Novella’s definition:

CAM is a political/ideological entity, not a scientific one. It is an artificial category created for the purpose of promoting a diverse set of dubious, untested, or fraudulent health practices. It is an excellent example of the (successful) use of language as a propaganda tool.

In any case, in order to measure the popularity of something, one has to define it in some way, and in the past assessments of how popular or widespread CAM use is have created misleading impressions due to dodgy definitions. For example, the 2007 National Health Interview Survey (discussed in detail here) reported 30% of Americans to be regular CAM users. A closer look at the details of the survey, however, showed that very little of this self-reported usage involved the application of the usual dubious CAM approaches (e.g. acupuncture, chiropractic, homeopathy, various herbal traditions, etc) to treat specific medical problems. Much of this supposed CAM usage involved the non-medical application of massage, yoga, tai chi, prayer, and so on to provide psychological comfort or facilitate relaxation.

Of course, if one argues that massage, yoga, or even prayer are effective in reducing the objective signs or disease, or even bringing about a cure, then one could argue these are forms of alternative medicine. But such methods are mostly employed to provide comfort and help patients cope with their illness, and as such they can be valuable and legitimate interventions. This does not make them medical therapies, however, alternative or otherwise.

The hospice care survey suffers from the same kind of problematic definition for “complementary and alternative.” According to the report’s technical notes, providers of hospice care were asked first to choose all the services they offered from a list, and “Complementary and Alternative Medicine (CAM)” was one of the choices. Those that indicated they offered CAM were then asked to indicate “Which of these complementary and alternative medicine therapies does this agency use?”

Here is the list:

1. Acupuncture
2. Aromatherapy
3. Art therapy
4. Guided imagery or relaxation
5. Massage
6. Music therapy
7. Pet therapy
8. Supportive group therapy
9. Therapeutic touch (a westernized version of reiki)
10. TENS (Transcutaneous Electrical Nerve Stimulation)
11. Other

Personally, I see little on this list that I would classify as CAM. Acupuncture, certainly, along with therapeutic touch (like reiki) and aromatherapy. But most of the rest, unless specifically marketed as treatments for disease, seem more like benign, pleasurable activities designed to provide comfort, relaxation, and enjoyable stimulation. As a veterinarian, I work with a lot of pet therapy dogs, and I have yet to run across a handler of one who thought they were practicing alternative medicine! (Though I suppose there might be some such folks out there). And TENS is a perfectly conventional intervention, often somewhat disingenuously confused with acupuncture.

The most popular of the “true” CAM therapies offered was therapeutic touch, available at 48.3% of facilities. Aromatherapy was offered by 39.7% of hospice providers. I cannot even find a number for acupuncture in the report. And by far the most popular “alternative” therapies offered were massage (71.7%), group therapy (69%), music therapy (62.2%), and pet therapy (58.6%).

The report also indicates that only 8.6% of patients discharged from a hospice facility that offered CATs actually received one of these therapies. So even under such a loose definition of alternative, there is no evidence that large numbers of hospice patients are receiving alternative medical treatments.

It wouldn’t surprise me if we begin to see advocates of alternative medicine proclaiming that this report shows CAM is widely available, popular, and even indispensible in hospice care. The 2007 National Health Interview Survey results were frequently used this way to create the impression that CAM is becoming mainstream and that resistance to it is the province of extremists and ultimately futile. The details of both surveys, however, indicate that even with aggressive expansion of the definition of CAM to include conventional therapies such as TENS and non-medical interventions like pet therapy, CAM is not truly as popular ubiquitous as its proponents claim.

There is little objectionable from a science-based medicine perspective in most of the therapies hospice care providers are offering, according to this study. I enjoy a good massage, relaxing music, and the company of a friendly dog as much as anyone. And those elements that are truly nonsense, such as therapeutic touch and aromatherapy, are unlikely to do harm or replace appropriate conventional therapies, and they seem in any case not to be especially popular with patients even when they are available. So regardless of what PR use is made of this study, it does not suggest that human hospice care is becoming predominantly the domain of CAM providers, as I might have feared. I only hope the same will be true of veterinary hospice care as that becomes, hopefully, more commonplace.

A couple of weeks ago, both Steve Novella and I criticized Dr. Mehmet Oz (a.k.a. “America’s doctor”) for not only hosting a man I consider to be a major supporter of quackery, but going far beyond that to defend and promote him. After that, I considered Dr. Oz to be a lost cause, with nothing to excuse him for his having embraced a man whose website is a wretched hive of scum and quackery almost as wretched as NaturalNews.com (in my opinion, of course). Unfortunately, apparently Dr. Oz’s defense of Dr. Mercola was only the beginning of the end of whatever minimal credibility Dr. Oz had left as a practitioner of evidence-based medicine.

This week, Dr. Oz put the final nails in the coffin of his credibility as a practitioner of science-based medicine. I realize that some would argue that he did that long ago. Fair enough. However, I always held out some hope that he might stop mixing pseudoscience like reiki with science. Then he embraced Dr. Joseph Mercola. Strike one! Unfortunately, strikes two and three followed over the last week or so.

Dr. Oz embraces the “bait and switch” of alternative medicine.

One message that we’ve been trying to get SBM readers to understand is that much of what falls under the rubric of “alternative” medicine, “complementary and alternative medicine” (CAM), and “integrative medicine” (IM) is in reality a fairly obvious “bait and switch,” as Steve Novella put it so well. The bait consists of various modalities that naturally fall into the bailiwick of science-based medicine (SBM)–or at least should. These modalities include diet, exercise, relaxation. Indeed, it irritates me to no end when various apologists and advocates for CAM claim that science-based physicians don’t recognize the importance of diet and exercise or how they can have a profound effect on health, in particular on diseases like type II diabetes. I ranted about this not long ago when i wrote about the woo of raw “living food” diets, in which “living food” advocates claim that it requires extreme raw vegan diets to “cure” diabetes. Diet and exercise are every bit a part of science-based medicine; yet CAMsters appropriate them as being somehow “alternative,” the better to bring in the real woo along with them. The pitch is, in essence, that diet and exercise clearly work and are “alternative.” Therefore there must be something to other forms of “alternative” medicine. That’s the “switch” in the bait and switch. Nowhere is this switch better demonstrated than in a segment from The Dr. Oz Show last week called Dr. Oz’s Holistic Health Overhaul.

At the beginning of part 1 of this segment, Dr. Oz appears on stage and announces his “holistic health overhaul.” What strikes me is that what Dr. Oz is supposedly “overhauling” is based on people who feel run down, who lack energy, who feel older than they are, symptoms virtually all of us feel at one time or another to one degree or another. (How is one supposed to know if one feels his or her age, anyway?) Right from the start, Dr. Oz promises that he can make you feel younger and better, all within 28 days. To do this, he immediately introduces a yoga instructor named Yogi Cameron Alborzian, who pontificates about spirituality, yoga, and how he became a yogi. As is typical for such practitioners, there are many softly lit and fuzzy shots of Yogi Cameron doing yoga poses in beautiful, natural surroundings (of course!) interspersed with shots of him talking about “mind-body” connections and how he asks his clients what they’re feeling.

In part 2, Patricia from New Jersey is introduced. Patricia is a stay-at-home mother of four boys, who describes herself as feeling “toxic, tired, and stressed-out.” Patricia’s biggest problem, according to her, is that she’s a self-admitted “junk food junkie” who doesn’t exercise. She also complains of sluggishness, feeling “hung over,” and wiped out. In response to her complaints, Yogi Cameron and Dr. Oz show Patricia her health risks, pointing out that her body mass index qualifies her as obese and that her waist size suggests that most of her fat is in her abdomen, which is known to be a risk factor for type II diabetes and a variety of other health problems. Basically, the problem is laid out. It’s (mostly) science-based. Unfortunately, the solution is a mixture of woo and science-based diet and lifestyle changes. That’s the bait. Enter Dr. Oz and Yogi Cameron and his “holistic health overhaul” in part 3 and part 4. Here’s where the switch comes in.

First, let me show why the woo that flows is not a surprise at all by referring you to Yogi Cameron’s own website and a promotional video for a book he’s written:

We learn that Yogi Cameron was a fashion model for several years and even was cast in what looks like the video for Madonna’s song Express Yourself. (He’s the buff, sweaty, half-naked worker seduced by Madonna near the end of the video.) Also on his website, Yogi Cameron opines about Ayruvedic medicine, referring to it as the “science of life”:

Before Western medicine, before homeopathic medicine, and before even traditional Chinese medicine, there was Ayurveda. This is an ancient system of healing created by sages in India over five thousand years ago. While yoga was developed as a science for the practitioner to bring balance and control to the mind, Ayurveda is a sister science developed for the practitioner to bring balance to the body.

Western medicine tends to treat a patient’s symptoms with different pills and medications without any attention to healing the cause of a disease that is feeding the symptom. It is like weeding a garden without taking out the roots; the weeds just grow back. Ayurveda works to define the cause of the patient’s symptoms and to treat the body with various methods for the sake of restoring balance to the system as a whole. These methods include eating in a way appropriate to one’s constitution, taking herbal supplements and remedies, and receiving treatments such as oil massage. Effective use of Ayurveda can help to alleviate digestive problems, allergies, insomnia, asthma, obesity, migraines, and many other bodily complaints.

The ancient sages who developed Ayurveda many centuries ago observed that our bodies are formed by three fundamental energy types or doshas. The first (Pitta) is responsible for metabolizing for the sake of processing oxygen and perpetuating life. The second (Kapha) forms our bodies, which serves as a container so that life can exist as matter. The third (Vata) shifts matter’s position in space through the act of motion.

And this is how Yogi Cameron treats his clients’ problems:

Through other Ayurvedic treatments such as Pancha Karma we also clean the inside of the body. Cleaning the inside of our system is fundamental to our wellbeing and without such cleanings we can never experience complete health and vitality, youth and vigor. When the inside of the body is clean we experience young skin and vibrate energy on the outside.

For those of you who don’t know what Pancha Karma is, it the name for five actions that make up an Ayruvedic method to purify the body. There are three stages of treatment. First, there is the pretreatment, which consists of oil therapy, massage, and something called formentation therapy. This part actually doesn’t sound too bad. Whether it cures anything or not, who knows? However, having your body oiled up and massage can’t be all bad. The formentation therapy is basically heat, either steam from herbs, sitting under the sun, or using warm blankets. Of course, this latter treatment, depending on what it is used for, is a perfectly fine science-based treatment. Be that as it may it’s the next part of the Pancha Karma that is supposed to do the purification. This consists of Nasya (nasal therapy), Vamana (emesis or vomiting), Virechana (purging) and two kinds of Vasti (therapeutic enema), Nirooha Vasti and Sneha Vasti.

Enemas? What is it with enemas? Truly, enemas seem to be the woo that knows no national or ethnic boundaries, the quackery that is truly world-wide. Fortunately, there does appear to be an alteration to this ancient art of purging in America:

Originally, this phase consisted of five practices: nasal cleansing, enemas, laxatives, emesis (vomiting), and blood-letting. Although the five practices are followed in India, the practice of emesis and blood-letting is omitted in North America.

I suppose we should be grateful for small favors in that the bloodletting is left out by our very North American woo-meisters. Dr. Oz then reveals the “switch,” describing week 1 of this plan as “detox.” For his part, Yogi Cameron helpfully chimes in that his methods “burn off toxins.” Dr. Oz then immediately asks Yogi Cameron about tongue examination. Now, there’s one thing you need to know about tongue examination. When an Ayruvedic practitioner or a practitioner of traditional Chinese medicine talks about tongue examination, he is not referring to the sorts of things I learned in medical school about tongue examinations, where we look for turgor, moistness, plaques, and a variety of other physical findings that can be indicative of disease. No, the Ayruvedic art of tongue diagnosis is very much like reflexology in that various organs are claimed to map to various parts of the tongue:

To be fair, some of the tongue diagnoses actually do agree with science-based medicine, for example, a yellow tongue being indicative of jaundice. However, someone with jaundice will also usually have yellow visibile in their sclerae, which are probably more sensitive. In reality, the Ayruvedic tongue diagnoses that match science-based medicine diagnoses are actually a classic case of being right for all the wrong reasons, and most of them are wrong, wrong, wrong, particularly the mapping of various organs to different parts of the tongue. Not that that stops Yogi Cameron from proclaiming that the “head is represented by the tip of the tongue.”

I will admit that there is one mildly amusing part of this entire segment. Yogi Cameron has a rather strong disagreement about the amount of sex people should have. Yogi thinks that people shouldn’t have sex too much; Dr. Oz is apparently a randy little bugger and thinks people should have sex all the time, the thought of which is an image I don’t want in my head. Be that as it may, one thing that strikes me about this argument is that it appears to be vitalistic in nature. Yogi Cameron claims you shouldn’t have too much sex because it’s about “conserving energy,” in essence implying that sex somehow saps your life energy. This is not unlike various pre-scientific beliefs that semen is the equivalent of life energy, which is why men shouldn’t have sex before battle, athletic contests, or anything that’s likely to require a large energy expenditure.

At this point, the “bait and switch” is complete. Dr. Oz had presented the story of a typical middle class mother who works hard, doesn’t eat right, is a bit obese, and as a consequence of her lack of exercise, her work, and her poor diet feels run down all the time. A perfectly fine science-based solution to her problems would involve a change in diet to something healthier, cutting out the junk food, and adding regular exercise (all things that I myself have a lot of problem managing to do, truth be told). Instead, what Dr. Oz and his guest Yogi Cameron present is an improved diet, plus yoga, plus woo that includes tongue diagnosis, “detox,” and “Nasya lite” (given that all Yogi Cameron had Patricia do was to place some Ghee in her nose, rather than shooting water in and out of it). At least he spared her the purging and enemas, but I bet if Patricia had come to Yogi Cameron’s center those would have been part of the mix. But there’s enough there, even the classic favorite of apologists for Ayruveda and traditional Chinese medicine, the appeal to ancient wisdom, the claim that, if people have been doing this for thousands of years, there must be something to it, they must know something we don’t.

Strike two!

To mix baseball and Star Wars metaphors (hmmm, light sabers instead of bats?), Dr. Oz has two strikes against him now, but is his journey to the Dark Side complete? He’s certainly controlled his message, but has he fully released his woo? Unfortunately, Tuesday’s episode this weeks demonstrates that Dr. Oz has truly become the master of woo.

Dr. Oz: Falling for faith healing

To abuse my Star Wars metaphors yet again, if Dr. Oz’s featuring of Yogi Cameron on an episode of his show last week was the equivalent of Anakin Skywalker slaughtering a tribe of Tusken raiders for having tortured and killed his mother Shmi, Tuesday’s episode was Anakin cum Darth Vader hitting the Jedi temple with a bunch of storm troopers and slaughtering all the younglings. Either that, or it was Anakin cutting off Mace Windu’s hand, allowing Emperor Palpatine the opening he needed to kill Windu. Take your pick. In other words, Dr. Oz’s credulous treatment of faith healing definitively marks the point of no return, the point at which Dr. Oz’s journey to the Dark Side is now complete. All he needs is a Darth Vader mask. Or maybe a mask of Samuel Hahnemann. Or something.

The reason Tuesday’s episode definitively marks a point of no return for Dr. Oz when it comes to his support for quackery is because he has apparently decided to follow his TV mentor Oprah Winfrey’s example in realizing that faith healing sells. Of course, Dr. Oz, as popular as he is, is not as well established as Oprah. Whereas Oprah got John of God, complete with his “psychic surgery,” Dr. Oz gets a second tier faith healer. He gets Dr. Issam Nemeh. Of course, Dr. Oz is a surgeon; so maybe he is less easily taken in by parlor tricks in which tiny superficial incisions are made. Or maybe not. Just being a physician does not guarantee not being taken in by faith healing nonsense, as we’ve seen many times. In any case, Dr. Nemeh must be very grateful to Dr. Oz, because when you look at his website, you’ll be greeted with a message:

Welcome Dr. Oz Viewers!

Dr. Nemeh has received an overwhelming response from the viewers of the Dr. Oz show. Medical office appointments with Dr. Nemeh are already filled for the next four months.

To add your name to the cancelation list, send an email with your name, phone number, and reason for treatment to appointments@drnemeh.com.

But how did Dr. Nemeh get so popular suddenly? Behold the power of Dr. Oz and his segment on Tuesday’s show entitled, Is this man a faith healer?

If you recall my discussion of Oprah Winfrey’s utterly credulous treatment of John of God, you might wonder if Dr. Oz did any better than Oprah’s staff and Oprah herself. Going in, I actually expected that Dr. Oz’s segment about Dr. Nemeh would be harder for me to deconstruct. Indeed, I expected it to be much harder to deconstruct. Dr. Oz is, after all, a cardiothoracic surgeon. Also, in the preview for the episode featuring Dr. Nemeh, there was a clip showing Dr. Oz with small pile of charts saying that he had asked to be allowed to examine the medical records of some of Dr. Nemeh’s patients. Given that and given that Dr. Nemeh is a physician himself, I figured that, between the two of them, Drs. Oz and Nemeh would be able to cherry pick cases that would appear truly convincing and thus be very difficult to refute. When that happened, I feared I’d be reduced to saying that single anecdotes are not convincing, which, while true, is a relatively hard sell to lay readers without medical training. Even some physicians remain unsatisfied by such an explanation, and it’s not hard to figure out why. Fortunately for me (and unfortunately for Drs. Nemeh and Oz), I needn’t have worried. You’ll see what I mean in a minute.

The first segment begins, as usual, with Dr. Oz introducing the topic. In this case, Dr. Oz breathlessly proclaims this to be a show “unlike any other we have done before” and describes how he has been “fascinated” by “this doctor in Cleveland.” We’re then shown several people in the audience who claim to have been healed by Dr. Nemeh, who is described as a doctor who doesn’t use drugs or procedures but “heals with his hands.” Dr. Nemeh, we’re told, uses a “high tech form of acupuncture” in his office and the laying on of hands and the use of spirit in churches and meeting halls, all to “heal.” During this voiceover, we’re treated to images of Dr. Nemeh in action, including a paralyzed patient who claims that he’s noticed some movement in his feet since Dr. Nemeh started treating him, a woman who implied that she had her vision restored, and a woman who claims that her multiple sclerosis is gone. Dr. Oz’s chief medical correspondent, Dr. Michael Roizen, tells us that he definitely believes “there’s something here,” and Dr. Nemeh himself proclaims that his goal is to “bridge the gap between science and spirituality.” Certainly, there is a receptive audience among Dr. Oz’s studio audience, as Dr. Oz cites a poll of his audience, which reveals that 86% of them believe in the power of faith to heal.

It’s in this segment when Dr. Oz shows Dr. Nemeh’s stack of medical records. Quite frankly, to me it looks like a pretty darned small stack. Be that as it may, Dr. Oz tells the audience that he’s had his medical staff investigate the cases and that he personally has discussed them with Dr. Roizen. That’s when the interview with Dr. Nemeh begins. Dr. Nemeh, it turns out, is a trained anesthesiologist who in addition to his faith healing activities sees patients at his office in Rocky River (a suburb of Cleveland). As the interview progressed, it became clear that Dr. Nemeh used a lot of different “alternative medicine” modalities in addition to his “electroacupuncture” (which is, of course, not really acupuncture at all, but transcutaneous electrical nerve stimulation, or TENS) and prayer services. Dr. Nemeh, of course, is also represented not just as the Brave Maverick Doctor but as the reviled Brave Maverick Doctor, with even his family disapproving of what he is doing. I can sympathize–with Dr. Nemeh’s family. If one of my siblings were a faith healer, I’d be pretty disapproving and embarrassed too.

In the next part of the segment, Dr. Oz tells the audience to judge for themselves whether Dr. Nemeh is a faith healer on the basis of the patients of Dr. Nemeh’s whose story he will tell. Of course, as an academic surgeon (which Dr. Oz was for a long time before turning to woo and, given that he is still a professor of surgery at Columbia University, technically still is even though he long ago abandoned science in favor of nonsense), Dr. Oz should know that single anecdotes say at best little or nothing and at worst mislead. The plural of “anecdote,” as we say, is not “data.” Yet anecdotes are what he provides–and then only two of them. No science. No statistics. No real detailed case reports. Not even a mention of scientific studies to be presented along with the human interest anecdotes, other than late in the segment, when he mentions the infamous intercessory prayer study that failed to find that prayer works in helping cardiac patients heal after their surgery. All we see are testimonials and utterly unconvincing cherry picked clinical test results.

First up is a woman named Cathy. Cathy is presented as having a mass in her left lung and states that she was “so sick” that she was coughing up blood. A CT scan is presented, which does show a worrisome mass in the lower lobe of the left lung. We are not informed whether Cathy is a smoker, which would have made me even more worried if I were Cathy’s physician. In any case, Cathy describes a two hour visit with Dr. Nemeh, who, she reports, used acupuncture, “infra-ray light” (whatever that is; probably she meant “infrared” light), and prayer to treat her, after which her breathing got much better. Later, a PET scan was ordered, and–miracle of miracles!–the mass was gone. The problem with this anecdote, as regular readers of this blog can probably spot right away, is that there was no tissue diagnosis. In the story, it is implied that the mass was some sort of horrible lung tumor. Yet her doctor violated the cardinal rule of oncology: He never got a tissue diagnosis; it’s unclear if he even tried.

Whatever Cathy’s pulmonologist did or didn’t do, that mass could have had any number of nonmalignant explanations, including tuberculosis, sarcoidosis, focal pneumonia, or fungal infection, to name a few. Whatever it was, if the Cathy’s physician thought it was cancer, he should have gotten a core needle biopsy. Indeed, reading between the lines, I wonder if Cathy’s doctor really thought the mass was cancer. The fact that he ordered a PET scan implied that he thought it might be (although infection can light up on PET as well), but his failure to obtain a biopsy expeditiously implies that he either wasn’t very sure or that he doubted that the mass was cancer.

All in all, it’s a somewhat confusing case, but there is no evidence whatsoever other than the post hoc ergo propter hoc fallacy that, just because Cathy got better after seeing Dr. Nemeh, it must have been Dr. Nemeh’s woo that cured her. To be fair, Dr. Oz points out the possibility that the mass might have been infectious in nature, but in reality to me he didn’t sound as though he really believed that. In fact, he came across more as playing Devil’s advocate than anything else. Unfortunately, Cathy’s doctor (Dr. Kelly) was not particularly skeptical and served up a custom-made quote that Dr. Oz read on the air, completing the picture of faith healing having cured Cathy.

Next up is a woman named Dr. Patricia Kane, who is introduced as having been diagnosed with idiopathic pulmonary fibrosis (IPF) in 1995, when she was told that she probably had less than five years to live. IPF is a disease in which the lungs develop scar tissue for reasons we don’t understand (hence the label of “idiopathic”), gradually decreasing air exchange. In Dr. Kane’s case, we are informed that she underwent a biopsy that confirmed the diagnosis. We are not really informed whether Dr. Kane has gotten better, but, as you might expect, idiopathic pulmonary fibrosis is a disease with a highly variable rate of progression that can range from a very rapid scarring of the lung with concomitant loss of lung function to slow progression that takes many years or even to long periods of time (years) with no detectable progression. Overall, the five year survival is reported to be between 30% and 50%, with this caveat:

Keep in mind that researchers have noted a considerable variation in these life expectancies based on the factors that were mentioned previously.

We are not told whether Dr. Kane had any of the factors associated with a less malignant course for IPF. I’m left to conclude that she is almost certainly a woman who is fortunate enough to be an “outlier” on the survival curve. Like all such patients who are lucky enough to be outliers and who chose “alternative” medicine, Dr. Kane underwent conventional therapy and Dr. Nemeh’s quackery, after which she did better than predicted and–of course!–attributed her much better than expected outcome to the faith healing. Again, Dr. Oz plays the “skeptic” a little bit (but only a very little bit) by challenging Dr. Kane gently with the possibility that the diagnosis was mistaken, which, while most definitely a possibility, was not the only possibility. More likely is the possibility that, as I mentioned before, Dr. Kane is fortunate enough to be an outlier.

Dr. Oz finishes this segment by interviewing Dr. Jeffrey Redinger. Remember him? He’s the same physician who was taken in by John of God, and he lays down the same sort of barrage of credulous nonsense that he did when he commented on John of God for Oprah Winfrey:

What I think at this point is that we are just not physical beings, we are also spiritual beings, physical beings need oxygen and spiritual beings need love. One research questioned I believe is whether there is a connection between love and healing? That is something that modern science is begining to tiptoe into.

Finally, in the next segment (which, unfortunately, does not appear to be on Dr. Oz’s website), we’re treated to what has to be one of the most pathetic faith healings I’ve ever seen. A woman named Mary Beth is brought up on stage. After she states that she has lower back problems that she attributes to arthritis, Dr. Nemeh does his thing. The best Mary Beth could come up with was that she felt “a little” better afterward. I don’t know about those of you who saw this episode, but I was so not impressed by this “healing” at all. Indeed, I was left scratching my head and thinking, “This is the best Dr. Nemeh could come up with?” You know that if Dr. Nemeh could come up with better cases, he would have brought them with him to Dr. Oz’s studio. For instance, where’s the paralyzed patient who said he was getting some motion back? Why wasn’t his case featured? What about the woman who claims her MS is gone? Why wasn’t she featured? It makes me wonder if the evidence for these patients’ claims is even weaker than the evidence for the “healing” of Dr. Kane or Cathy. Not that any of this stops Dr. Nemeh from proclaiming:

You don’t have to be religious, you don’t have to have faith, you can be an Atheist, what matters is we were talking before about one very important principle, the love that we have. Because the heart of God himself is Love. No you don’t have to have any faith to be healed.

Imagine how relieved I am to hear this. Strike three! Or dip Dr. Oz in a lake of lava and slap a black metal respirator suit on him, whichever metaphor you prefer. Dr. Oz is done.

I often wonder how a man as obviously intelligent and well-trained as a surgeon as Dr. Oz can fall for such utter tripe. In his case, I suspect that it’s become more about the fame, the money, and the image that has developed as “America’s doctor.” Whatever the reason, Dr. Oz’s journey to the Dark Side is complete. When Dr. Oz left Oprah, he was but the learner. Now he is the master. The master of woo. Yes, yes, I know the analogy is flawed in that it inappropriately likens Oprah to being one of the good guys (i.e., Obi-Wan Kenobi), but I just love that line.

Unfortunately, it’s not just Dr. Oz, though, who suffers from a profound lack of skepticism and critical thinking when it comes to medicine. It’s all too many physicians. After all, Cathy’s doctor apparently believed that she had been the beneficiary of some sort of miraculous healing solely on the basis of the thinnest of thin evidence. And he is a pulmonologist! This should serve as a reminder to us physicians that, unless we apply skepticism, science, and critical thinking to our practices, we are just as prone as anyone else to confusing correlation with causation, the post hoc ergo propter hoc fallacy, and, above all, an over-reliance on our own personal experience and anecdotes. Indeed, from my perspective, it is the over-reliance on personal experience and anecdotes that is most prone to leading physicians astray, and physicians have to learn how not to confuse “my clinical experience” with science. Instead of educating about this pitfall, Dr. Oz, sadly, has apparently tried to capitalize on it to promote faith healing and other forms of quackery. In my opinion of course.

Also in my opinion, I’ll try to find different metaphors next time.

During my recent stint covering the Neuro ICU I noticed for the first time a checklist posted above each patient bed. The checklist covered the steps to undergo whenever performing an invasive procedure on the patient. I was glad to see that the checklist phenomenon had penetrated my hospital, although the implementation of safety checklists is far from complete.

A recent study published in the BMJ offers support for the efficacy of using checklists to reduce complications and improve patient outcomes. This is a retrospective study looking at mortality and length of stay in Michigan area ICUs, comparing those that had implemented the Michigan Keystone ICU project (including a safety checklist for the placement of central lines) with local ICUs that had not implemented the project. They found a 10% decrease in overall mortality, but the results were not significant for length of stay. Because this was a retrospective study it was not designed to prove cause and effect, but it is highly suggestive of the efficacy of implementing such checklists.

The checklist trend represents a culture change within medicine – and a good one. This change received its greatest boost with the publication of The Checklist Manifesto by Dr. Atul Gawande. He presents a compelling case for the need and efficacy of using checklists in order to minimize error.

He argues that historically medicine has had a culture of quality control through individual excellence and training. This culture still pervades medicine. Each year, for example, I have to go through a long list of safety and other training – the standard response of the powers that be is to institute a new training and certification program for each new regulation or identified safety issue. Training is good, but increasingly there is recognition that it is not adequate.

The problem, Dr. Gawande points out, is that there are areas of our complex civilization that are too complex for mere humans to adequately master. Or you can look at it from the perspective of minimizing error. Training to deal with a complex system can only minimize error to a certain degree. There are inherent human limitations of memory, attention, and consistency that mean that error will be inevitable. In situations where minor mental errors can have catastrophic consequences (like flying planes or performing major surgery) relying on training alone is folly. In such situations the implementation of a simple checklist can significantly reduce error far below what training alone can. It is a lot easier to remember to follow the checklist than to remember each item on the checklist.

There is no question that medicine is a high stakes and complex game. While I am a strong advocate of science-based medicine, we have to recognize its limitations. The opportunities for catastrophic error in medicine are enormous – from prescribing the wrong medication or dose, to forgetting important steps in a complex procedure, to removing the wrong limb. Even minor errors or oversights can have extreme consequences.

In medicine the overarching consideration of any intervention is risk vs benefit. We only use interventions that have potential benefit that is greater than the potential risk (while also understanding that our information is probability-based and imperfect). Often our knowledge is based upon clinical trials which are highly controlled, and therefore do not have the same risk of error that is likely to exist when implemented in the “real” world outside of a clinical trial. In any case minimizing error is key to minimizing risk and optimizing the risk-benefit ratio of medical interventions.

It seems that we have pushed the limits of training. Medicine has become highly technical, specialized, and complex. While extensive training is necessary, it is no longer sufficient to minimize risk. We are now entering the age of the checklist. This is a simple procedure that can significantly improve human performance. The latest study is further evidence in support of this. A 10% reduction in mortality is highly significant.

A related phenomenon, in my opinion, is the movement toward a team approach to patient care, especially in highly complex cases. There is increasing recognition that group intelligence can vastly outperform individual intelligence, and that a group can be smarter than even its smartest member. Complex or high risk cases can benefit from a team of experts, especially with a variety of specialties, collaborating on care. This is nothing new in medicine – tumor boards and multi-disciplinary clinics have been around for decades. But there is movement toward greater reliance on teams than on individual experts.

This is related to the checklist phenomenon in that both trends represent a movement away from over-reliance on the individual and training to minimize error and maximize performance. Both recognize the crushing complexity of modern medicine, and the need to be humble before this complexity.

To broaden the context further, I think these phenomena represent increasing recognition that we need to pay attention in medicine to how our knowledge is implemented, not just to the acquisition of greater knowledge. Pushing the limits of medical knowledge is, of course, incredibly important. But we also have to pay attention to how that knowledge is disseminated, how it is received by the public, how it affects regulation, and how it is implemented by systems and by individuals. We also need better understanding of these processes – we need increased  medical meta-knowledge. We need to learn how to deal with this vast body of scientific information we are rapidly accumulating.

For a touch of the lighter side, here’s Dr. Paul Offit appearing on The Colbert Report to discuss his new book:

Looks like a win to me. I particularly like how Dr. Offit says that the question of whether vaccines cause autism has been “asked and, fortunately, answered.” Heh. That’s a shot across the bow to J.B. Handley, who, as Steve Novella has pointed out (as have I) is utterly clueless about science and how to interpret the medical literature, as he has demonstrated time and time again with his “14 Studies” nonsense. Of course, anyone who calls Handley out on his ignorance is subject to personal attack. Reporters have felt it. Steve Novella has felt his wrath. So have I. Meanwhile Handley gloats over the decline in confidence in vaccines that his organization Generation Rescue has helped foster.

Fortunately, Colbert appears to get it. I like how Colbert does a faux rejection of one of Dr. Offit’s points by pointing out that he is “ruled by fear.” I particularly like how he mentions Andrew Wakefield, but not by name (rather like Lord Voldemort), and how he asks Dr. Offit a bunch of questions based on talking points the anti-vaccine movement likes to use to frighten parents. No wonder the anti-vaccine collective at Age of Autism is going crazy, having posted (and reposted) numerous attacks old and new on Paul Offit ever since it was announced that he was going to be on The Colbert Report last night, all topped off with one by J.B. Handley himself in which he calls Dr. Offit a “blowhard liar.”

Stay classy, J.B. Stay classy.

Dr. H. Gilbert Welch has written a new book Over-diagnosed: Making People Sick in the Pursuit of Health, with co-authors Lisa Schwartz and Steven Woloshin.  It identifies a serious problem, debunks medical misconceptions and contains words of wisdom.

We are healthier, but we are increasingly being told we are sick. We are labeled with diagnoses that may not mean anything to our health. People used to go to the doctor when they were sick, and diagnoses were based on symptoms. Today diagnoses are increasingly made on the basis of detected abnormalities in people who have no symptoms and might never have developed them. Overdiagnosis constitutes one of the biggest problems in modern medicine. Welch explains why and calls for a new paradigm to correct the problem.

Where to draw the line? FDR had a BP of 200/100 at the time of his re-election in 1944 and subsequently died of a stroke with a BP of 300/190. At that time, elevated BP was not commonly recognized as a problem requiring treatment. Then studies showed that the higher the BP, the greater the risk, and now everyone diagnosed with HBP is treated. That has undoubtedly saved many lives; but for someone with only a mild elevation, the risk of heart attacks and strokes is smaller and the risk of complications from treatment becomes less acceptable. So where do you draw the line and start treatment?  When the limit of 160 systolic was dropped to 140, the new definition instantly turned 13 million people with “normal” BP into patients with hypertension. Not all of those new patients were better off with treatment. Welch gives the example of an 82 year old man who was treated for mild HBP at a level where the number needed to treat for one person to benefit (NNT) was 20; he passed out from medication side effects and declined further treatment.

Changing the Rules: We’ve changed the diagnostic thresholds for many diseases, so that people who were previously classified as normal are now diagnosed with diabetes, high cholesterol and osteoporosis. Dropping the threshold of fasting blood sugar from 140 to 126 instantly created 1.6 million new diabetics, diabetics who were less likely to develop symptoms and complications and were less likely to benefit from treatment. He tells about one of his patients who was put on blood sugar-lowering medication because of the new rules and passed out while driving and broke his neck because the medication brought his blood sugar too low.

Osteoporosis: here are the numbers for treatment of decreased bone density:

• Winners (treatment saved them from a fracture): 5%
• Treated for naught (had a fracture anyway, despite treatment): 44%
• Losers (treated but never would have had a fracture without treatment): 51%

Seeing too much: New technology allows us to detect abnormalities that would never have caused harm. In people without back pain, over 50% have bulging discs on MRI; 10% of asymptomatic people have gallstones on ultrasound. In patients without symptoms, what’s the value of knowing about these findings? In people with symptoms, such findings may lead to a false diagnosis.

10% of the general population and 7% of people under the age of 50 have findings of stroke on MRI. Whole body CT scanning finds abnormalities in 86% of asymptomatic people. The higher the resolution of your testing method, the more anomalies you will detect; but how many of them are important to know about? How will finding them affect health outcomes?

Prostate cancer: the harder you look, the more you find, and the smaller the cancers you detect, most of which would never have hurt the patient. Welch estimates that for every prostate cancer death avoided by screening, between 30 and 100 patients are harmed by unnecessary treatment.

In breast cancer, for every death prevented by mammography, 2 to 10 women are overdiagnosed and treated unnecessarily, 5 to 15 are diagnosed earlier without any effect on final outcome, 250-500 will have a false alarm and half of these will be biopsied. 999 out of 1000 women do not benefit from mammography. A study in Norway showed that screening resulted in 22% more diagnoses of invasive cancer; apparently some invasive breast cancers in the unscreened group had spontaneously regressed.

Other cancers: In an autopsy study, researchers determined that almost everyone has small thyroid cancers; so many that they could be considered “normal” findings. The US Preventive Services Task Force (USPSTF) recommended against screening for thyroid cancer, since it increases the diagnosis rate without affecting the death rate, and increases morbidity from unnecessary surgery and other treatments.

There is overdiagnosis of melanoma and lung cancer. For colon cancer and cervical cancer there is overdiagnosis of precancerous abnormalities.

The good news: We are learning that many, perhaps most, small cancers either regress or never progress. Spontaneous remissions may be far more common that we ever imagined. In one study, 14% of kidney cancers got smaller without any treatment. So we don’t really need to know if any cancer is present: we need to know if a cancer is present that is likely to progress and harm the patient. And so far we have no way of distinguishing which these are.

Incidentalomas are nodules or other unexpected findings noticed on imaging studies, often in body parts adjacent to the area being studied. About half of virtual colonosopies detect abnormalities outside the colon. More than 99% of the time, these are not cancers and not important to know about; but they lead to anxiety, further studies, surgeries, and complications. Protocols are being developed to follow incidentalomas suggestive of kidney and lung cancers over time rather than immediately pursuing diagnosis.

Routine electronic fetal monitoring has minuscule benefits and results in many more C-sections.

Vascular screenings: The Lifeline company and other commercial ventures offer tests direct to the public, tests that the USPSTF doesn’t recommend and that have not been shown to benefit those screened.

Genetic screening. These tests are not done for symptoms, and do not even detect signs of early disease, but just estimate future risks using inadequate data. Welch reminds us that genetics is not destiny and abnormal genes do not equal disease. The predictive value of these tests is small, and we seldom know what to do about the risk after we identify it. Low risk for a condition doesn’t mean you can’t get it, and everyone is at high risk of something.

A paradigm shift is needed, but it will be difficult to achieve for many reasons:

• It is hard to ignore information.
• Most people believe the more information, the better.
• Accepted wisdom and common sense are hard to overturn.
• Most people are convinced that it is always in people’s interest to detect health problems early, even though the data say otherwise.
• There is a common belief that early detection is cost-effective, even though the data show it actually ends up costing more.
• We find it hard to tolerate uncertainty.
• Commercial interests benefit from screening and overdiagnosis.
• Doctors fear being sued if they omit tests.
• Anecdotes about lives saved are emotionally persuasive.

We are easily impressed by anecdotes from people who believe their lives were saved by early detection; but we don’t hear anecdotes from people who were harmed by a diagnosis of a condition that would never have hurt them, mainly because we have no way of knowing which ones they were. I am a case in point: I had a suspicious mammogram and an excisional biopsy that removed a lobular carcinoma in situ. That is not really a cancer, but more like a risk factor for cancer. Did my surgery remove a part of my breast that would have eventually developed invasive cancer and killed me, or did it uselessly remove a harmless chunk of tissue? Did it save my life or just mutilate me? I will never know.

What’s the solution? Maintaining a healthy skepticism about early diagnosis. Informed consent for screening tests, based on accurate information. Resisting over-simplified hype about the benefits of screening. Putting our efforts into prevention (exercise, smoking cessation, healthy diet, etc.) rather than pursuing early detection. Pursuing health without paying too much attention to it and without developing anxieties about it. Welch argues for not even mentioning incidentalomas on imaging reports, but I think radiologists and lawyers would object to that strategy. He says

Severe abnormalities warrant action because net benefit is likely. But the best strategy for mild ones may be to leave well enough alone, otherwise net harm is likely. In fact, it may be better not to look for them in the first place…An overdiagnosed patient cannot benefit from treatment… [but] can only be harmed.

He doesn’t offer prescriptions. He recognizes that different individuals will assess the risk/benefit ratio differently; based on the same data, some will choose to be screened and some won’t. But they deserve accurate information to base their decisions on, and this book offers a lot of good data and thought-provoking analysis.

I couldn’t help but like this book, since it says many of the same things I have been saying about screening tests , colonoscopy, osteoporosis treatment, PSA tests , not always treating , ultrasound testing , overuse of CT angiograms, genetic testing in general and in specific situations, and the pitfalls of diagnostic tests. It explains complicated concepts like lead-time bias in simple terms and spices the story with patient anecdotes. I found it a bit repetitive but that is probably an asset for driving the message home to a general audience. Both patients and doctors would benefit from reading this book and thinking about the issues it raises.

I must admit that I have a bit of a love-hate relationship with breast implants. On the one hand, as a breast cancer surgeon, I see them as a major benefit to my patients who are unfortunate enough to require mastectomy in order to control their disease. The armamentarium of techniques for reconstructing breasts after mastectomy generally falls into one of two categories, either various form of muscle flaps or breast implants. However, some women are, for various reasons, not eligible for various muscle flap reconstructions. That leaves either breast implants–or nothing. Certainly, some women are perfectly fine with no reconstruction after mastectomy, but many, if not most, women are not. For these women, it would be difficult to overstate how much of a boon to body image and self-esteem reconstruction can be, particularly given how much better at it plastic surgeons have become over the last couple of decades.

On the other hand, breast implants make my life as a breast cancer surgeon more difficult for a variety of reasons. First, they tend to make mammography more difficult by obscuring part of the breast, thus decreasing the sensitivity of mammography. Good mammography facilities can get around this to some extent by using various displacement techniques, but it takes some effort, and it doesn’t completely correct the problems that implants cause for mammographic screening. Moreover, when a woman who has had implants placed for cosmetic reasons comes to see me for a breast mass or an abnormal mammogram, the presence of the implants can complicate treatment decisions. If the abnormality or mass is close to the implant, we worry about rupturing it in the process, particularly if the implant is not below the pectoralis major muscle. Even when the implant is subpectoral, the muscle overlying it frequently ends up being so stretched out that the muscle in essence forms part of the capsule around the implant and ends up being a lot thinner than you might expect. Let me tell you, my anal sphincter tone is always much tighter when operating near an implant, particularly a silicone implant. True, I’m perfectly capable of removing an implant if it’s accidentally ruptured, but such an outcome is not desirable, particularly with silicone implants, where cleaning up the leaking silicone can be difficult.

It doesn’t help that silicone breast implants have been the subject of controversy since the late 1980s and early 1990s, when thousands of women with silicone implants reported a variety of ailments, including autoimmune disease and a variety of other systemic illnesses. These reports led to a rash of lawsuits and, ultimately, the banning of silicone breast implants for general use in 1992. After that, silicone breast implants were only permitted in women requiring breast reconstruction or women enrolled in clinical trials studying breast implants. This ban was partially lifted in 2006, as evidence accumulated that the claims of autoimmune diseases and increased cancer risk due to silicone breast implants were not supported by clinical and scientific evidence and two products made by Allergan Corp. (formerly Inamed Corp.) and Mentor Corp. Not surprisingly, given that the furor over silicone breast implants as a cause of autoimmune and other systemic diseases is based on about as much solid scientific evidence as the antivaccine furor over vaccines as a cause of the “autism epidemic,” there was widespread criticism of this decision. Even now, it is not difficult to find articles about breast implants with titles like Breast Implants: America’s Silent Epidemic and websites like the Humantics Foundation and Toxic Breast Implants . I do note, however, that the number of such sites and articles does appear to be declining and, at least to my impression, seems to have decreased markedly over the last 10 years or so.

Having reviewed the literature and found evidence for a link between silicone breast implants and the systemic diseases attributed to them to be incredibly weak at best, I had little problem with the FDA’s decision. Actually, the only thing I had a problem with at the time, my opinions of how breast implants interfere with breast cancer detection and treatment notwithstanding, is that the FDA was probably being more cautious than the evidence warranted after 14 years.

Was I wrong?

Breast Cancer and anaplastic large cell lymphoma (ALCL): The FDA steps in

I ask this question because last week there was a widely reported story about a warning that the FDA issued regarding breast implants. Indeed, on Wednesday, our press people were circulating copies of the advisory and asking if any of us were available to comment to the press before the evening news deadlines. Unfortunately (or fortunately, depending on your point of view), I was holed up for our NCI site visit rehearsal and thus in essence unavailable. So it was that the national media missed its opportunity to hear me opine my wisdom on the matter to a breathlessly waiting nation. Talk about dodging a proverbial bullet (our nation, that is). Be that as it may, this FDA advisory led to stories in the media like this one by ABC News, FDA Reports Link Between Breast Implants and a Rare Cancer:

The FDA advisory states:

After an intensive review of known cases of a rare form of cancer in breast implant recipients, the Food and Drug Administration says women with implants may have a very small, but increased risk of developing anaplastic large cell lymphoma, or ALCL.

FDA scientists reached that conclusion after examining scientific literature that focused on cases of ALCL in 34 women with breast implants, as well as information from agency reports, international regulatory agencies, scientific experts, and breast implant manufacturers.

But with an estimated five to 10 million breast implant recipients worldwide, agency experts say the known ALCL cases are too few to say conclusively that breast implants cause the disease. FDA believes there are about 60 of these ALCL cases worldwide, though that number is difficult to verify because not all of them were chronicled in scientific publications and some reports may have been duplicated.

This is the sort of epidemiological question that drives physicians and scientists crazy. The reason is quite simple. ALCL is a rare type of non-Hodgkin’s lymphoma (NHL). Indeed, it is classified as a “rare disease,” which for purposes of U.S. policy is defined as affecting less than 200,000 Americans. In actuality, ALCL affects far fewer Americans than that. According to the Surveillance, Epidemiology, and End Results (SEER) Program of the National Cancer Institute (NCI), approximately 1 in 500,000 women is diagnosed with ALCL in the the U.S. every year. ALCL of the breast is even more rare, with 3 in 100 million women per year being diagnosed with the disease. Not surprisingly, that means it’s incredibly hard to get enough patient numbers to make firm conclusions regarding whether the risk of ALCL is truly higher in women with breast implants, and the FDA report, Anaplastic Large Cell Lymphoma (ALCL) In Women with Breast Implants: Preliminary FDA Findings and Analyses, reflects this uncertainty.

Reading the FDA’s report, I was struck by how little there evidence is one way or the other because of the relative rarity of the disease. Basically, the evidence portion of the FDA report concentrates on case studies and the three existing studies that tried to determine whether there is an association between breast implants and ALCL. Given that the report strikes me as being pretty accessible to the lay person, I recommend reading it, because it reveals a careful sifting of the thin gruel of evidence and how the FDA came to its decision to issue this warning. I’ll try to summarize its 21 pages for you and give you my take on the studies used to justify the warning, but go to the full report for details.

Breast implants and ALCL: The evidence

The FDA performed a review of the scientific literature. This included a search of PubMed, Embase, Web of Science, Cambridge Scientific Abstracts (CSA), EBSCO, and BIOSIS for published papers and abstracts about ALCL and breast implants. After duplicates were accounted for, the FDA found that the entire world scientific literature has reports of 34 women with breast implants who were diagnosed with ALCL of the breast. As pointed out above, the number might be as high as 60, as is described in the report:

In a thorough review of scientific literature published from January 1997 through May 2010, the FDA identified 34 unique cases of ALCL in women with breast implants throughout the world. The FDA’s adverse event reporting systems also contain 17 reports of ALCL in women with breast implants. Additional cases have been identified through the FDA’s contact with other regulatory authorities, scientific experts, and breast implant manufacturers. In total, the FDA is aware of approximately 60 case reports of ALCL in women with breast implants worldwide. The exact number is difficult to verify because reports from regulatory agencies and scientific experts often duplicate those found in the scientific literature.

It’s estimated that there are between 5 and 10 million women in the world with breast implants. Given these numbers, the number of women with breast implants who have developed ALCL of the breast is higher than would be expected from SEER data alone. Moreover, another thread of association that is concerning derives from the spatial pattern noticed in these case reports:

Of the 34 cases, the median time from breast implant placement to ALCL diagnosis was 8 years, with a range from 1 year to 23 years. Most patients were diagnosed when they sought medical treatment for implant-related symptoms such as persistent seromas, capsular contractures, or peri-implant masses warranting breast implant revision operations. In each case, lymphoma cells were found in the effusion fluid (seroma) surrounding the implant, in the fibrous capsule, or within a peri-implant mass. Typically, there was no invasion beyond the fibrous capsule into the breast parenchyma.

It should also be noted that it couldn’t be determined whether there was a higher risk of ALCL that could be attributed to silicone versus saline implants, as twenty-four had silicone implants, seven had saline, and the type of implants was unknown. Similarly, there didn’t appear to be a correlation between the indication for implant placement and the risk of ALCL. Of the 34 cases, eleven patients had their implants placed for breast reconstruction, nineteen patients received implants for breast augmentation, and in four cases no reason for placement of the implants was reported.

Unfortunately, these case reports are not particularly illuminating.

Given that, perhaps the epidemiology will be more revealing. Except that it isn’t. There are only three studies cited looking at whether there is an association between the presence of breast implants and ALCL of the breast. There were no prospective cohort studies. Indeed, all three studies were in essence retrospective studies. Of these, only one of them was designed to look specifically at a correlation between breast implants and ALCL of the breast, rather than observations of non-Hodgkin’s lymphoma and other cancers in women with breast implants. This study (de Jong et al, 2008) is an individually matched case-control study that mined a nationwide population cancer database from the Netherlands. Since 1971, all reports on cytological and pathological diagnoses generated by every pathology department in the Netherlands have been stored in a central database (PALGA, Pathologisch Anatomisch Landelijk Geautomatiseerd Archief).

Going to show that lymphoma of the breast is a rare entity, between 1990 and 2006, only 429 cases of histologically proven lymphoma of the breast were found, and, of the 389 women eleven had a diagnosis of ALCL. Using these cases as the basis, de Jong et al performed an individually matched case-control study thusly:

Subsequently, we performed an individually matched case-control study, nested in the same cohort of 389 female patients. For each case patient with ALCL in the breast, we attempted to select 3 to 7 controls with other lymphomas in the breast, matched on age at diagnosis (±5 years) and year of diagnosis (±2 years). For all 47 potential controls, we obtained pathology reports. Furthermore, for all cases and controls, we sent a standardized questionnaire to the treating physician to obtain information on medical history, including previous malignancies, staging results, and presence of a breast prosthesis, including mammography results.

Conditional logistic regression analysis was performed to estimate the odds ratio (OR) of ALCL associated with breast prosthesis, using EGRET for Windows, 1999 (CYTEL Inc, Cambridge, Massachusetts).21? The OR was used as a valid risk estimate of relative risk and is therefore referred to as such. An estimate for absolute risk was made based on breast prosthesis sales figures for 1999 to extrapolate the number of women with breast prostheses.

Based on this analysis, de Jong et al estimated the odds ratio of ALCL associated with breast implants to be 18.2 (95% CI 2.1-156.8). What this means is that the odds of having a breast implant were 18.2 times higher in ALCL patients than in the control lymphoma patients. Personally, I have a few problems with this analysis. First, the matching was done on only two criteria, age and year of diagnosis. Although there was no statistically significant difference in age between the groups, there’s no way of knowing if there were any confounding factors that were associated with ALCL of the breast. The numbers are just too small. Consequently, it’s hard to say much about this series except that it is suggestive that there is an elevated risk of ALCL due to breast implants. As the authors themselves say:

Although an 18-fold increased odds for the development of a specific lymphoma in the breast may cause significant concern among women with breast prostheses, it should be realized that the absolute risk remains very low due to the exceedingly rare occurrence of ALCL in the population (estimated incidence at all sites 0.1/100,000 per year).

Which is about one in a million. Even if the estimate made by de Jong et al is accurate, that would put the risk at around 18 in a million.

As for the other two studies, they’re not exactly studies. One (Brinton et al) is a systematic review of the literature looking for evidence of an association between breast implants and cancers at other sites. Brinton et al concluded that breast lymphomas in women with breast implants tend to be associated with the periprosthetic capsule, or in proximity to the implant. Moreover, in the general population, breast lymphomas tend to be a rare entity and most are of B-cell origin. In contrast, breast lymphomas in women with implants tend to be of T-cell origin. The second, Lipworth et al, examined five long term studies of women with breast implants including 43,000 women to assess the risk of lymphoma in these women. This review actually found that there was a slightly decreased risk of lymphoma in women with breast implants, but, as the FDA report noted, it had a at least two weaknesses. First, all the studies began following women more than 35 years before the study, and the entity of ALCL was not defined pathologically until 1985. Second, the number of women studied was inadequate to rule out a rare relationship between breast implants and ALCL.

As you can see, the evidence for a link between ALCL and breast implants is fairly sparse. Of the evidence, de Jong et al is probably the most suggestive, but even it is relatively weak, at least based on numbers alone. However, another piece of evidence comes from the characteristics of implant-associated lymphomas. The FDA report includes a good illustration to show where the lymphoma cells were typically found. In all cases, they were either found in fluid surrounding the implant (it’s not uncommon for implants to have a fluid collection surrounding them) or in the connective tissue capsule that develops around many breast implants:

Add this to the seeming statistical association between breast implants and ALCL, and there might just be something there. It’s not possible to conclude with any degree of certainty that there is such a risk right now; there are simply too few cases and ALCL is too rare, both in the general population and in women with breast implants.

What should be done?

Despite the controversy over the years over breast implants, particularly silicone breast implants, there has been no convincing evidence of a link between systemic diseases, such as autoimmune diseases or cancer. Indeed, since the 1990s, there have been at least a dozen comprehensive systematic reviews looking at a potential link between silicone breast implants and systemic diseases (conveniently listed at Wikipedia), none of which have found convincing evidence for a link. In 2006, Brinton et al found an increased risk of death from lung cancer and suicide in women with implants, but these risks were attributed to increased smoking and psychiatric disorders in women who have implants placed.

This report from the FDA suggests that there might be an increased risk of a rare cancer in women with breast implants, but the numbers are so low that it’s difficult to conclude anything with much certainty, which is why the FDA concludes:

1. There is a possible association between breast implants and ALCL.
2. At this time, it is not possible to identify a specific type of implant associated with a lower or higher risk of ALCL.
3. There is uncertainty about the true cause of ALCL in women with breast implants.

Adding:

Based on available information, it is not possible to confirm with statistical certainty that breast implants cause ALCL. Because ALCL is so rare, even in breast implant patients, a definitive study would need to collect data on hundreds of thousands of women for more than 10 years. Even then, causality may not be conclusively established.

These are reasonable conclusions based on the current state of the evidence, which is inconclusive at best, weak at worst. Given the high degree of uncertainty, what the FDA has done is not entirely unreasonable, although one could argue that it’s a tad alarmist. Basically, the FDA wants clinicians to consider ALCL in women with implants who have persistent seromas (fluid collections) around their implants, recommending that seroma fluid be sent for cytological analysis to rule out lymphoma. In addition, the FDA recommends that any confirmed cases of ALCL associated with implants be reported and is establishing a registry in collaboration with the American Society of Plastic Surgeons to track cases of implant-associated lymphoma. Even this might not be able to detect or confirm a link between implants and ALCL, given the rarity of the disease, but it is a start.

Even accepting the most pessimistic assumption, namely that there really is a significantly elevated risk of ALCL in women with breast implants due to the implants, which has been suggested but not by any means established, this risk, if it exists, should be put into perspective. For example, it should also be noted that, based on what we know, in women who choose implants for reconstruction after breast cancer surgery, the risk of recurrence of their breast cancer is orders of magnitude greater than any theoretical risk of ALCL due to implants. Indeed, in women who have never had cancer and choose implants for breast augmentation, the risk of developing breast cancer is also orders of magnitude higher than of developing ALCL. There is no evidence that implants increase the risk of breast cancer or breast cancer recurrence after breast cancer surgery.

In fact, the most significant risk due to breast implants is not the risk of systemic diseases, such as autoimmune diseases or cancer. Far more significant is the rate of local complications, such as capsular contracture or implant rupture. Due to such complications, many women with implants require reoperation. Indeed, reoperation rates have been estimated to be as low as 3% after seven years to as high as 20% over three years. These are by far the most significant risks due to breast implants.

While I am not a big fan of elective breast augmentation or cosmetic surgery in general (that’s just me, I guess, in that my assessment of the risk-benefit ratio of having a surgeon cut into me to make me better looking probably won’t come down on the side of surgery unless I eer suffer some sort of traumatic injury that leaves me disfigured enough to require reconstructive surgery), if there is informed consent in which the risks of breast augmentation are clearly explained based on science and clinical evidence and not inflated by the addition of claimed risks that are not supported by science, women should be free to choose implants if they so desire. From my perspective as a clinician, more importantly I strongly believe that, for women with breast cancer, implants are an important option to be made available for reconstruction after mastectomy. In particular, given how small the risk would be even if it is confirmed, this new information from the FDA regarding breast implants and ALCL does not change that.

I was interviewed for Birmingham Skeptics and you can hear the result at

http://brumskeptics.blogspot.com/2011/01/podcast-posting-interview-with-mark.html.

It is bad enough listening to myself as I correct my podcasts, so I have no idea how good the interview is; it was fun at the time.   And the picture makes me look fatter than I am. Oh well.

For those who are new to the blog, I am nobody from nowhere. I am a clinician, taking care of patients with infectious diseases at several hospitals in the Portland area. I am not part of an academic center (although we are affiliated with OHSU and have a medicine residency program). I have not done any research since I was a fellow, 20 years ago. I was an excellent example of the Peter Principal; there was no bench experiment that I could not screw up.

My principal weapon in patient care is the medical literature, accessed throughout the day thanks to Google and PubMed. The medical literature is enormous. There are more than 21,000,000 articles referenced on Pubmed, over a million if the search term ‘infection’ is used, with 45,000 last year.

I probably read as much of the ID literature as any specialist. Preparing for my Puscast podcast, I skim several hundred titles every two weeks, usually select around 80 references of interest and read most of them with varying degrees of depth. Yet I am still sipping at a fire hose of information

The old definition of a specialist is someone who knows more and more about less and less until they everything about nothing. I often feel I know less and less about more and more until someday I will know nothing about everything. Yet I am considered knowledgeable by the American Board of Internal Medicine (ABIM), who wasted huge amounts of my time, a serious chunk of my cash, and who have declared, after years of testing, that I am recertified in my specialty. I am still Board Certified, but the nearly pointless exercise has left me certified bored. But I can rant for hours on Bored Certification and how out of touch with the practice of medicine the ABIM is.

My concept of an expert is a combination of experience and understanding of the literature. I used to say mastery of the literature, but no one can master a beast that large; I am just riding on the Great A’Tuin of medical writings. Experience comes with time, and I have read that it takes 10 years to become competent in a field. Whether true or not, it matches my experience. I remember as a resident reading notes on patients I had cared for as an intern, and being appalled with what an ignorant dufus I was. In my first year of practice I had a patient who died of miliary tuberculosis, and the diagnosis was, unfortunately, made at autopsy. It was an atypical manifestation of a rare (in the US) disease. About a decade later the case was presented as an unknown to a visiting professor; I had completely forgotten the case, but I piped up from the audience to pontificate on how this had to be miliary Tb. Afterwards I was shown the chart and nice documentation as to how clueless I had been a decade earlier. When it comes to being a diagnostician, there is not substitute for experience.

When it comes to treatment? That is where I tell the residents that the three most dangerous words in medicine are ‘In. My. Experience.’ You cannot trust experience when deciding on therapy, especially for relatively unusual diseases. Sometimes I will ask a doc why they use a given antibiotic, usually in a situation where it is being used in a way that is, shall we say, old fashioned. Often the response is “I like it”‘ as if the choice of a drug is like choosing a beer.

I rely on the literature — such as it is, and limited by my lack of an Ethernet jack in my brain — in deciding the best course of therapy for a patient. The literature is always unsatisfactory. That has always been known. Even with the best studies, there is always the issue of wondering if the literature applies to your patient and their particular co-morbidities, and, perhaps, genetics. As an example, it is becoming evident that the literature on the presentation and treatment of Cryptococcus, which is based on the experience with C. neoformans, is not applicable to C. gattii, a new strain of the fungus in the NW. So how to use a literature that may not be totally relevant to my local conditions? I wing it. It is an educated and experienced winging, but winging it I do.

Given the breadth and depth of the literature, it is nice to have systematic reviews, meta-analysis, and guidelines. As a practicing physician, I find them helpful as they provide an overarching understanding, a conceptual framework, for understanding a disease or a treatment. They are the Reader’s Digest abridged version of a topic, and the references are invaluable. Usually most of the relevant literature is collected in these reviews and make it easier, especially in the era of the Googles and on-line references, to find the original literature.

All three have their flaws, and if you are well versed in a field, you recognize the issues and try and compensate.

As was noted in the recent Archives, the literature to support the recommendations of the Infectious Disease Society America are not necessarily based on the best of evidence. Really? I’m shocked. Next up, water is wet, fire is hot, and the Archives confirms the obvious.

Results In the 41 analyzed guidelines, 4218 individual recommendations were found and tabulated. Fourteen percent of the recommendations were classified as level I, 31% as level II, and 55% as level III evidence. Among class A recommendations (good evidence for support), 23% were level I (1 randomized controlled trial) and 37% were based on expert opinion only (level III). Updated guidelines expanded the absolute number of individual recommendations substantially. However, few were due to a sizable increase in level I evidence; most additional recommendations had level II and III evidence.

Conclusions: More than half of the current recommendations of the IDSA are based on level III evidence only. Until more data from well-designed controlled clinical trials become available, physicians should remain cautious when using current guidelines as the sole source guiding patient care decisions.

Big duh. Anyone who is a specialist understands the weaknesses in all guidelines, but we also understand their importance. When I was a fellow, one of my attending was, and still is, one of the foremost experts in the US on Candida, and anothers areas of expertise is S. aureus infections and endocarditis.

Both have spent a career thinking deeply on their respective areas of expertise. You learn that while no one is perfect, the breadth and depth of their knowledge and experience gives their recommendations extra weight. Who would you want at the controls of your plane in a unexpected and unusual weather conditions? An experienced pilot, or someone who spent a few days on the X-Plane simulator? The same with all the guidelines. When someone with a lifetimes of work in a field helps write a guideline, you pay attention to their expertise. You know the recommendations are not necessarily right, but odds are their opinions are better than mine, just as my opinion is usually better than a hospitalist, as least as far as infections are concerned. With residents, I try make a point of differentiating when my recommendation is no better than the next doc, and when my recommendation is the Truth, big T, and based on the best understanding of the literature at the moment.

This attitude, trusting authority, held by many in medicine, goes against the University of Google approach where a day of searching and a quick misreading of the abstracts renders everyone an expert. I wonder if other fields are plagued with these quick pseudo-experts. Law is, when the accused attempt to defend themselves.

I certainly would be in favor of more money being spent on infectious disease research, and, one hopes, infectious disease doctors. In a perfect world, every disease would be subjected to careful, extensive clinical trials and I would know, for example, the best therapy for invasive Aspergillus pneumonia in a neutropenic leukemia patient. Until that time, I am, in part, going to rely on the guidelines written by those who have spent a career thinking about the diseases I have to treat. To quote Dr Powers,

“Guidelines may provide a starting point for searching for information, but they are not the finish line…Evaluating evidence is about assessing probability,” Dr. Powers commented in a news release. “Perhaps the main point we should take from the studies on quality of evidence is to be wary of falling into the trap of ‘cookbook medicine,’” Dr. Powers continues. “Although the evidence and recommendations in guidelines may change across time, providers will always have a need to know how to think about clinical problems, not just what to think.”

I was struck by a recent Medscape headline:

Cochrane Review Stirs Controversy Over Statins in Primary Prevention

Having been irritated of late by Cochrane reviews in my area of expertise, I clicked the link. The first three paragraphs are

A new Cochrane review has provoked controversy by concluding that there is not enough evidence to recommend the widespread use of statins in the primary prevention of heart disease.

The authors of the new Cochrane meta-analysis, led by Dr Fiona Taylor (London School of Hygiene and Tropical Medicine, UK), issued a press release questioning the benefit of statins in primary prevention and suggesting that the previous data showing benefit may have been biased by industry-funded studies. This has led to headlines in many UK newspapers saying that the drugs are being overused and that millions of people are needlessly exposing themselves to potential side effects.

This has angered researchers who have conducted other large statin meta-analyses, who say the drugs are beneficial, even in the lowest-risk individuals, and their risk of side effects is negligible. They maintain that the Cochrane reviewers have misrepresented the data, which they say could have serious negative consequences for many patients currently taking these agents.

Newsweek and The Atlantic both refer to the Cochrane review as a “study.” A review is not what I would consider a study, usually synonymous with a clinical trial. The use of the term makes it sound like the Cochrane folks were doing a clinical trial, patients being randomized to one treatment or another. My sloppy non scientific poll of people (all people in the medical field, but that is who I have contact with) suggests the no one considers a review of clinical trials to be a study. A review of a novel is not the same as writing a novel.

Sloppy and potentially misleading language from major news outlets. What a surprise.

I have always liked meta-analysis for the same reason I like guidelines: they provide an overarching conceptual framework for understanding a topic. But only a fool would make clinical decisions based upon a meta-analysis. Yet, meta-analysis seem to be creeping to the top of the list of the clinical information rankings to be believed.

There are issues with meta-analysis.

The studies included in a meta-analysis are often of suboptimal quality. Many spend time bemoaning the lack of quality studies they are about to stuff into their study grinder. Then, despite knowing that the input is poor quality, the go ahead and make a sausage. The theory, as I said last week, is that if you collect many individual cow pies into one big pile, the manure transmogrifies into gold. I still think it as a case of GIGO: Garbage In, Garbage Out.

It has always been my understanding that a meta-analysis was used in lieu of a quality clinical trial. Once you had a few high quality studies, you could ignore the conclusions of a meta-analysis.

Evaluations of the validity of the conclusions of meta-analysis have demonstrated that the results of a meta-analysis usually fail to predict the results of future good clinical trials. The JREF million is safe from the Cochrane, I suppose. Their conclusions are no more reliable than the studies they collect and are no more valid than the rest of the medical literature.

We identified 12 large randomized, controlled trials and 19 meta-analyses addressing the same questions. For a total of 40 primary and secondary outcomes, agreement between the meta-analyses and the large clinical trials was only fair (kappa= 0.35; 95 percent confidence interval, 0.06 to 0.64). The positive predictive value of the meta-analyses was 68 percent, and the negative predictive value 67 percent. However, the difference in point estimates between the randomized trials and the meta-analyses was statistically significant for only 5 of the 40 comparisons (12 percent). Furthermore, in each case of disagreement a statistically significant effect of treatment was found by one method, whereas no statistically significant effect was found by the other.

Once there was a quality definitive trial or three, the meta-analysis becomes, I thought, moot. A quality clinical trial trumps the meta. I guess. I am not so certain that is the attitude anymore given the freak-out in the media about Cochrane and statins.

It seems that the producers of meta-analysis have characteristics like the March of Dimes. Polio was conquered, but rather than folding up their tents and stealing away, they continue to march. That may be a good thing too, as there could be a polio resurgence if some anti-vaccine wackaloons have their way.

If there is a definitive trial, rather than declaring the question settled, the new, perhaps higher-quality, study is folded in with the prior studies and a new meta-analysis is generated. But newer studies are diluted by the older, less robust trials, so the more reliable results are lost in the wash. The best drowned in a sea of mediocrity.

For example, I see no need for a meta-analysis on the efficacy of Echinacea. The last several trials, combined with basic science/prior probability, provides sufficient evidence to conclude Echinacea does not work. Good trials win. Ha.

As a practicing specialist, no matter how much I read, I rely in part on guidelines, meta-analyses and systematic reviews as nice overviews to be used as flawed stopgaps awaiting large high quality clinical trials, that, like Godot, may never come.

I have sick patients who need treatment. I need to know what to do. I have to fight the battles with the weapons I have. I have the medical literature and I am not afraid to use it.

One cannot play charades forever.

European veterinary groups have long been more skeptical about “alternative” veterinary practices than their American counterparts. For example, the European Board of Veterinary Specialties refuses to grant continuing education credits for non-scientific endeavors attempting to masquerade as a way to improve one’s clinical knowledge, and the practice of veterinary homeopathy is forbidden in Sweden.

Now comes good news (for pets, and pet owners), out of the UK. In an effort to improve animal health and welfare, the Veterinary Medicines Directorate’s (VMD) has targeted “alternative” remedies, which, of course, pose both real and potential dangers to pets. The VMD is the body responsible for the authorization of veterinary medicinal products in the United Kingdom.

Specifically, the VMD is targeting a number of unauthorized products that lack scientific proof of effectiveness, including:

• homeopathic “nosodes” (substances that are the homeopathic equivalent of vaccination, with the notable exception being that they don’t work)
• various herbal products
• “neutraceuticals,” the cleverly coined combination of “nutrition” and “pharmaceutical,” which really aren’t either, a fact which, of course, hasn’t dented their popularity, nor dampened their claims (including improved mental ability in pets)
• Herbal deworming products, which may claim to irritate the bowel and make it less inviting for parasites (an absurd claim)

Giving a pet an ineffective remedy invites direct harm, if a sick animal is treated with an ineffective remedy, as well as indirect harm, if an effective therapy is avoided in favor of the heavily advertised ineffective one.

Director of Operations of the VMD, John FitzGerald, said, “Animal owners have a right to know if a product does what it claims. The products claim to treat diseases which can cause serious welfare problems and in some circumstances kill animals if not properly treated. So in some cases owners are giving remedies to their pets which don’t treat the problem.” British Veterinary Association President Harvey Locke, in support of the VMD, noted, “As veterinary surgeons we rely on the use of safe, effective and quality medicines for the health and welfare of animals under our care – and there must always be sound scientific evidence to back up medicinal claims made by the manufacturer of any product.”

The VMD intends to contact manufacturers of “alternative” products to make sure that they are safe, and provide the claimed benefits. If they don’t (which they won’t), the VMD will make the manufacturers rebrand the products so that consumers will know that the products are not medicinal.

Now one might legitimately ask, “Why don’t the US veterinary authorities and organizations take some action such as this?” Well, in my opinion, veterinary authorities are more interested in getting animals to be treated by veterinarians than they are in the particular remedies that are being used. So far, in the US, it’s been a triumph of economics over science. How long that stance holds up, particularly in light of the legitimate strides at curbing non-scientific practices in other countries, remains to be seen.

A recent Cochrane review of the use of cholesterol-lowering statin drugs in primary prevention has sparked some controversy.  The controversy is not so much over what the data says, but in what conclusions to draw from the data.

Statin drugs have been surrounded by controversy for a number of reasons. On the one hand they demonstrably lower cholesterol, and the evidence has shown that they also reduce the incidence of heart attacks and strokes. The data on whether or not they reduce mortality has been less clear, although this latest data actually supports that claim. However, statins have also been blockbuster drugs for pharmaceutical companies and this has spawned concerns (some might say paranoia) that drug companies are pushing billions of dollars worth of marginally effective drugs onto the public.

So are statins a savior or a scam? Life does not always provide nice clean answers to such simple dichotomies. The evidence clearly shows that statins work and are safe. However, pharmaceutical companies do like to present their data in the best light possible, and they need to be watched closely for this. The recent review does call them on some practices that might tend to exaggerate the utility of statins. Finally, the real question comes down to – where should we draw the line in terms of cost-benefit of a preventive measure like statins.

Let’s look as this recent review of the data to see what it actually shows.

First, for context, this Cochrane review looked specifically at statins for primary prevention – prevention of vascular events (mainly heart attacks, strokes, and overall mortality) in those who are at low risk for heart disease and who have not already had any vascular event. The evidence for statins for secondary prevention, after a heart attack, is more robust – decreasing risk of a second heart attack by about one-third. This makes sense, and is generally what we see. The higher the risk of disease the greater the potential benefit for any preventive measure, and the easier it is to measure this benefit in clinical trials.

Further, as the risk of the disease becomes smaller, the risk-benefit ratio and cost-benefit ratio of preventive measures goes down. At some point the side effects from the treatment become greater than the risk of the disease being prevented. Generally clinical trials divide risk into two broad categories – primary prevention and secondary prevention. However, in reality there is a spectrum of risk. A person without a history of a vascular event may still be at high risk if they have a lot of risk factors – hypertension, age, high cholesterol, diabetes, and smoking. And of course, since statins are cholesterol lowering agents, high cholesterol at baseline is a reasonable marker for the potential of benefit from statins.

Statins also have to be compared to other measures – like diet and exercise – for relative effectiveness and cost-effectiveness. No one doubts that it would be best if every patient had a healthy diet and weight and exercised regularly. Some argue that statins should be reserved for those who fail these lifestyle interventions, or who have genetically high cholesterol refractory to diet and exercise. The reality is that it is very difficult to get individual patients to change their behavior. In fact, a recent Cochrane review concluded:

Interventions using counselling and education aimed at behaviour change do not reduce total or CHD mortality or clinical events in general populations but may be effective in reducing mortality in high-risk hypertensive and diabetic populations. Risk factor declines were modest but owing to marked unexplained heterogeneity between trials, the pooled estimates are of dubious validity. Evidence suggests that health promotion interventions have limited use in general populations.

This is not very encouraging. Clearly we need to work on societal interventions and improving patient interventions to achieve a healthier lifestyle as a society. But also it is clear that lifestyle intervention is not a quick or easy fix, and so there will continue to be a role for medical intervention in vascular prevention.

Statins for Primary Prevention

When the Cochrane reviewers looked at the evidence for primary prevention they found that many trials included patients at high risk, or did not measure LDL levels. Essentially they felt that the data was contaminated in such a way as to exaggerate the benefit for primary prevention. Their review sought to correct those biases. They reviewed the data from 14 trials involving 34 272 patients. What they found was that total mortality had a relative risk reduction of 17%, risk of heart attacks was reduced by 28%, and strokes by 22%. In low risk patients the number needed to treat in order to prevent one death per year was 1000. The review also did not show any additional adverse events in those treated vs placebo groups.

The authors do not challenge the legitimacy of these results. The data is fairly robust – there is a reduction in risk of death and vascular events from statins in primary prevention. Study author, Dr. Shah Ebrahim, is quoted by Heartwire as saying:

“If you look at the hard end points of all deaths and coronary deaths, the effects are consistent with both benefit and with the play of chance. But importantly, the absolute benefits are really rather small—1000 people have to be treated for one year to prevent one death. It is probably a real effect, but it means a lot of people have to be treated to gain this small benefit. As we don’t know the harms, it seems wrong-minded to me to treat everyone with a statin. In these circumstances, lifestyle changes and stopping smoking would be far preferable.”

And that is where the controversy comes in. Other researchers think the authors are making conclusions that go beyond their own evidence. Heartwire also quoted Dr. Colin Baigent, a clinical researcher from Oxford, as saying:

“I object to the conclusions they have drawn from their review. They say there is not good evidence of benefit, but their own data show significant reductions in deaths and cardiac events. They didn’t show any increase in adverse events in their review, but they then say the benefit is not worth the risk. That doesn’t make sense.”

This does make for an interesting science-based medicine conversation. In this case the two sides largely agree on the data, but differ in terms of how to apply that data to the practice of medicine. This, I feel, can be a very constructive controversy. This is exactly the kind of question that should be agonized over by experts. While I think the Cochrane reviewers are displaying a negative bias against statins, they do provide balance to the pro-statin bias of pharmaceutical companies who sell statins. In the end, the data is out there and practitioners and patients will be better informed in making decisions about statin use. I am concerned about media reporting of this issue. It is easy to oversimplify the take-home message as “statins do not work” and I have already read commentaries quoting this study to support that position.

My read of this evidence is that there is solid evidence that statins have a real benefit for primary prevention. This benefit is small, which is exactly what I would predict for a preventive measure in a low-risk population. The data also show that statins are safe. The major risk is for the development of an inflammatory muscle disease, but that is very rare. For interventions that prevent death – that lower mortality – I think even small benefits are worthwhile. Further, having a heart attack or stroke, even if it is not a fatal event, has a very negative effect on quality of life. Taken together, one person per year out of several hundred taking statins for primary prevention will avoid a heart attack, stroke, or death. From a purely medical point of view, that sound pretty good to me.

What seems reasonable is to use statins for primary prevention in those who have some risk factors for vascular disease, in patients with genetically high cholesterol, and in those with high cholesterol or significant risk factors in whom lifestyle counseling has not yielded adequate results. Try diet and exercise first – and always in conjunction with medication, but statins are a reasonable choice in selected patients, even for primary prevention. We could use more studies to better delineate where to draw that line, but that will be difficult as any difference in outcome is likely to be slight and therefore massive trials will be needed to get statistically significant results.

Cost effectiveness is a tougher issue, because we then have to arbitrarily decide what a human life is worth in terms of medical expense. This issue has become more acute as health-care costs rise and everyone is looking for ways to cut back. What I have not seen is a calculation of the cost of statins for primary prevention vs the cost savings from reduced vascular events. Having a stroke or heart attack is expensive, and pays for a lot of prevention. The question is – exactly where is the line crossed in terms of the vascular risks of the population being treated.

The good news is that many statins are now becoming available as generics, with a marked reduction in cost. There is already a Spanish analysis showing that the availability of generics is making statin treatment more cost effective.

Conclusion

This recent Cochrane review of statin use for primary prevention supports the conclusion that statins are safe and effective in reducing vascular events and overall mortality even in primary prevention. The benefits are statistically small, which is expected for a preventive measure in a low risk population. It is still unclear where to draw the line in terms of which patients should receive statins, but these data will help practitioners and patients make individualized decisions about cholesterol management and vascular prophylaxis.

Because this is ultimately a judgment call, the results of this study can be spun to a variety of conclusions. The study authors chose to present an overall negative conclusion – that the effect size is too small to be worth it. While other experts, looking at the same data, have come to the opposite conclusion – that statins are worth it. It is important to emphasize that the debate is not about whether or not statins have a real effect – they do, but about the cost-benefit of statins as an intervention for primary prevention.

One could also argue that Cochrane reviewers, given that their purpose is to provide objective and thorough reviews of existing evidence for specific clinical questions, should take a more neutral approach to interpreting the data. This is not the first Cochrane review discussed on SBM that can be criticized for taking a decidedly biased approach to the evidence in their conclusions. This should prompt some soul-searching, in my opinion, on the pat of the Cochrane collaboration.

I wasn’t really surprised to learn that camel milk is being promoted as a medicine. I long ago realized that the human power of belief is inexhaustible. The news did make me laugh, probably because camels are rather funny-looking animals, because I am easily amused, because it reminded me of some of my favorite camel jokes, and because it wouldn’t do any good to cry.

Camel milk has been claimed to cure or benefit patients with diabetes, tuberculosis, stomach ulcers, gastroenteritis, cancer, allergies, infections, parasites, autism, even AIDS.  This isn’t really quite as silly as it might sound.  PubMed does list several studies showing health benefits from camel milk. A handful of studies have suggested that camel milk improves control of blood sugar in diabetes, but they are preliminary studies that typically compare standard treatment to standard treatment plus camel milk rather than using a blinded control. There are also a few small, poor quality studies suggesting a possible benefit in allergies, in peptic ulcers, in infections such as hepatitis, and in schistosomiasis. All in all, the research doesn’t amount to much. Camel milk can only be classed as experimental treatment. The existing studies justify doing more (and better quality) research, but they don’t justify prescribing it to treat patients.

Among other properties, camel’s milk is high in vitamin C, low in vitamin A, and low in fat compared to cow’s milk, and it is tolerated by those who are lactose intolerant. It is different from cow’s milk in many other ways that I won’t attempt to list, but the clinical significance of those differences is not clear.

A couple of studies suggested reasons for caution. A study in Saudi Arabia, where brucellosis is endemic, showed that the main source of brucellosis infection was unpasteurized camel milk  and there is a report of anaphylaxis to camel’s milk in a child with atopy.

The founder of the American Camel Coalition, Millie Hinkle, ND, says

The high levels of insulin in camel’s milk and the antibodies, which are much simpler in structure than human milk antibodies, enable it to penetrate deeper into the human tissue and cells [whaat?], which means that the milk has the potential to serve as a major weapon against many human illnesses.

She thinks that studies done in other countries on autism, diabetes, cancer, heart disease, Crohn’s, Parkinson’s, food allergies and a variety of other illnesses have been “impressive.” I couldn’t locate the studies she cites for some of these claims, and I didn’t think the ones I did find were “impressive.”  The one she cites for autism is not original research but just a speculative rumination that includes brief, unconvincing reports of three patient observations and talks about highly controversial and disproven hypotheses as if they were proven facts. She wants to repeat previous studies using pasteurized camel milk; obviously pasteurization is a wise move, but rather than repeating previous studies, why not do better designed, controlled studies?

I didn’t know what to make of one study I found on PubMed. Its abstract said

Camels’ milk, women’s milk and cows’ milk were kept at 30 degrees C and refrigerated at 4 degrees C. This explains the necessity to immediately freeze milk if it needs to be kept even for a few days. Cows’ milk remained good for days if stirred and then turned sour, enabling the making of cheeses and butter. Camels’ milk did not sour at 4 degrees C for up to 3 months. This means that camels’ milk is mainly good only for drinking, as was promised to this animal by the Prophet.

Isn’t it inappropriate to make religious comments in scientific articles? Is religious support pertinent? In Sunni Islam, the Sahih Bukhari, one of the six major Hadith collections, does include several verses where the prophet Muhammad is said to have advocated drinking camel’s milk and urine as medicine. For instance,

The climate of Medina did not suit some people, so the Prophet ordered them to follow his shepherd, i.e. his camels, and drink their milk and urine (as a medicine). So they followed the shepherd that is the camels and drank their milk and urine till their bodies became healthy. Then they killed the shepherd and drove away the camels.

The Sahih Bukhari’s medical advice is not reliable or even consistent. It also says

Healing is in three things: A gulp of honey, cupping, and branding with fire (cauterizing). But I forbid my followers to use (cauterization) branding with fire.

Does that mean Muslims shouldn’t bother with any medical treatment but honey and cupping?

I’m not just picking on Muslims. Other religious texts also give questionable medical advice.  In the Essene Gospel of Peace, Jesus gave detailed instructions for colon cleansing using river water and a long-necked gourd:

Seek, therefore, a large trailing gourd, having a stalk the length of a man; take out its inwards and fill it with water from the river which the sun has warmed. Hang it upon the branch of a tree, and kneel upon the ground before the angel of water, and suffer the end of the stalk of the trailing gourd to enter your hinder parts, that the water may flow through your bowels.

I’d rather drink camel milk than do that.

There is even a book Love Thine Enemas and Heal Thyself. One of the customer reviews on Amazon.com says “This book helps people understand the love of God, in a very intimate area.”  You can find the darnedest things on the Internet! But I digress…

There is a website, Camel Milk for Health that recounts one (only one!?) story about a young man who had an undiagnosed condition that allegedly made him “allergic to all foods…unable to eat or digest any foods, unable to absorb any food nutrients” so that he was “subsisting on a tablespoon of rice and a tablespoon of rice milk per day.” Do you believe that? His parents claim he was cured by drinking camel milk, and they tell how they had to battle the authorities to get special permits to import the milk into Canada. The website announces a symposium to be held in Vancouver BC on February 9^th with 3 panelists entitled “Camel Milk: A New Alternative Medicine.” The main speaker is a retired professor of veterinary medicine from Israel who has done some of the research. The symposium is sponsored by an orthodox Jewish congregation, the oldest and largest synagogue in Vancouver. I am puzzled, because camels and camel milk are trayf (not kosher) and are forbidden to orthodox Jews.

While looking for evidence of possible health benefits, I came across some intriguing camel trivia in the Wikipedia article:

• Camel milk can’t be made into butter by conventional churning methods.
• The Abu Dhabi Officer’s Club serves camelburgers. Bubonic plague has been transmitted by eating camel liver.
• The ancient Roman emperor Heliogabalus enjoyed eating camel’s heel.
• Camel blood is consumed in Northern Kenya.
• Camel lasagna is available in Alice Springs, Australia.

It’s nice to know these things. Forgive the digression.

If you want to try camel milk, you can’t. Selling it is illegal in the US. According to a CBS news report in July 2010,

The FDA allows people to drink camel milk, but it can’t be imported or sold in the U.S. until a test for drug residues is validated, said FDA spokesman Michael Herndon.

Could this be a conspiracy by Big Dairy to prevent competition?

The Camel Milk for Health website links to the Oasis Camel Dairy website, which is interesting and has some cool pictures. The OCD is producing camel milk but is not legally allowed to sell it. What they can and do sell is camel milk soap for $5.00 in varieties like “gold frankincense and myrrh” and “rosemary mint.” They also sell camel milk chocolate bars.

All of the research seems to be on one-hump camels. It’s not clear whether Bactrian camel milk is equally efficacious. The “one hump or two” question remains to be answered; but there’s no rush, since we can’t get either kind of milk. Jabalicious and other brands have recently come on the market in the UK, but those of us who live in the US will have to either wait for FDA approval or buy our own camel and milk it ourselves.

Occasionally, there are topics that our readers want — nay, demand — that I cover. This next topic, it turns out, is one of them. It’s a link to a TED Talk. I’m guessing that most of our readers have either viewed (or at least heard of) TED talks. Typically, they are 20-minute talks, with few or no slides, by various experts and thought leaders. Many of them are quite good, although as the TED phenomenon has grown I’ve noticed that, not unexpectedly, the quality of TED Talks has become much more uneven than it once was. Be that as it may, beginning shortly after it was posted, readers of both this blog and my other super-not-so-secret other blog started peppering me with links to a recent TED Talk by Dr. Deborah Rhodes at the Mayo Clinic entitled A tool that finds 3x more breast tumors, and why it’s not available to you.

At first, I resisted.

After all, I’ve written about the issues of screening mammography, the USPSTF guideline changes (here, too), the early detection of cancer (including lead time and length time bias, as well as the Will Rogers effect), and a variety of other topics related to the early detection of breast cancer, such as overdiagnosis and overtreatment. Moreover, to put it bluntly, there really isn’t anything radically new in Dr. Rhodes’ talk, at least not to anyone who’s been in the field of breast cancer for a while. Certainly, there’s no new conceptual breakthrough in breast imaging and screening described. As I will discuss in more depth later in this post, there’s an interesting application of newer, smaller, and more sensitive detectors with a much better spatial resolution. It’s cool technology applied to an old problem in breast cancer, but something radical, new, or ground-breaking? Not so much. What Dr. Rhodes describes in her talk is the sort of device that, when I read about it in a medical journal, produces a reaction along the lines of, “Nice technology. Not ready for prime time. I hope it works out for them, though. Could be good.” So it was with molecular breast imaging (MBI), which is the topic of Dr. Rhodes’ talk. So I continued to resist for about two or three weeks.

Then our very own Harriet Hall sent me the link. I cannot resist Harriet. When she suggests that perhaps I should blog about a topic, it’s rare that my response would be anything other than, “Yes, ma’am. How soon would you like that post and how many words?” I keed, of course, but only just. The best I could come up with was a wishy-washy “But this isn’t really anything all that new,” which is true enough, but the way Dr. Rhodes tried to sell the audience on the idea of her technology brings up a lot of issues important to our audience. I also thought it was important to put this technology in perspective. So here I go. First, I’ll start by describing what really set my teeth on edge about Dr. Rhodes’ talk. Then I’ll go to the primary literature (namely her brand, spankin’ new article in Radiology describing the technology) and discuss the technique itself.

The truth? You can’t handle the truth!

What irritates me about Dr. Rhodes’ TED Talk starts right at the beginning:

There are two groups of women when it comes to screening mammography — women in whom mammography works very well and has saved thousands of lives and women in whom it doesn’t work well at all. Do you know which group you’re in? If you don’t, you’re not alone. Because the breast has become a very political organ. The truth has become lost in all the rhetoric coming from the press, politicians, radiologists and medical imaging companies. I will do my best this morning to tell you what I think is the truth. But first, my disclosures. I am not a breast cancer survivor. I’m not a radiologist. I don’t have any patents, and I’ve never received any money from a medical imaging company. And I am not seeking your vote.

Later in the talk, Dr. Rhodes says:

If this technology is widely adopted, I will not benefit financially in any way. And that is very important to me, because it allows me to continue to tell you the truth.

I bet you can guess what irritates me about these statements. Actually, it’s two things. First, it’s Dr. Rhodes’ invocation of “The Truth.” Strictly speaking, there is no “truth” in science or medicine. There are hypotheses that are supported by evidence, experimentation, and, in medicine, clinical trials, and there are hypotheses that are not. Most scientific hypotheses are not black and white “true” or “false,” either. Rather, individual hypotheses fall somewhere closer to being true or false, based on the evidence, and they can move closer to or father away from being “true” as new evidence comes in. Indeed, the object of scientific investigation is to falsify hypotheses. Hypotheses that are easily falsified fall by the wayside quickly. Those that are not advance to more intense testing. Those that have withstood the most attempts to falsify them and provide highly useful explanatory and predictive value might eventually graduate to being full-fledged theories.

The next thing that irritates me is Dr. Rhodes’ implication that everyone else has an ax to grind (and is therefore probably lying to you), where as she does not because she doesn’t receive money from medical imaging companies, isn’t a breast cancer survivor, and is not a radiologist. Don’t get me wrong; financial conflicts of interest (COIs), particularly undisclosed ones, are very, very important to know about, because they can (and all too often do) warp the perspective of even the most diligent, honest, rigorous scientist. However, financial COIs are not the only COIs. Just because someone proclaims that she has no financial COIs (or professional COIs) does not mean that she does not have biases or COIs that can be just as strong as the financial COIs of scientists who stand to make a lot of money if their research results in a marketable drug, treatment, or medical device.

Before I examine Dr. Rhodes’ invention (and her claims for it) in a bit more detail, in the interests of full disclosure, I’ll point out that I actually have a non-financial interest in a competing imaging technology for the breast. Researchers at our cancer institute have developed what I consider to be a truly innovative and promising breast imaging device. It’s based on ultrasound and can produce images of the breast almost as striking as those produced by breast MRI. This device has even resulted in a startup company that was featured in our governor’s state of the state address last week. As you can see, it’s a big deal to our cancer center. Since, through some fluke of flukes, I’ve somehow managed to find myself in leadership positions within the clinical and research breast cancer programs at my cancer center (obviously, the cancer center director hasn’t realized his mistake yet), I have a stake in the success of this device. More importantly, not only do we see the chance to have a major positive impact on women’s health if this device is validated but it would bring all sorts of prominence to our institution in general and my programs in particular, just as the success of Dr. Rhodes’ MBI device would improve women’s breast screening and bring all sorts of glory to the Mayo Clinic and her programs. None of this means that either Dr. Rhodes or I are likely to be lying or stretching the truth, but we both have COIs based on our belief in our respective devices. Arguably, my COI is less intense, because this device had been developed before I accepted my current job, and I am not directly involved in its commercialization. A COI does, however, exist nonetheless, and I acknowledge it.

Another thing that bugs me about Dr. Rhodes’ talk is her implication that the radiology world is somehow closing ranks to keep her from bringing this technology to the masses. While she does have a point that some radiologists were utterly shameless in protecting their turf and launching what can only be called histrionic attacks on the new guidelines (the quote by Dr. Daniel Kopans, a very prominent breast imaging radiologist at Harvard, about the USPSTF guidelines that Dr. Rhodes cites at 2:49 in the video being an excellent example), it’s a bit of a stretch to claim that somehow radiologists are so biased against her technology that they won’t give her a fair shake. Before I explain, I mention a couple of points that Dr. Rhodes makes that are correct. First, Dr. Rhodes is correct that breast density appears to be an independent risk factor for breast cancer that has only relatively recently been appreciated as such. Unfortunately, it is in dense breasts where mammography has the biggest problem in detecting cancer. Indeed, that’s part of the reason why it’s not as good in women under 50; their breasts tend to be denser. Second, she is most likely correct that digital mammography is probably not more sensitive or specific for detecting breast cancer, particularly in women with dense breasts, although I will point out that the evidence is not as cut and dried for this assertion as Dr. Rhodes makes it out to be.

Dr. Rhodes also fails to mention that digital mammography does have some major advantages over conventional film mammography and that they are not inconsequential advantages, either. These include permanent storage and duplication of as many copies of a study as needed. In other words, if a woman goes for a second opinion, instead of taking a jacket full of films (what I was used to until relatively recently), she can take a CD and provide an exact copy of her suspicious mammogram to the consulting surgeon. In the old days, she would have to sign out her jacket or bring an inferior film copy. Lost films are a thing of the past, and instead of huge rooms full of large manila envelopes stuffed with X-ray films, the studies can now be stored on hard drives and backed up off site. Digital mammography also allows for the digital manipulation of the image, not to mention the development of image analysis algorithms that can assist the radiologist in detecting suspicious lesions. Finally, digital mammography probably requires less radiation, particularly in women with dense breasts, the very group in whom Dr. Rhodes is trying to increase sensitivity and specificity of breast cancer detection, although this benefit hasn’t been fully verified yet.

Now, I wouldn’t be all that surprised if there was some skepticism over her idea. I’ll explain why in a bit more detail in the next section. However, I really do think Dr. Rhodes goes overboard in implying in at least two parts of her talk that the radiology world and the mammography world are somehow conspiring (or at least so resistant to new ideas that the practical effect is the same as conspiring) to prevent new breast imaging technologies (like hers) from gaining a foothold. Perhaps the most egregious example is this passage, where she talks about having submitted her article to four different journals and having it rejected by each one:

After achieving what we felt were remarkable results, our manuscript was rejected by four journals. After the fourth rejection, we requested reconsideration of the manuscript, because we strongly suspected one of the reviewers who had rejected it had a financial conflict of interest in a competing technology. Our manuscript was then accepted and will be published later this month in the journal Radiology.

At least she didn’t mention Galileo or Ignaz Semmelweis. I’m grateful for small favors.

In any case, this is a highly explosive charge to make so casually, without describing the evidence that led the authors to suspect that one of the reviewers had a financial COI in a competing technology. In fact, if three journals rejected her manuscript before Radiology apparently rejected it and then reconsidered it, did it ever occur to her that perhaps her manuscript just wasn’t very novel? Actually, the manuscript that ultimately was published in Radiology was pretty decent, but there are other reasons that papers, sometimes even good papers, have trouble being published. For instance, she didn’t say which journals she tried first before settling on Radiology. For instance, if she had tried the New England Journal of Medicine, The Lancet, and the Journal of the American Medical Association (JAMA), the reason her manuscript was rejected would be fairly obvious. It’s an interesting new technology with promising preliminary results, but not interesting enough to a broad enough audience to be likely to be published in such high impact journals. None of this implies that Dr. Rhodes’ work isn’t solid work, but the reluctance of journals to publish her results doesn’t suggest that MBI is a technology “they” don’t want you to know about, either.

Everything old is new again

There is no doubt that Dr. Rhodes is charming and a very effective advocate for her preferred breast imaging technology. Nowhere is that more clear than the middle portion of her talk, where she describes the genesis of the MBI. Who wouldn’t be moved at her description of a friend who found a lump in her breast while she was pregnant (and Dr. Rhodes was pregnant as well), the fear it engendered, and how that event inspired Dr. Rhodes to wonder if there was a better way to detect breast cancer? Who didn’t find her account of serendipitously bumping into physicists, who told her about a new kind of gamma detector that was much smaller than previous generations of such devices, compelling? Who wasn’t appreciative of her description of the first device the medical physicists and Dr. Rhodes cobbled together with duct tape and those early primitive tests of its ability to detect radiotracer concentration in the breast? Certainly not me. The image of a bunch of brand new tiny gamma detectors cobbled together with duct tape was priceless. Nor do I in any way want to detract from the hard work and development that went into her MBI device. I do, however, have a small problem with how Dr. Rhodes discussed it in her TED Talk.

That problem is that scanning the breast with radiotracers as a means of looking for breast cancer is not a new technology at all. It’s been around for at least three or four decades. Go back to the 1960s, in fact, and it’s not difficult to find references to the detection of breast cancer using various injected radioisotopes. Since then, at various times such techniques been called scintimammography or sestamibi breast imaging, and they’re all based on the same concept as many nuclear medicine imaging modalities: Inject a small amount of radiotracer that is differentially taken up (or not taken up) by the cell of interest or by cells exhibiting the disease process of interest (i.e., cancerous cells), and then take pictures. Positron emission tomography (PET) scans work this way. So do MUGA scans and bone scans. About 20 years ago, ^99mTechnetium sestamibi became the most commonly used radioisotope for breast cancer detection in the breast (as opposed to looking for metastatic disease).

Indeed, if you read Dr. Rhodes’ recently published study in Radiology, you’ll quickly see that what Dr. Rhodes and her team are doing is nothing more than sestamibi breast scanning. Specifically, she is using technetium (^99mTc) sestamibi scanning combined with mammography. Currently, only one ^99mTc sestamibi compound, Miraluma, which is manufactured by DuPont Pharmaceuticals, is FDA-approved for breast imaging in the United States, which is why such scans are sometimes referred to as Miraluma scans. The same isotope is sold for cardiac imaging under a different name (Cardiolite). The problem with the various nuclear medicine breast scans for screening purposes has always boiled down to an unacceptable lack of specificity and sensitivity. That’s why I have to wonder if the reason for the skepticism that greeted Dr. Rhodes’ results isn’t at least in part due to a collective shrug of the shoulders, as reviewers thought, “Been there, done that.” Certainly, fair or unfair, that probably would have been my initial reaction if this paper had come across my desk.

Not that it’s a bad study or a technology without promise. Neither is true. In fact, as a proof-of-principle study it’s perfectly acceptable, and that’s all it’s billed as in the paper. The findings of the paper are summarized quite well in this figure (click to enlarge):

As you can see, the study started out with 1,007 women with heterogeneously dense breasts enrolled and ended up with 936 women who completed all imaging whose data were available for analysis and whose cancer status was verified. Positive cancer status was defined as a positive biopsy showing cancer, while negative cancer status was defined as women who had a subsequent negative mammogram in the following year, a negative biopsy, or a negative prophylactic mastectomy. Reported sensitivities were 27% for mammography alone, 82% for gamma imaging (MBI) alone, and 91% for a combination of mammography and gamma imaging. The corresponding specificities were 91%, 93%, and 85%. All in all, these are good numbers, particularly when compared to mammography alone in women with dense breasts. Diagnostic yield was 3.2 per 1,000 for mammography alone, 9.6 per 1,000 for gamma imaging alone, and 10.7 per 1,000 for both. Finally, the positive predictive value (the chance of having cancer if the test is abnormal) was 3% for mammography, 12% for gamma imaging, and 8% for both. Basically, the study suggests that the addition of MBI with ^99mTc sestamibi can increase the sensitivity and specificity of breast cancer detection in women with dense breasts.

Still, it has to be pointed out that there were only 11 women with breast cancer in the entire population. This is lower than one would normally expect for a typical study of mammography, likely because of the large number of younger women (as young as 25) in the population studied. The reason for this was in order to include a lot of women with dense breasts. Unfortunately, this means that it doesn’t take very much to skew the numbers one way or another. What this implies is that a much larger study is very much indicated in order to get a more precise estimate of what the sensitivity and specificity of this test is and how much it really adds to mammography. My guess is that it will add something to the mammographic screening of dense breasts, but probably not as much as this initial study suggests. Sadly, the decline effect will likely rear its ugly head.

There is another consideration here, namely the question of radiation. Dr. Rhodes touches on this issue in her talk:

So now that we knew that this technology could find three times more tumors in a dense breast, we had to solve one very important problem. We had to figure out how to lower the radiation dose. And we have spent the last three years making modifications to every aspect of the imaging system to allow this. And I’m very happy to report that we’re now using a dose of radiation that is equivalent to the effective dose from one digital mammogram. And at this low dose, we’re continuing this screening study, and this image from three weeks ago in a 67 year-old woman shows a normal digital mammogram, but an MBI image showing an uptake that proved to be a large cancer. So this is not just young women that it’s benefiting. It’s also older women with dense tissue. And we’re now routinely using one-fifth the radiation dose that’s used in any other type of gamma technology.

When I heard this, I wondered something. What is the effective total body dose? Injecting a radioisotope is a different thing than aiming an X-ray beam at the breast. ^99mTc has a half-life of around six hours, which means that it takes 24 hours for the radiation levels to fall to 1/16 of the original. In this study, Dr. Rhodes administered ^99mTc equivalent to 20 mCi. According to the chart included in this drug information, the estimated radiation dosimetry for a dose of 30 mCi of this tracer is 0.2 rads (approximately 0.2 cGy) to the breast, meaning that the dose used in this study was approximately 0.13 rads. Dr. Rhodes is correct that this is approximately the same amount of radiation as a single mammogram administers. However, she leaves out consideration of the dose of radiation to which other organs are exposed. For instance, 20 mCi of Miraluma results in a dose of approximately 2 rads to the small intestinal wall, 2.6 to 3.6 rads to the wall of the large intestine, 1.0 rads to the ovaries, and 0.3 rads to the bone marrow. In a single dose, these doses are not very high, but remember that we are talking about a screening test that is meant to be administered repeatedly, possibly even yearly, to women with dense breasts. Over the course of 30 years (or even more) of screening, the radiation dose to tissues other than the breast could rapidly add up, in addition to also adding to the dose of radiation to the breasts. These are not trivial concerns, particularly the potential for a significant cumulative whole body dose of ionizing radiation over decades of screening.

The return of the revenge of the Will Rogers effect

Finally, I hate to be a spoilsport, but some of the images that Dr. Rhodes displayed to me did not impress me that much. For example, she showed an image where a mass was not seen on mammography but showed up on MBI. Here’s what bugged me: It was a 5 cm tumor, and, quite frankly, the signal from MBI was not that impressive at all. I also wonder if anyone actually — oh, you know — examined the patient. Most 5 cm tumors are palpable as masses. True, I’ve seen the occasional patient where such a large tumor is difficult to detect, but these patients are relatively uncommon. In another part of the talk, Dr. Rhodes showed one slide in which the mammogram showed one lesion, but the MBI showed three, one of which was only 3 mm in diameter. Unfortunately, she did not say whether pathology of the resected tissue verified that these were indeed separate foci of cancer.

What this leads me to believe is that Dr. Rhodes either doesn’t acknowledge or doesn’t seem to understand the concept of the Will Rogers effect, more formally known as stage migration. This is the phenomenon where a new imaging modality detects tumor that couldn’t be detected before. The name is based on Will Rogers’ famous joke: “When the Okies left Oklahoma and moved to California, they raised the average intelligence level in both states.” This little joke describes very well what can happen after a new imaging modality is introduced into cancer diagnosis. Basically, the increased sensitivity of a new technique (like MBI) can result in a migration of patients from one stage to another that does the same thing for cancer prognosis that Will Rogers’ famous quip did for intelligence. For example, patients who would formerly have been classified as stage II cancer (any cancer) have additional disease or metastases detected that wouldn’t have been detected in the past, thanks to the new imaging modality. They are now, under the new conditions and using the new test, classified as stage III, even though in the past they would have been classified as stage II. This leads to a paradoxical effect in which the survival of both groups (stage II and III) appears better, even though there has not been any actual change in the overall survival of the group as a whole. This paradox comes about because the patients who “migrate” to stage III tend to have a lower volume of disease or less aggressive disease compared to the average stage III patient and thus a better prognosis. Adding them to the stage III patients from before thus improves the apparent survival of stage III patients as a group. Meanwhile, the patients who have extra disease detected by the new technology tend to be the stage II patients who would have recurred and done more poorly compared to the average patient with stage II disease; i.e., the worst prognosis stage II patients. But now, they have “migrated” to stage III, leaving behind stage II patients who truly do not have as advanced disease and thus in general have a better prognosis. Thus, the prognosis of the stage II group also ends up appearing to be better with no real change in the overall survival from this cancer.

There’s another effect, as well, an effect that was first noticed when breast MRI began to be widely used for the preoperative workup of breast cancer. Because of the greater sensitivity of MRI, frequently more disease was discovered than expected, leading to more extensive surgery. The mastectomy rate, which had been falling for decades as a result of the greater understanding among surgeons that breast conserving surgery resulted in the same survival rate as mastectomy, began to rise again. Over the last few years, evidence has been accumulating that the routine use of preoperative MRI does not improve survival rates or increase the rate of lumpectomies with negative surgical margins but does increase the rate of mastectomies (blogged here). In other words, when it comes to screening, as I have described many times before, more sensitivity is not always better. It might be better in the case of MBI because the sensitivity of mammography in women with dense breasts is pretty low, but we won’t know until we do the studies.

Which brings me to another part of this talk that irritated me:

Mammography isn’t perfect, but it’s the only test that’s been proven to reduce mortality from breast cancer. But this mortality banner is the very sword which mammography’s most ardent advocates use to deter innovation. Some women who develop breast cancer die from it many years later. And most women, thankfully, survive. So it takes 10 or more years for any screening method to demonstrate a reduction in mortality from breast cancer. Mammography’s the only one that’s been around long enough to have a chance of making that claim.

Elsewhere, on the TED Blog, Dr. Rhodes says that she thinks we should stop debating mammography:

So the problem is whenever a new technology comes around, the mammography mafia, as we call them, says, “Your test is no good, because you can’t demonstrate a mortality benefit.” Well, of course we can’t demonstrate a mortality benefit. Mammography’s been around since the 1960s; they’re the only ones who have a prayer of demonstrating a mortality benefit, because it takes that long to demonstrate.

First, we need to stop debating mammography and put our resources into developing and evaluating alternative screening techniques for women with dense breasts. MBI is certainly a very promising technique, and there are other promising techniques. Second, we need to accept an endpoint for success that is not strictly mortality-based. Although mortality is the most important outcome, there are intermediate outcomes that can serve as acceptable proxies for mortality. For example, instead of insisting that each technique must demonstrate a reduction in mortality from breast cancer, I believe it is acceptable instead to evaluate whether one technique can find tumors at an earlier stage – in other words, small tumors that have not spread to the lymph nodes.

The “mammography mafia”? Nice. I also wonder if Dr. Rhodes has considered the possibility of lead time bias and length bias in her screening test. Whatever the case, in essence, Dr. Rhodes’ argument boils down to a case of special pleading, wherein she insists that a different, more lenient, standard be applied to her favorite technology than was applied to mammography and than is applied to any other screening test. I can’t agree. MBI should be subject to the same standards as any other screening test for breast cancer. If there’s one thing we’ve learned over the last 30 years or so of mammographic screening is that it’s harder than it seems it should be to save lives with a screening test and that screening tests have unintended costs and produce unintended harms. These have to be balanced against the benefits. We will never know for sure what these risks and benefits are for MBI if we don’t do the studies, and if it takes a decade or more to find out what they are, then so be it, particularly given that there is potential for harm as well as benefit. That’s how long it will take. Also remember this: MBI subjects the entire body to radiation in order to try to save lives from breast cancer. How do we know that repeated doses of ^99mTc sestamibi won’t result in the increased incidence of, for example, colorectal or ovarian cancer that cancel out any decrease in mortality observed that is attributable to better screening for breast cancer?

We don’t. And we won’t unless we do the studies.

Dr. Rhode’s MBI methodology is an example of a test that is evolutionary, not revolutionary. There is nothing whatsoever wrong with that, either. That’s how science advances, building incrementally on what has gone before. In fact, Dr. Rhodes and her coworkers are to be commended for taking a test that never caught on widely because of its low sensitivity and specificity and recognizing that the technology had developed to the point where it might be possible to overcome these limitations. I just wish she wouldn’t sell it to the general public as though it were some radical new test that the “mammography mafia” don’t want you to know about.