As Harriet Hall has written (http://www.sciencebasedmedicine.org/?p=353), psychiatry bashing is a popular media sport. There seems to be a bias against treatment of psychiatric disabilities, and a common claim is that antidepressants are no better than placebo. The New York Times illustrated both the perpetuation of the myth that antidepressants are ineffective, and the increasing and disturbing tendency of major media organizations to confuse the wholesale acceptance of medical press releases with medical journalism.

In Popular Drugs May Help Only Severe Depression The New York Times credulously publicized the findings of a recent study that claimed to show that antidepressants are ineffective in treating mild and moderate depression. Yes, that’s what the study showed, but the study itself is so limited, so fraught with problems, and the conclusions are so misleading that the article is a terrible disservice.

Before we consider what the study showed, let’s think about what kind of evidence we’d need to conclude that antidepressants don’t work.

First, although there are different types of antidepressants, the term used colloquially refers to antidepressants of a specific type, SSRI’s or selective serotonin receptor inhibitors. There are other, older types of antidepressants that are rarely used today because of their unpleasant side effects. Hence any study that claims to show that “antidepressants” are ineffective, must look at SSRIs.

Second, there are literally thousands of studies of SSRIs, and it would be helpful to aggregate the results. Aggregating results can be done in a type of paper known as metaanalysis. Metaanlysis adds the results of multiple similar studies to find trends that might not be apparent in individual small studies. But a metaanlysis is subject to several important limitations that must always be considered. The most important limitation is that the authors of the metaanalysis choose the papers to be included. Bias can be introduced by examining only papers that have a desired outcome; that can be accomplished by restricting the inclusion criteria in arbitrary ways.

Let’s look at the study, Antidepressant Drug Effects and Depression Severity. According to the abstract:

Randomized placebo-controlled trials of antidepressants approved by the Food and Drug Administration in the treatment of major or minor depressive disorder were selected. Studies were included if their authors provided the requisite original data, they comprised adult outpatients, they included a medication vs placebo comparison for at least 6 weeks, they did not exclude patients on the basis of a placebo washout period, and they used the Hamilton Depression Rating Scale (HDRS). Data from 6 studies (718 patients) were included…

… The magnitude of benefit of antidepressant medication compared with placebo increases with severity of depression symptoms and may be minimal or nonexistent, on average, in patients with mild or moderate symptoms. For patients with very severe depression, the benefit of medications over placebo is substantial.

According to one of the authors was interviewed for the NYTimes article:

“The message for patients with mild to moderate depression,” Dr. DeRubeis said, “is, ‘Look, medications are always an option, but there’s little evidence that they add to other efforts to shake the depression — whether it’s exercise, seeing the doctor, reading about the disorder or going for psychotherapy.’ ”

Let’s go back and compare the paper to the criteria we identified above. The first criterion was to look at the antidepressants currently used in clinical practice. But out of the 6 studies in the metaanalysis, 3 looked at imipramine, a tricyclic antidrepressant that has not been the standard of care for over a decade because of its unpleasant side effects. The other three studies looked at Paxil (paroxitene). Paxil is an SSRI, but it is only one member of the class of SSRIs. Although all SSRIs share the same mechanism of action, they have different profiles of effectiveness and side effects. Therefore, generalizing from Paxil to all SSRIs cannot be justified.

So in terms of clinically relevant information, the paper included only 3 studies of an SSRI. How did the authors whittle down thousands of papers on SSRI effectiveness to only 3? According to the authors:

The criteria for inclusion required studies to be randomized placebo controlled trials of an FDA-approved antidepressant in the treatment of the full range of patients with major or minor depressive disorder … In addition, the studies had to include an ADM/placebo comparison of at least 6 weeks’ duration and HDRS scores at intake and at the end of treatment. Studies were excluded if they excluded patients on the basis of a placebo washout period. The final inclusion criterion was that individual patient-level data had to be available for analysis.

Are these criteria relevant? Certainly, the inclusion of only RCTs is a reasonable criterion. However, it is not clear why the availability of patient level data is a relevant criterion. Most RCTs, from an enormous range of clinical investigations, do not include patient level data, and using that as a criterion is bound to exclude most studies.

Finally, the decision to remove studies that included a placebo washout period also excludes a vast swath of psychiatric studies. That decision is more defensible, however, since there is disagreement among psychiatric researchers about whether a placebo washout period introduces bias into the study. A placebo washout period involves treating everyone in both arms of the study with placebo for an initial period of time, often 3 weeks. People who respond to placebo are then excluded from the study. The theory is that excluding known placebo responders makes it easier to identify real effects.

Others have argued that excluding known placebo responders up front necessarily makes the drug effects look better than they would have. For example, in a traditional placebo controlled RCT, there might be 30% who respond to placebo and 50% who respond to the medication under study, for a difference of 20%. If some placebo responders are identified during a washout period, let’s say 20% of patients, they will be excluded. The final results may be that 10% responded to placebo and 50% responded to the medication under study, for a difference of 40%, making the medication under study look better.

There is one indisputably arbitrary criterion that is acknowledged by the authors. The initial analysis identified 23 studies, but they could only gain access to the data in 6 studies, so they simply ignored the other 17.

In summary, then, by using questionable exclusion criteria, the authors accessed only 3 clinically relevant studies (the Paxil studies), involving only one SSRI. It is not clear that these studies are representative of existing studies on SSRIs, or even if they can be generalized to other SSRIs. Dr. Rubeis’ assertion that for patients with mild to moderate depression there is little evidence that “medications” add to efforts to treat the depression cannot be justified by the findings in his study. I find his claims to be irresponsible. The paper adds to the literature on antidepressants but is so limited that it cannot tell us whether antidepressants are effective for mild to moderate depression.