As for the drugs, there is no great mystery about the efficacy of antidepressants. We have access to the results of large-scale trials whose protocols were published in advance and whose data have been analyzed openly at every stage. Study after study shows a response rate on the order of 50 percent to 60 percent, where the response to a placebo pill is 35 percent to 40 percent. In general, most of the positive change occurs in the sicker patients. The more stringent the study, the more robust the outcome. In research on hard-to-treat depression, like depression in conjunction with strokes or heart disease, antidepressants prove useful.
Another word about the measure Turner utilizes: Effect size was developed to assess interventions in education and psychotherapy. Studies in those fields cover widely different outcomes, using a variety of tests. In order to integrate and compare unlike measures, statisticians wanted a formula that would put results on a level playing field, by taking into account the intractability of a target for change. The mathematical correction factor is indirect, but the idea is that you ought to get more credit for changing stable phenomena and less for phenomena that fluctuate naturally.
It turns out that most psychotherapies have large effect sizes. A brief course of psychotherapy has more influence on mental health than addingnine months of reading instruction, halving class size, or introducing computers to a classroom does for academic success. In most head-to-head trials, antidepressants are at least as effective as psychotherapy. And in recent large studies where psychotherapy failed or showed minimal results, antidepressants succeeded. So, the power of antidepressants, properly tested, is likely to be many times what the aggregate data in the Turner analysis suggest.
To turn the matter on its head, the low effect sizes Turner reports for the FDA studies are for the very same drugs that perform well in research that vigorously tracks outcomes. If that's the case, what do greater effect sizes signify? In the Turner analysis, Celexa demonstrated a low effect size: 0.24; but Celexa has just been shown, in a quite rigorous study, to bring benefit to about half of patients with complicated, chronic depression that had not responded to prior treatment. Effexor weighed in at 0.40. If the ratios Turner discusses are to be taken seriously—if Effexor is understood to be more than half, again, as effective as Celexa—then certain antidepressants may be very effective indeed. Some psychiatrists have always argued that Effexor, which directly influences a broader range of brain pathways than Prozac or Celexa, ought to be a better antidepressant—along the lines of older drugs, like Elavil, that may also have helped a higher percentage of patients, but with harsher side effects.
Then again, flawed studies are flawed studies. Perhaps the best thing to say about this newdata analysis is that it bears no news at all about antidepressants. They are just as good or as bad as we imagined them to be. The article's contribution is to show that the publication process obscures negative studies about drugs; it adds nothing original about the drugs themselves.
And, of course, academic publishers have moved to clean up their acts. In 2004, the International Committee of Medical Journal Editors, a group that includes representatives of the New England Journal of Medicine, JAMA, the Lancet, and others, vowed not to publish outcome studies unless the trials had been publicly registered before the enrollment of the first patient. In 2007, the FDA moved to require more open registration of drug trials as well. The result should be much-desired transparency, from early in the course of a drug's evaluation. But that transparency carries a risk—it might highlight poor studies that lead us to abandon promising medications. Here's hoping that more open reporting will create pressures for better quality research.