Years ago, mathematician John Allen Paulos described a brilliant stock scam, in which a crooked broker sends out a huge number of letters to potential clients. Half the letters say the market will rise; the other half predict the opposite. As Paulos explains, “No matter whether the index rises or falls, a follow-up letter is sent, but only to [people] who initially received a correct prediction.” On that half, the scam is repeated. After several iterations, the broker has hooked some fraction of his initial marks—the guy correctly predicted a half-dozen or so market moves!—and the unsuspecting rubes are ready to be fleeced.
Pharmaceutical companies employed a similar ruse to propel sales of drugs for “off-label” indications, or uses not green-lighted by the Food and Drug Administration. The FDA approves new prescription drugs only after clinical trials show a benefit in the form of a pre-specified outcome, such as fewer asthma attacks, better blood sugar control, or some other metric. (For example, Neurontin was approved to control seizures.) Companies are forbidden to advertise or promote the drug for any other problem. However, no law stops doctors from prescribing the drug for whatever reason they like—and they certainly do.
In 1996, a young researcher named David Franklin left Parke-Davis due to his disgust over off-label marketing of the anti-seizure drug Neurontin. (Franklin reported that one company executive told him, “[W]e need to be … holding [physicians’] hands and whispering in their ear, Neurontin for pain, Neurontin for monotherapy, Neurontin for bipolar, Neurontin for everything.”) A resulting class action suit was settled for $430 million in 2004, and thousands of pages of corporate documents soon ended up in a searchable digital library at the University of California-San Francisco.
Three years ago, Stanford researchers found that 20 percent of all prescriptions are written for “off-label” use, such as using an anti-seizure drug to treat attention-deficit disorder, and the vast majority of such uses “had little or no scientific support.” By one conservative estimate, the annual U.S. market for off-label prescription drugs is about $44 billion, or about one-fifth of the value of all domestic automobile sales.
According to a remarkable analysis of the Neurontin documents, published last month, many clinical trials of the drug took a shotgun approach. Study patients took the drug, and researchers measured tons of possible outcomes (like pain with touch, pain with cold, excessive pain with pinpricks, more than a dozen different scales for psychiatric symptoms, and so on). By random chance, if you measure enough outcomes, at least some of them will appear better after drug treatment. When the time came to report the findings, however, the researchers systematically omitted the outcomes on which the drug had no effect—and presented only the data showing benefit. That’s like dealing dozens of hands of poker to yourself but showing only the hand with good cards.
Like the victims of Paulos’ investment swindle, patients and doctors have no way of knowing about the outcomes that weren’t winners since they were previously hidden. All they see is that the drug looks like it works for certain off-label indications. In truth, the drug may be no better than a placebo. But rumors build at academic conferences and journals, subtly encouraged by the manufacturers, and doctors start writing prescriptions. At the height of Neurontin’s popularity, for example, the drug made $3 billion annually, mostly from patients with problems other than seizures.
What’s worrisome is that such statistical manipulation occurs even when no drug companies are clearly involved; an astounding 40 percent of drug trials funded by the Canadian government selectively omitted key outcome data, perhaps because researchers want their pet projects to be winners.
In 2004, in an attempt to fight the problem, the editors of three leading American medical journals decided to publish only clinical trials in which a single outcome was pre-specified and registered with a public database, like clinicaltrials.gov, and many other journals followed this lead. In other words, researchers had to first pick a “primary” outcome like survival from breast cancer, perform the clinical trial of the drug, and then report the primary outcome measures without making excuses. (Notably, the lobbying arm of the drug industry, the Pharmacy Research and Manufacturers of America, didn’t endorse public registries.)
But most medical journals are largely supported by pharmaceutical advertising—and many don’t take the registration requirements seriously, even though they supposedly required this procedure for all clinical studies. In September, for example, an international group of researchers reviewed clinical trials published in 10 high-impact medical journals in 2008. Fewer than half the published studies were adequately registered. Even among those, about one-third had some unexplained discrepancies in reporting that largely favored the drug under investigation. The review authors concluded, depressingly, that “selective outcome reporting is prevalent.”
To be sure, some off-label uses are important, particularly in rare conditions without accepted treatments, such as unusual cancers or rare genetic diseases. But for the most part, the proliferation of off-label prescription undercuts legitimate medical practice since it skirts accepted safety and efficacy standards. (Most often, psychiatric drugs are the offender. An example: Doctors increasingly prescribe the anti-hyperactivity drug Adderal as a weight loss aide for overweight kids.) Unfortunately, the FDA recently loosened restrictions on manufacturers’ ability to promote off-label use—companies are now allowed to distribute scientific papers promoting off-label use, within certain boundaries.
Without greater oversight from medical journals, the public must rely on whistle-blowers to come forward, as with Neurontin (such folks collect a hefty bounty from damage awards). But it would certainly be better for our collective health if clinical studies were kept honest from the beginning.
Become a fan of Slate on Facebook. Follow us on Twitter.
