Creating new treatments for cancer patients is exceedingly difficult. Only 3 percent to 5 percent of drugs that enter clinical development are approved—less than 25 of the 600 or so currently being studied—and it can take 15 years to bring a new medication from the laboratory to the pharmacy. Part of the problem is simply the complexity of the science. But part of it is poor study designs, which deviate from the way in which trials are conducted in other areas of medicine, wasting money, time, and a scarce resource: patients who are willing to be research subjects.
The worthiness of new cancer drugs is tested through three stages of clinical trials. Using a small group of patients, Phase 1 studies find the highest dose at which a drug can be given safely. Next, Phase 2 studies, which take a few months, look for initial signs of benefit among at most 100 patients. Finally, a drug enters Phase 3. These studies enroll hundreds of patients, take years, and serve as the basis for FDA approval by showing whether the drug helps patients live longer and, if so, with what side effects.
The problem in cancer research lies with Phase 2. In other areas of medicine, Phase 2 studies generally compare two treatments directly. Patients are randomly assigned to receive an already approved therapy or that same therapy plus the new drug. Often the studies are blind; that is, patients, and ideally investigators, do not know who is receiving which treatment. But none of this is true in Phase 2 cancer work. Instead, studies have just a single treatment group: All patients get an approved therapy plus the new drug. The results are compared not with a second study group, but with older studies and hypothetical barometers of success.
The disconnect between cancer and non-cancer Phase 2 studies is dramatic. The latter are eight times more likely to directly compare a single group of patients randomly assigned to different treatment groups, and 30 times more likely to keep the assignments hidden. The non-cancer approach is efficient, accurate, and reliable because it better isolates the impact of the drug in question. The cancer approach lacks all of these qualities.
There are legitimate reasons for the divergence of cancer studies from the norm. The desperate need for new medications and acceptance of harsh side effects tends to lower expectations for new cancer drugs. Also, early signs of response to a medication, like the shrinking of a tumor, can be assessed without direct comparison. But the softer approach isn’t working. For several reasons, single-group Phase 2 studies make drugs look better than they really are.
First, as Dr. Robert Glassman of Merrill Lynch and Cornell’s Weill Medical College explains, when comparing two studies, as opposed to two groups of patients in the same study, it is impossible to be sure that the only difference between them is the treatment. Results that seem connected to the drug may really be due to some other factor, such as unknown genetic variations between the patient groups, a variable that’s easier to eliminate when patients are randomly assigned to two groups in a single study. In addition, patients in the new study could be younger or healthier than those in the old one, or have more subtle advantages, like a lower likelihood of reporting side effects or succumbing to fatigue—personal qualities that can’t be measured but affect the data nonetheless.
According to Dr. Mark Ratain of the University of Chicago, the Phase 2 cancer study design also leaves a lot of room for interpretation. Often, doctors set an imaginary mark of failure—if tumors do not shrink by more than X amount, then the drug is considered ineffective. But many researchers draw the mistaken conclusion that if responses are better than X, then the drug works. This is not always true; a decrease in tumor size does not guarantee a longer life. Alternatively, sometimes two targets are set, one high and one low—if responses are below X, the drug doesn’t work, and if they are above Y, it does. But, as happened in a recent breast cancer study, investigators may simply disregard the higher measure of benefit if the drug fails to meet it. And so drugs with effects that fall between X and Y are called “promising,” or another vaguely flattering term, pumping up what are really mediocre results.
All of these factors mean that many drugs make it through Phase 2, only to fail in Phase 3. The poor predictive value of Phase 2 cancer trials is well-documented. One study found that 81 percent of Phase 3 cancer studies had worse outcomes than their Phase 2 precursors, meaning that the drug turned out to help fewer people than expected. And not one of the 181 Phase 2 studies included in this report reliably predicted the drug’s effectiveness. By contrast, the standard randomized and blind Phase 2 design does far more to eliminate all variables except the drug and produce results that are more likely to be replicated at Phase 3.
So why do drug companies continue to do single-group studies? Many of the small biotechs developing today’s cancer treatments have only one or two drugs in the pipeline. Wooing investors depends on the quality of those drugs, and quality is gauged by advancement through clinical trials. Even if the drug turns out to be a dud, touting positive results along the way to Phase 3 can keep a company afloat while its scientists generate new drug candidates.
Other incentives also come into play. Phase 2 cancer studies are often run by younger doctors for whom a successful study means a prominent publication, which leads to grant funding and job offers. In fact, many scientists do Phase 2 trials with no intention of ever proceeding to Phase 3; they just want the good results. Academic institutions benefit, too: Patient recruitment is a lucrative business for medical centers, and any positive trial is an opportunity to enhance its reputation. These incentives coupled with inertia from having done studies the same way for so long are a recipe for inefficiency. There are more Phase 2 trials in cancer than in any other area of medicine; almost all new cancer drugs look good at that stage, and very few of them end up helping patients.
But so what? What does it matter if companies use Phase 2 studies to lure investors, and doctors use them to lure job offers? Shouldn’t good drugs make it through clinical trials anyway? As it turns out, the answer is no. Fewer than 5 percent of cancer patients enroll in clinical trials, far too few for testing all the drugs in clinical development. As Dr. Glassman notes, these volunteers are wasted on the majority of drugs that turn out to be no good, while promising drugs languish because there is no one left to test them on. So not only do the cancer patients willing to participate in clinical trials lose out, so do all the other patients anxiously awaiting better medicines. The wastefulness of the Phase 2 cancer study design is its own source of harm.
