How poorly designed trials for cancer drugs are hurting patients.

Health and medicine explained.
Jan. 10 2008 12:02 PM

Group Therapy

How poorly designed trials for cancer drugs are hurting patients.

Creating new treatments for cancer patients is exceedingly difficult. Only 3 percent to 5 percent of drugs that enter clinical development are approved—less than 25 of the 600 or so currently being studied—and it can take 15 years to bring a new medication from the laboratory to the pharmacy. Part of the problem is simply the complexity of the science. But part of it is poor study designs, which deviate from the way in which trials are conducted in other areas of medicine, wasting money, time, and a scarce resource: patients who are willing to be research subjects.

The worthiness of new cancer drugs is tested through three stages of clinical trials. Using a small group of patients, Phase 1 studies find the highest dose at which a drug can be given safely. Next, Phase 2 studies, which take a few months, look for initial signs of benefit among at most 100 patients. Finally, a drug enters Phase 3. These studies enroll hundreds of patients, take years, and serve as the basis for FDA approval by showing whether the drug helps patients live longer and, if so, with what side effects.


The problem in cancer research lies with Phase 2. In other areas of medicine, Phase 2 studies generally compare two treatments directly. Patients are randomly assigned to receive an already approved therapy or that same therapy plus the new drug. Often the studies are blind; that is, patients, and ideally investigators, do not know who is receiving which treatment. But none of this is true in Phase 2 cancer work. Instead, studies have just a single treatment group: All patients get an approved therapy plus the new drug. The results are compared not with a second study group, but with older studies and hypothetical barometers of success.

The disconnect between cancer and non-cancer Phase 2 studies is dramatic. The latter are eight times more likely to directly compare a single group of patients randomly assigned to different treatment groups, and 30 times more likely to keep the assignments hidden. The non-cancer approach is efficient, accurate, and reliable because it better isolates the impact of the drug in question. The cancer approach lacks all of these qualities.

There are legitimate reasons for the divergence of cancer studies from the norm. The desperate need for new medications and acceptance of harsh side effects tends to lower expectations for new cancer drugs. Also, early signs of response to a medication, like the shrinking of a tumor, can be assessed without direct comparison. But the softer approach isn't working. For several reasons, single-group Phase 2 studies make drugs look better than they really are.

First, as Dr. Robert Glassman of Merrill Lynch and Cornell's Weill Medical College explains, when comparing two studies, as opposed to two groups of patients in the same study, it is impossible to be sure that the only difference between them is the treatment. Results that seem connected to the drug may really be due to some other factor, such as unknown genetic variations between the patient groups, a variable that's easier to eliminate when patients are randomly assigned to two groups in a single study. In addition, patients in the new study could be younger or healthier than those in the old one, or have more subtle advantages, like a lower likelihood of reporting side effects or succumbing to fatigue—personal qualities that can't be measured but affect the data nonetheless.

According to Dr. Mark Ratain of the University of Chicago, the Phase 2 cancer study design also leaves a lot of room for interpretation. Often, doctors set an imaginary mark of failure—if tumors do not shrink by more than X amount, then the drug is considered ineffective. But many researchers draw the mistaken conclusion that if responses are better than X, then the drug works. This is not always true; a decrease in tumor size does not guarantee a longer life. Alternatively, sometimes two targets are set, one high and one low—if responses are below X, the drug doesn't work, and if they are above Y, it does. But, as happened in a recent breast cancer study, investigators may simply disregard the higher measure of benefit if the drug fails to meet it. And so drugs with effects that fall between X and Y are called "promising," or another vaguely flattering term, pumping up what are really mediocre results.

  Slate Plus
Jan. 27 2015 11:18 AM Slate Voice: “American Liar” Listen to Mark Joseph Stern read his piece about Jesse Ventura’s defamation suit against Chris Kyle.