Bitwise

The Uninsured Are Now Unpaid Alpha Testers for the Government

They should get discounts on their premiums.

Jeffrey Zients
Jeffrey Zients testifies before the Senate Budget Committee on April 11, 2013, in Washington.

Photo by T.J. Kirkpatrick/Getty Images

Jeff Zients, former director of the Office of Management and Budget and current leader of the “tech surge” team to fix healthcare.gov, has drawn a line in the sand: Nov. 30. That is the date by which “healthcare.gov will be smooth for a vast majority of users,” he said on a press call last week.

I don’t think Zients set that date. Here’s why. Dec. 15 is the deadline for people to sign up in order to have insurance by Jan. 1, which is when marketplace coverage begins. Obviously the earlier more people sign up, the better. But past Jan. 1, you then have three months—until March 31—to get insurance before you’re penalized under the mandate.

The administration does not want to change these dates any further, and they definitely want coverage to start on Jan. 1. Insurers are already warning of premium sticker shock if open enrollment goes beyond March 31—whether their warnings are legitimate or not, I don’t think the administration wants to find out. So for all of Zients’ prestige and authority, I’m assuming the administration’s response to any request for time beyond Nov. 30 was: “But it’ll work on Dec. 1, right?”

On his Friday call, Zients differentiated between healthcare.gov’s “scale” issues and its “functionality” issues. I believe this marks the first time that the administration has admitted that healthcare.gov’s problems went far beyond the site being overloaded. Zients said that account creation had now been fixed, with more than 90 percent of users now able to create accounts. But only 30 percent have been able to complete an actual insurance application. And that’s not even to say that the application is correct, owing to reports of children getting listed as multiple spouses and the like.

If only 30 percent of people can actually complete an application on a website, why on earth is the website still up? So people can play insurance-application roulette with 7–3 odds against them? Why not take the site down until it works?

Well, remember how there was very little testing done on the system? It seems that not only was very little testing done, but testing frameworks weren’t set up. That means the team fixing healthcare.gov not only has a lot of bugs to fix, but they don’t have infrastructure in place to identify and reproduce the bugs, which are the first step to fixing them. Under a tight deadline, any such infrastructure will be ad hoc and inadequate. So it’s important that healthcare.gov stays open simply so that potential applicants can run into bugs and report them.

In effect, the uninsured are now serving as alpha testers—i.e., guinea pigs—of a mostly untested system. As they are now serving as de facto government workers, I think they deserve a discount on their premiums, preferably taken out of the contractors’ fat paychecks.

Nonetheless, Zients’ stats and admissions of reality were refreshing and bode well. I am more ambivalent about the promotion of data-hub contractor QSSI, owned by insurer UnitedHealth Group, to the role of “overseeing the entire operation.” QSSI is one of those contractors that “have not met expectations,” to borrow a line from Department of Health and Human Services Secretary Kathleen Sebelius, who testifies on healthcare.gov before the House on Wednesday. As we saw, QSSI admitted that their $85 million component broke down on Day One. QSSI also screwed up Housing and Urban Development’s reverse-mortgage HERMIT system in 2009–2011, for which they were paid a cool $32 million.

So promoting QSSI seems like a bit of an unearned reward; I shudder to think how much they’ll get in addition to the $85 million the government already paid them for healthcare.gov. It does signal, however, that the $200-plus million contractor CGI Federal is in the doghouse, since, to paraphrase CGI’s Cheryl Campbell, CGI made the whole front of the house that is healthcare.gov, and QSSI only made “the front door”; evidently the front door is now looking better than the rest of the place. (I previously suspected that CGI bad-mouthed the administration to the New York Times, which would be another strike against them.)

“There’s a punch list of fixes,” Zients said, “and we’re going to punch them out one by one.” He makes it sound so easy. It is, sometimes. Like lines of code, all bugs are not created equal. Fixing bugs is an asymptotic process. Let’s say there are 100 bugs in a piece of code. The easiest and most commonly occurring ones will be fixed first. That leaves the rarely occurring ones and the trickiest ones. Those will take more than half the debugging time. One software maxim says the hardest 10 percent of bugs take 90 percent of the time to find and fix.

Here is a rough approximation of the debugging of healthcare.gov over time:

Prior to release, minimum testing was done. Bugs were found and fixed haphazardly but not comprehensively. After release, a tremendous number of bugs were found, and some were fixed, but there was evidently little process in place, as it took three weeks just to get someone—who turned out to be Zients—to say the first word about process. With some bug triage methodology and management in place, I optimistically believe that the pace of fixing issues has picked up and that they have entered the hump in the graph of fixing more bugs faster.

Contractors testified that systems testing only took place in the last weeks before Oct. 1, rather than the months you really need. Surprisingly, this is somewhat good news from a bug perspective. If virtually no testing was done, that means many of those “easy” bugs are still there, and so the programmers attacking the system right now will be able to make a lot of progress in a hurry—as long as they know whether something is a bug. If the system integration among the contractors’ pieces and the government systems and the insurance companies has been underspecified, a lot of that will need to be worked out in a hurry. With good lines of communication, though, this is still possible. (Zients presumably has the authority to remove anyone who gets in the way, like greedy contractor executives.)

As the more crucial bugs are fixed, the number of fixes trails off. The goal is to reach relative stability, indicated by the last line on the graph. I say “relatively” because it’s an asymptotic line: You never know for sure that you’ve fixed every bug; you only know that fewer and fewer bugs are being discovered. That line signifies the point where people breathe easy and bugs are no longer coming in faster than they can be fixed. Zients’ goal is to get as close as he can to relative stability by Nov. 30 so that the site is functioning for “a vast majority of users.” Presumably there will be contingency plans in place for people who still have trouble, but those numbers will be small enough that the administration can announce that things are more or less working—though given the tone of the congressional hearings in the past week, I expect Republicans to leap on even the rarest of bugs as evidence that the surge has failed and we need to send in David Petraeus.

On the other hand, there are some disturbing indications that healthcare.gov isn’t anywhere near getting over that bug-fix hump. Consider that the administration had to bring in Zients—someone wholly outside of the Centers for Medicare and Medicaid Services (CMS)—before we got anything resembling a straight answer about the state of the project. Even if the administration felt it necessary to bring in a crisis manager to fix things in a hurry, it should not have taken a month to find someone to admit statistics and confess to functional problems.

Worse, with the exception of the data hub being fixed, Zients made no mention of any progress since the rollout. Nothing should have stopped him from citing bugs being fixed, insurance applications working better, or any other good news. But he didn’t.

So Zients did not give the impression of jumping into an ongoing repair process, but of starting one. He even made it sound like the list of problems was something he had drawn up, not one he had inherited. None of these are auspicious signs, nor is the Verizon data center crash that took down healthcare.gov on Sunday, and then again on Tuesday. Zients has a month—a deadline he likely did not choose—to make healthcare.gov good enough. It’s possible. But my dead-reckoning says he has very little margin for error.