This article is part of the Big Shortcut, an eight-part series exploring the exponential rise in online learning for high school students who have failed traditional classes.
When Jeremy Noonan thinks of what was wrong with online education at the Douglasville, Georgia, high school where he used to teach, his go-to analogy is The Big Short. Like Michael Lewis’ account of the financiers and traders who gradually became aware of the looming 2008 financial crisis fueled by overvalued financial products, Noonan had concluded that his own school’s graduation rate growth was also the result of data-based fraud—namely, online credit recovery courses that he says are way too easy for students to game. And, in recent months, he’s embarked on a mission to prove it.
In 2015, Noonan had been tasked with overseeing part of Douglas County High School’s online credit recovery program, in which students retake classes they have failed, ranging from math and sciences to social studies and foreign languages. Looking at the software his students were using—which was built by the for-profit company Edgenuity—Noonan says he noticed that the same questions would appear over and over again when students retook online quizzes and tests. Concerned, he asked a neighbor to write a simple software code that enabled him to track the frequency with which test questions were being repeated.
Noonan says the results of his informal study show how gameable online credit recovery can be. When nine of his students reattempted a final, 50-question cumulative test at the end of the course after failing the first time, they received on average 37 of the same 50 questions, making it comparatively easy to score well based on sheer trial and error.
Tests and quizzes taken over the course of the class also had a relatively high rate of repeated questions, according to Noonan’s data. He says 20 students who repeated 10-question quizzes saw an average of 5.7 of the same questions on the second attempt. Of 20 students who reattempted 25-question “unit tests,” an average of 13.5 questions were repeated. “I don’t think they really use a randomization model at all,” said Noonan, who suspected this repetition may have been intentional, to the detriment of actual learning. “It’s set up to help kids pass, no matter what.”
Edgenuity, which operates in all 50 states, disputes Noonan’s conclusions. “The testing engine pulls questions and quizzes at random,” a company spokeswoman insisted in a statement. While “questions can repeat,” she said it would be “highly unusual” for students to take a repeat final exam with so many duplicate questions.
Deb Rayow, Edgenuity’s vice president of curriculum and credit recovery, said the ease of an online course depends in part on district implementation, acknowledging that school officials can make it easier for students to pass by allowing them to take the tests over and over again, for instance. “That’s a product of local control,” she said. But Rayow said most districts are not so permissive. She added that she’s confident the majority of districts implement Edgenuity courses in a meaningful way aimed at helping students learn the material rather than pushing them through as fast as possible. She declined to comment specifically on the situation at Douglas County High School.
Noonan alleges that his school wasn’t so rigorous in its implementation. He said the school administration actively encouraged students to reattempt quizzes and tests as many times as possible, adding that his predecessor had instructed him to “give [students] as many retakes as possible to maintain their grades at a B-level.”
The school district’s superintendent, Gordon Pritz, denies that claim. He said the district guidelines allow for “three retakes.” The district also challenged Noonan’s assumptions surrounding the dangers of some repeated test questions. “Failing a driving test doesn’t mean you must use different instruments the next time,” a district spokeswoman said.
Noonan contends the district’s over-reliance on credit recovery has boosted graduation rates. District officials agree, although the spokeswoman said there are multiple reasons for the increased rates. She suggested Noonan was choosing to ignore many positive aspects of credit recovery, including its role in preventing some disengaged students from dropping out. That may be true, Noonan says. But it’s also created a “second-class credit,” which in turn is contributing to a “second-class diploma,” he said.
Of course, right now there is no real way of knowing whether the results from Noonan’s small survey reflect something big about credit recovery. While some states are starting to rank and evaluate online courses, how exactly individual schools decide to implement the courses—how permissive they are in letting students repeat tests, for instance—is, indeed, a matter of local control. And there is no broad and public data on how many duplicate test questions students encounter when those tests are in fact taken multiple times over a short period of time. So for now, Noonan’s protests boil down to a he said–she said dispute that probably—given the widespread use of online credit recovery and the pressures on districts to improve graduation rates—has much broader implications.
Noonan’s one-man crusade has had a somewhat unhappy ending: After being assigned to the credit recovery room again last fall, he refused and resigned, considering it a punishment for his outspoken behavior. He is now home-schooling his kids and working to draw attention to what he sees as the major flaws in how credit recovery is rolled out in schools. He says he is motivated by a deep concern that a whole generation of students is being let down by a system that prioritizes graduation over knowledge. “Across the system, we’re failing our kids,” Noonan said.
