Value-Added Rankings for Teachers

What Women Really Think
Dec. 27 2010 12:09 PM

Value-Added Rankings for Teachers

/blogs/xx_factor/2010/12/27/is_it_the_right_time_to_release_nycs_data_on_valueadded_rankings_for_teachers/jcr:content/body/slate_image
Emily Bazelon Emily Bazelon

Emily Bazelon is a staff writer at the New York Times Magazine and the author of Sticks and Stones

This NYT article about ranking teachers based on students' test scores-what's known as value-added ranking-is the first thing I've read that really lays out the tensions about the meaning of this data. It made me a lot more nervous about releasing the information publically right now, as a dozen news organizations including the Times have sued to do, in a case that's pending. The rankings sound like they have the potential to become a really useful tool, but also like they are not ready for prime time yet. If the data are released now and unfairly hurt some teachers' reputations, could the backlash prevent the rankings from developing into what we really need?

Advertisement

Value-added rankings are supposed to tell us how much the quality of an individual teacher contrributed to boosting the scores of his or her students. Researchers say that with confidence, they can tell us that a teacher who is in the bottom 10 percent year after year is doing a bad job and a teacher in the top 10 percent is doing a good job. But about the middle, they tell us much less:

"In math, judging a teacher over three years, the average confidence interval was 34 points, meaning a city teacher who was ranked in the 63rd percentile actually had a score anywhere between the 46th and 80th percentiles, with the 63rd percentile as the most likely score. Even then, the ranking is only 95 percent certain. The result is that half of the city’s ranked teachers were statistically indistinguishable."

Another more serious problem for individual teachers: The rankings have a high error rate, which means you need years of data to get a fairly accurate picture:

"One national study published in July by Mathematica Policy Research, conducted for the Department of Education, found that with one year of data, a teacher was likely to be misclassified 35 percent of the time. With three years of data, the error rate was 25 percent. With 10 years of data, the error rate dropped to 12 percent. The city has four years of data."

One more problem: It turns out that the city was using a narrow and too easy test. Now they're trying to fix that, but "Daniel Koretz, a Harvard professor whose research helped persuade the state to toughen standards, said that as a result it was impossible to know whether rising scores in a classroom were due to inappropriate test preparation or gains in real learning." The city won’t have rankings with the higher standards til the next academic year. And would we really need several year’s worth to know anything really useful?

It would be nice to think that the city could release its data now, with a full explanation of the flaws, and everyone would calmly take it for what its worth and refrain from jumping to conclusions about individual teachers. But is that likely? When the Los Angeles Times released this data about L.A. teachers last August, some teachers said they got burned . Do we know about yet to be able to tell whether that cost was worthwhile, on balance? I'm sure it depends who you ask, but it seems to me that value-added rankings, in their current guise, invovle making individual teachers bear a big risk of being misjudged in return for information that could improve the system. Isn't that a lot to ask?

TODAY IN SLATE

History

Slate Plus Early Read: The Self-Made Man

The story of America’s most pliable, pernicious, irrepressible myth.

Rehtaeh Parsons Was the Most Famous Victim in Canada. Now, Journalists Can’t Even Say Her Name.

Mitt Romney May Be Weighing a 2016 Run. That Would Be a Big Mistake.

Amazing Photos From Hong Kong’s Umbrella Revolution

Transparent Is the Fall’s Only Great New Show

The XX Factor

Rehtaeh Parsons Was the Most Famous Victim in Canada

Now, journalists can't even say her name.

Doublex

Lena Dunham, the Book

More shtick than honesty in Not That Kind of Girl.

What a Juicy New Book About Diane Sawyer and Katie Couric Fails to Tell Us About the TV News Business

Does Your Child Have Sluggish Cognitive Tempo? Or Is That Just a Disorder Made Up to Scare You?

  News & Politics
History
Sept. 29 2014 11:45 PM The Self-Made Man The story of America’s most pliable, pernicious, irrepressible myth.
  Business
Moneybox
Sept. 29 2014 7:01 PM We May Never Know If Larry Ellison Flew a Fighter Jet Under the Golden Gate Bridge
  Life
Dear Prudence
Sept. 30 2014 6:00 AM Drive-By Bounty Prudie advises a woman whose boyfriend demands she flash truckers on the highway.
  Double X
Doublex
Sept. 29 2014 11:43 PM Lena Dunham, the Book More shtick than honesty in Not That Kind of Girl.
  Slate Plus
Slate Fare
Sept. 29 2014 8:45 AM Slate Isn’t Too Liberal, but … What readers said about the magazine’s bias and balance.
  Arts
Brow Beat
Sept. 29 2014 9:06 PM Paul Thomas Anderson’s Inherent Vice Looks Like a Comic Masterpiece
  Technology
Future Tense
Sept. 30 2014 7:36 AM Almost Humane What sci-fi can teach us about our treatment of prisoners of war.
  Health & Science
Bad Astronomy
Sept. 30 2014 7:30 AM What Lurks Beneath The Methane Lakes of Titan?
  Sports
Sports Nut
Sept. 28 2014 8:30 PM NFL Players Die Young. Or Maybe They Live Long Lives. Why it’s so hard to pin down the effects of football on players’ lives.