I also criticized Story's story for citing the results from an e-mail survey of 138 Yale undergraduate females without crediting the survey to any author. For a story so dependent on the fuzzy quantifier "many," it's important for readers to know the provenance of Story's purported hunk of hard data, that "roughly 60 percent [of respondents] said that when they had children, they planned to cut back on work or stop working entirely."
Story says she conducted the survey from November 2004 to January 2005 while at Columbia. Earlier this week, the editors of Gelf Magazineposted a list of questions from the survey, and remarked how some of them were "loaded," that is, likely to produce a biased result. For instance, the first question—"When you have children, do you plan to stay at home with them or do you plan to continue working? Why?"—assumes that the respondents intend to have children.
Story says she revised the survey after its weaknesses were pointed out to her. But in her final tabulation, she retained the responses from about 30 women who answered the first survey because, she said, the responses didn't vary significantly from those given to the revised version. She concedes the survey wasn't conducted with social-science rigor but calls it "a very good journalistic questionnaire."
The problem with Story's e-mail survey is not that she asked a lot of students questions. Reporters are supposed to ask lots of people questions. But if a journalist wants readers to be impressed by numbers like "roughly 60 percent," they must 1) say who collected the numbers and 2) explain how the numbers were collected. The Times and Story failed the reader by not stating that these findings were about as anecdotal and impressionistic as, say, the findings of a columnist like David S. Broder based on 100 interviews he conducted in Iowa to take the state's political temperature. Broder would never write, "roughly 60 percent of 100 Iowans interviewed believe the president is doing a bad job," because a hard number indicates that such numerical findings have real significance (especially when their source is not divulged). Instead, Broder would typically place the results in their proper anecdotal context, by using a phrase like "most of the Iowans I met with. ..." That would dispel the misapprehension that he thought his interviews had the same footing as as a Gallup poll. Story, on the other hand, presented her results to sound like good sociology.
Story has strong and adamant defenders. Master's adviser Sylvia Nasar, a former Times reporter, accuses me of "shooting first and asking questions later" because I published my first critique without conducting any interviews with the principals, and only later talked to Story and others connected to the piece. While I sometimes do interviews for press critiques, I feel no more obliged to phone the author of a work of journalism to discuss it with him before writing a press column than a book critic feels that he should phone up the author of a book before reviewing it. The article or book is the thing. Unlike most book reviewers, though, if my piece is called into question I am willing to 1) correct it, and 2) do additional research and interviews for a follow-up, as I have here.
Story wrote for the Times business section over the summer. Her Times colleague, the much decorated David Cay Johnston, is a major fan of her work. "Among the young journalists I have mentored over the past three decades, Louise Story is in a league by herself," he writes in an e-mail. She quickly grasps "subtle issues at the level of theory and principle whose significance and context she writes about in plain English."
One criticism of Story's article is that college students are poor predictors of what sorts of adults become. To test this idea I conducted some purely anecdotal research of my own: I Googled the lead character of the 1980 New York Times story, Mary Anne Citrino. Within minutes, I reached her at her New York City office at the Blackstone Group, an investment and advisory group, where she is a senior managing director.
Citrino laughed at this week's Times story when she read it, recalling her role in the similarly squishy Times story from a generation ago. She says the Times reporter misrepresented what she said, attributing to her sentiments that were "the exact opposite of what I meant."
"I never wanted to be a full-time mother," says Citrino. She says she was considered the most gung-ho career woman among her classmates, never stopped working after finishing school, has three children, and put in 20 years at Morgan Stanley before joining Blackstone a year ago.
"I never even considered giving up my career," Citrino says.
But that's just one anecdote, mind you.