After Google, Wikipedia might be the single most powerful new influence on how we as a culture organize, disseminate, and access information. For millions of Web-connected citizens, the online encyclopedia is the place of first resort for looking up everything from Shirley Sherrod to sickle-cell anemia . There’s no question about its scope or popularity: It has 3.3 million articles in English alone (compare that to the roughly 120,000 articles in the online edition of the Encyclopedia Britannica ) and attracts nearly 78 million visitors each month. There’s also no question that it’s an astonishing triumph of open-source development: The entire colossus was built by a bunch of largely anonymous and entirely unpaid contributors.
There is, however, a great deal of argument — and consternation — about the accuracy of Wikipedia entries. (A headline in the Onion made the point nicely: “Wikipedia Celebrates 750 Years of American Independence.”) That’s why I went looking for Larry Sanger, who co-founded Wikipedia along with Jimmy Wales, then quit the project over disputes about its governance and the quality and credibility of its content. Sanger is also a trained philosopher with a focus on epistemology — the study of knowledge — which made him an attractive person to talk to about how technology is changing what we know, what we think we know, and how we think we know it. After leaving Wikipedia, Sanger founded Citizendium , a rival online encyclopedia, and now spends most of his time on WatchKnow , a nonprofit organization that uses wiki principles to organize and rate nearly 20,000 educational videos for kids.
What got you interested in encyclopedias? Did you have some kind of longstanding fascination with them, or was it just an accident of history?
It was pure accident. I was circulating an idea for a Web site around different Internet acquaintances and one of them happened to be Jimmy Wales. He responded by saying, “Well, I’m trying to get this encyclopedia project going; would you be interested in coming to work on it?”
That was Nupedia — he had registered the domain name, but at that point it was just an idea — and I got hired for that job. And then I found that it was a fascinating problem to organize people online to create encyclopedias.
People have been trying to validate, organize, and disseminate information for a long time. Did you look back to other efforts in history to do so?
When I was first starting Nupedia and Wikipedia, everything was moving so fast that I didn’t have time to go back and read Diderot and D’Alembert and all that, which would have been useful. I did read them later. I can’t remember when I read The Professor and the Madman , but that made a big impression me.
That’s pretty funny, considering that it’s a book about the relationship between the editor of a major reference work and a certified lunatic.
It actually resonated very much with the experience I had trying to organize Wikipedia. It’s very interesting to me that here you have the editor of the Oxford English Dictionary , and one of his most prolific contributors was in an insane asylum. A lot of the most prolific Wikipedians, or at least many of them, also seem to have a screw loose. But that doesn’t mean their work is useless.
Do you have a theory about this? Is there something about the project of organizing knowledge that attracts slightly nutty people? Or that turns normal people nuts?
There are a lot of theories on that, actually. But I think the most important thing to say is that Wikipedia has very few practical constraints about people behaving according to normal rules of politeness and fair dealing. They’ve got a zillion rules, of course — that’s part of the problem — but there is no easy way to rein in the bad actors. And unfortunately the bad tend to drive out the good. A lot of the more sane, sensible people out there are just can’t take too much of it.
Yeah, I can imagine that the social dynamics get pretty ugly. But my understanding is that you left Wikipedia over deeper philosophical schisms.
I had lots of deep philosophical schisms with Wikipedia in the end, although also some agreements. The first problem was what we were just talking about: reining in all the bad actors, doing something to reduce the number of trolls and the amount of time we spent dealing with them.
The other problem was that there needed to be some sort of mechanism — it didn’t have to be anything like editorships or review before publishing or anything like that — but some sort of low-key role for experts in the system.
Why did you feel so strongly about involving experts?
Because of the complete disregard for expert opinion among a group of amateurs working on a subject, and in particular because of their tendency to openly express contempt for experts. There was this attitude that experts should be disqualified [from participating] by the very fact that they had published on the subject — that because they had published, they were therefore biased. That frustrated me very much, to see that happening over and over again: experts essentially being driven away by people who didn’t have any respect for those who make it their lives’ work to know things.
Where do you think that contempt for expertise comes from? It’s seems odd to be committed to a project that’s all about sharing knowledge, yet dismiss those who’ve worked so hard to acquire it.
There’s a whole worldview that’s shared by many programmers — although not all of them, of course — and by many young intellectuals that I characterize as “epistemic egalitarianism.” They’re greatly offended by the idea that anyone might be regarded as more reliable on a given topic than everyone else. They feel that for everything to be as fair as possible and equal as possible, the only thing that ought to matter is the content [of a claim] itself, not its source.
It seems to me that this conflict between amateurs and experts boils down to a conflict between egalitarianism and credibility. You gestured toward this conflict in an essay on the Edge.com , where you wrote, “It’s Truth versus Equality, and as much as I love Equality, if it comes down to choosing, I’m on the side of Truth.” Do you find that it really is a zero-sum game — that, as a practical matter, we need to choose between these two goods?
I doubt very much that it’s a zero-sum game. I think it’s absolutely a great thing that people regardless of their credentials can contribute to the shaping of knowledge. And I think we have to creatively design ways of recognizing both the value of amateur work, on the one hand, and the objective value of the knowledge of people who are experts in various fields.
And that’s what you were calling for at Wikipedia, right? For human experts to exert at least some baseline control over the content?
Well, I suppose. But you know, I don’t like the word “control,” because I myself am pretty libertarian in my outlook on these things, quite frankly. It makes me nervous to think of handing the keys over to the experts. But one thing that Wikipedia could do that would not spoil the system — except in the sense that it would cause a huge ruckus among Wikipedians — is simply create a program in which articles are reviewed or rated by experts.
This idea is not new. It’s something that we discussed before I even left. It’s sort of a perennial idea on Wikipedia, in fact, but they’ve never done it.
What would qualify someone as an expert?
Well, I can tell you how we do it on Citizendium. Generally we say that to contribute as an editor in a professional field, you need to have the credentials to practice in that field. So if you want to be an editor in law, then you should have a law degree and a few years of experiences. For more academic fields, we generally require a Ph.D.
None of that is to say that we aren’t open to other people, but they have to establish their expertise up to that level in some other way. So if someone has a master’s degree or even a bachelor’s degree in history but has written a bunch of well-respected books — if there’s enough evidence for another history editor to recognize the person as legitimate — then we would bring such a person onboard.
Have you read some of the criticisms of expertise that have come out in the last several years — for instance, Philip Tetlock’s Expert Political Judgment: How Good Is It? How Can We Know? or, more recently, David Freedman’s new book, Wrong: Why Experts Keep Failing Us and How to Know When Not to Trust Them ?
No, I’m afraid I haven’t looked at those books. I’d like to — they sound fascinating.
I suspect that most people who aren’t involved in the making of Wikipedia, who interact with it as an end product, feel that it’s basically good enough. They get that it’s not perfect, but they have the impression that it’s essentially pretty reliable. In your opinion, how right or wrong is that impression?
I think it’s pretty dangerous to rely in an uncritical way on Wikipedia. Wikipedia frequently gets things wrong — or, more often, states things in a misleading or biased way. The problem isn’t as bad in the hard sciences, but in the humanities and social sciences, certain views tend to come to the forefront, or the leading views on a subject are completely omitted or are left as a footnote. When it comes to subtly representing a dialectic, Wikipedia does a very bad job. But maybe people don’t go to Wikipedia to try to master a debate.
What about factual matters? Are those reliable?
When it comes to just basic facts — statistics about geography or demographics, things like that — then as far I can tell, and as far as I’ve ever heard, those are fairly accurate. They are probably not much less reliable than any traditionally fact-checked source.
One of the ways we get things wrong is by belonging to a community that enforces, often tacitly, one set of beliefs while obscuring or rejecting another. Do you think Wikipedia constitutes a community in that sense, and if so, how would you describe its belief system?
If you’re talking about political biases, I actually think that that’s one of Wikipedia’s least-worst problems. [Laughs] It’s really not as bad as the people at, say, Conservapedia seem to think. I do think that there is a liberal bias on most topics where such a bias is possible, and I think that’s probably a reflection of the fact that, again, the people who work the most on Wikipedia tend to be really comfortable with the most radically egalitarian views. And those people tend to be either liberals or libertarians.
I think the kind of biases that are in some ways more interesting and more pervasive have to do with individual biases not on political issues but on a host of very specific academic issues. An article can reflect the bias of a few people who just happen to be most motivated to work on it. This is a general problem with Wikipedia: What is praised as consensus decision-making or crowd-sourcing often just means that the person with the loudest voice or the most time on his or her hands is the one who’s going to win.
Do you think that as information technology changes, the responsibility of getting it right is changing, too? Is the onus shifting away from the creator of information and toward the consumer?
I think the people who are putting the information out in front of an audience and presenting it as factual are the people who are responsible for it. Unless we’re actually in there editing the articles, then it doesn’t make sense to say that we have an obligation to make sure that the article reads the way it ought to read.
But maybe the question is, how ought we to read it? Should people use Wikipedia in the same way they would use, say, Encyclopedia Britannica, or do we need to start thinking and acting differently?
What Wikipedians themselves would say — and I agree with them on this one — is that Wikipedia has finally awakened in people an understanding that even carefully edited resources can frequently be wrong and have to be treated with skepticism and that ultimately we are responsible for what we believe. That means constantly going back and checking what we thought was established or what we thought we knew. Wikipedians often say that you should never trust any one source, including Wikipedia.
That’s not anything new; it’s always been the case that you should check your source against another source. It’s just that the way that the Internet has exposed the editorial process has, for more critical-minded people, made it absolutely plain just how much responsibility we ourselves bear to believe the right thing.
Which online resources do you yourself turn to most often when you’re looking for information?
Which resources I turn to greatly depend on what sort of information I’m looking for. One of my favorite information resources is Google Maps and Bing Maps . I’ve often used Google Scholar for an essay I’ve been working on lately. When I’m looking for some quick fact, of the sort one finds from an almanac or other reference book, I generally search in Google and then pick a non-Wikipedia source. If there doesn’t seem to be anything as efficient, I’ll fall back on the Wikipedia source. If I’m doing serious research, I don’t spend much time on Wikipedia at all, I’m afraid. I do look in on Citizendium’s offerings from time to time, when I think it might have something on the topic. I also not infrequently grab various books from my bookshelves, the old-fashioned way.
When you started Citizendium, what kind of practical measures did you use to try to counteract the problems you’d seen at Wikipedia?
The main policies that distinguish Citizendium from Wikipedia are that we make use of real names [for contributors], we do make a low-key, guiding role for expert editors, and we started the project with some ground rules. I think we certainly did succeed in making a much more polite, collegial project. And the average contribution to Citizendium is of much higher quality than the average contribution to Wikipedia.
As measured by what? Caliber of the writing, caliber of the thought, accuracy of the content?
Everything. All of the above.
One of the objections over at Wikipedia to the idea of assigning a special role to experts was that it would slow growth by creating a bottleneck. Were they right to worry about that? You started Citizendium in 2007 and predicted explosive growth, but you currently have only about 14,000 articles to Wikipedia’s 3 million-plus.
I don’t think that the way editors participate in the project constitutes a bottleneck at all. If there’s one bottleneck that has made it more difficult for us to grow than Wikipedia, it’s the sign-up bottleneck. One of the things that allowed Wikipedia to grow explosively and with as little friction as possible is that it was not necessary to even create an account in order to participate. On Citizendium, you have to sign up for an account and get yourself approved with an e-mail address, so that adds some friction, that does constitute a bottleneck.
But beyond that, once you’re into the system, if anything there’s less friction than in Wikipedia. It’s easier to work on articles, you’ll experience less resistance on the part of the people who are at work on the wiki with you.
What have you yourself been most wrong about?
[laughs] Oh, boy. That’s a hard one. I’ve been wrong about so many things. I don’t know what the most important error is that I’ve ever made; it would require much more thought to figure that out. But I can tell you the one that bothers me more than any, because it’s one that, off the cuff, does seem to have made the biggest difference. When I was getting Wikipedia started, I didn’t realize just how deeply important matters of governance were going to be. I wasn’t thinking about the problem we would face if we were truly successful. I think there’s a lot of things I could have done in the first few months that would have allowed the project to take off the way it did and yet avoided some of the long-term governance issues.
Like what?
Among other things, I could have established a charter that would allow important editorial decisions to be made through a representative body, as opposed to essentially mob rules — what they [Wikipedians] are pleased to call consensus but which of course really isn’t.
What do you think kept you from doing that?
I was constantly thinking about how to manage the project, but most of that was about managing the day-to-day aspects rather than the long term development. I think I was really taken with the success of the project, and I was busy just trying to build it up and encouraging people to get involved. I didn’t anticipate the potential downsides.
Whom do you wish you could hear interviewed about being wrong?
Hmm. This is one of those questions where as soon as I hang up I’m going to think of what I should have said. There’s any number of people from the Bush administration. And I think it would be very interesting if you could interview Jimmy Wales.
Kathryn Schulz is the author of Being Wrong: Adventures in the Margin of Error . She can be reached at kathryn@beingwrongbook.com . You can follow her on Facebook here , and on Twitter here .
This blog features Q and As in which notable people discuss their relationship to being wrong. You can read past interviews with NASA astronaut-turned-medical-error-guru James Bagian , hedge-fund manager Victor Niederhoffer , mountaineer Ed Viesturs , This American Life host Ira Glass , celebrity chef Anthony Bourdain , Sports Illustrated senior writer Joe Posnanski , education scholar and activist Diane Ravitch , and criminal defense lawyer and pundit Alan Dershowitz .
