An Economist book review asserts, “Humans will never again create a project quite like Wikipedia.”
Written on nearly 23,000 rolls of manuscript, the Yongle encyclopedia, commissioned by a Ming emperor in the 15th century, was the largest reference work in history. It held that title for around 600 years—until Wikipedia, a startup run out of a chaotic office in Florida, overtook it after just six years in business. Today the collaborative online encyclopedia runs to 30bn words in more than 300 languages. Its Latin edition alone has more articles than the “Encyclopedia Britannica”.
As Wikipedia celebrates its 25th anniversary this year, it faces big problems. It is shedding both readers and volunteer editors. It is at the centre of an escalating culture war. And AI poses an existential threat. As Richard Cooke, an Australian journalist, observes in a thought-provoking new book, Wikipedia is the largest encyclopedia ever written by humans—and it is likely to hold that title for ever. (Any future rival is sure to be made with the help of AI.)
As an early adopter and occasional editor, I’m simultaneously amazed at the scope of the project and shocked at how often notable subjects, mostly people, have only the most cursory entries or lack one at all.
As the world’s go-to crowd-written reference source, Wikipedia serves as both an oracle and a mirror. Its story can be traced back to a childhood purchase. When Jimmy Wales was three years old his mother bought a copy of the “World Book Encyclopedia”, which the family updated each year using stickers sent by post. Decades later, Mr Wales was casting around for business ideas. An online encyclopedia never need go out of date, he reasoned. So with a colleague, Larry Sanger, he founded Nupedia in 2000. Its seven-step process of editing and fact-checking meant that after five months it had approved a single article (on “atonality” in music). So to speed things up, they changed tack, launching a crowd-sourced encyclopedia in 2001.
Wikipedia was dreadful—and this was the secret of its rapid early growth. “The average person seeing the young Wikipedia for the first time thought they could do better,” Mr Cooke writes. Later that year the site came into its own, as more than 100 articles about the September 11th terrorist attacks were added in less than a month.
While a crowdsourced encyclopedia was certainly subject to error, as Stephen Colbert delightfully lampooned, it has long been established as more accurate than even Britannica, the former gold standard. Indeed, it went out of print in 2012. It turns out that people with niche expertise will work for free to create and update entries. Which, it turns out, has its drawbacks.
Mr Cooke’s book is an enjoyable rabbit hole of information, rather like its subject. It paints an affectionate portrait of the many oddballs and misfits who have made the site what it is. The second-largest language on Wikipedia is Cebuano, a minority tongue of the Philippines, which has 6m articles—mostly about insects—that have been uploaded by a bot made by a Swede who married a Filipina. A user called Giraffedata cannot bear the phrase “comprised of”, and has spent 15 years removing tens of thousands of occurrences. No one will be surprised to learn that most Wikipedia editors are men.
The make-up of Wikipedia editors is reflected in the site’s content. Its article about Scrappy Doo, a cartoon character, was at one stage 2,000 words longer than the one for Poland. And although editors wear their national dress on the first day of the annual Wikimania conference, in a multinational display of diversity, they are not an especially diverse bunch. For Wikipedia’s first ten years, half of all edits came from five countries; there were more editors in the Netherlands than in Africa.
While amusing, none of that is surprising.
But now Wikipedia faces even bigger challenges. One is the febrile political climate. Certain topics have always provoked “edit wars”: the most contested pages include Israel, Jesus and Donald Trump. There have long been problem contributors (the Church of Scientology was the first organisation to be banned from editing Wikipedia, in 2009). But recent arguments about misinformation and censorship have been harder to deal with. Mr Sanger, whose relationship with Mr Wales soured, has denounced Wikipedia’s “globalist, academic, secular and progressive” bias. Last year Elon Musk, a former fan of the site, announced Grokipedia, an alternative to what he called “Wokepedia”.
It turns out that Colbert’s complaints—two decades ago, now—about “wikiality” and “truthiness” were prescient:
You see, any user can change any entry, and if enough other users agree with them, it becomes true. … If only the entire body of human knowledge worked this way. And it can, thanks to tonight’s word: Wikiality. Now, folks, I’m no fan of reality, and I’m no fan of encyclopedias. I’ve said it before. Who is Britannica to tell me that George Washington had slaves? If I want to say he didn’t, that’s my right. And now, thanks to Wikipedia, it’s also a fact. We should apply these principles to all information. All we need to do is convince a majority of people that some factoid is true. … What we’re doing is bringing democracy to knowledge.
While there are certainly a lot of facts about Israel, Jesus, and Trump, it turns out that there is no neutral way to describe them upon which we can all agree. For example, how do we describe the events of January 6, 2021? Or the jury findings in the E. Jean Carroll case? And even if we could agree on the language, how prominently should they be placed in the entry?
The charged political atmosphere has exacerbated a second problem: a lack of editors. Early Wikipedia welcomed newbie contributors. But after some sticky cases of defamation, it tightened up its rules, particularly regarding articles about living people.
And yet a prominent defense intellectual with whom I worked at the Atlantic Council was accused by Deborah Jeane Palfrey, the so-called “DC Madam,” of being one of her clients as part of her defense strategy. He denied the accusation, and she provided no evidence to support it. Yet it remains part of his Wikipedia entry almost two decades later. I would argue it doesn’t belong there, both because it’s unverifiable and because it’s irrelevant to his expertise. Wikipedia’s editors disagree.
Since 2017 new editors have been banned from creating fresh entries. Trigger-happy editors even deleted an entry by Mr Wales—on Mzoli’s Meat, a South African barbecue restaurant—for failing the notability test. As Mr Cooke puts it, anyone can edit Wikipedia in the same way that anyone can, in theory, represent themselves in court. For amateurs it can be a complex, hostile experience. The number of active editors in English has fallen by a third since its peak in 2007.
Wikipedia’s “notability test” has always frustrated me. With a printed encyclopedia, such choices need to be made. But electrons are essentially free.
Because the existence and depth of pages are purely a function of user interest, wild disparities exist. The above-cited example of the Scrappy Doo page being more extensive than Poland’s is amusing. I’m routinely surprised, although I guess I shouldn’t be, of the number of prominent military leaders—four-star heads of combatant commands, for example—whose entries are just a line or two. And some have no page at all.
The third, most ominous threat is AI. Wikipedia played a crucial role in training large language models (LLMs) with its vast, free, multilingual dataset. It is probably “the single most important training corpus in the history of artificial intelligence”, writes Mr Cooke. This has had the side-effect of immortalising the site’s many errors and hoaxes. Ask ChatGPT who invented the electric toaster, and it may still tell you it was Alan MacMasters, a Scottish prankster who once credited himself as its inventor on Wikipedia as a joke.
AI’s absorption of Wikipedia means that more people than ever are reading its content. Yet it also means that fewer people are visiting the site itself. When a Wikipedia link is suggested by a search engine, nearly a tenth of users click through. When Wikipedia is cited by an LLM, the clickthrough rate is only 0.33% (and not all credit their sources). Since 2022 Wikipedia has lost almost a quarter of its readership—and with it, opportunities to solicit donations and recruit new editors. Wikipedians are “a smaller club working harder to maintain something fewer people see”, laments one longtime contributor.
Wikipedia’s billions of words, written, edited and categorised by millions of human volunteers, helped create the conditions for AI. Now AI’s use of that material threatens to suffocate the very source that helped bring it to life.
Google started doing this years ago, and the integration of AI into search has radically accelerated that trend. But, yes, almost all of the summaries for things I’ve searched in recent months have either been based on Wikipedia or Reddit entries. Indeed, I’ve had one of my own Reddit write-ups quoted back to me.
From the standpoint of the people doing the hard work of creating those entries—much less site owners trying to make a living—that seems very much like theft. But most end users likely appreciate being saved the trouble of clicking through.
