War on Wikis: Critical Standards and Standards of Criticism

A Wikipedia article summing up the common objections and answering them.

Also see Wikipedia's article on Why Wikipedia Is So Great.

And, of course, their article on Why Wikipedia Is Not So Great.

Information aside, the interesting thing here is the attempt at honest self-criticism. This is another important difference between the practices of the new media at its best and the way things are done in the old media world.

The standard for the new media is that criticism should be frankly acknowledged and frankly answered. It should not be ignored, dismissed or overborne by claims of expertise.

To say this standard is an ideal rather than universal practice is putting it mildly. A lot of new media practitioners don't do it this way and some of them are pretty scathing in the way they apply the older techniques.

The difference is they suffer for it much more.

The web is in essence a conversation. Communication is two way and neither of the participants has such an overwhelming advantage they can drown out the other party.

This was one of the first things I noticed when I started writing for the web over a decade ago. Articles on web sites drew a lot more comment, especially critical comment, than I was used to in magazines and newspapers.

Personally I thought this was great. While my fear of being wrong in public approaches an obsessive-compulsive disorder -- not uncommon in those schooled in the journalistic paradigm -- I appreciated the fact that I could learn from my readers. For one thing, it meant I could target future articles more accurately to what my readers were interested in. That's very hard to do with a magazine where there is a several-month lag between writing the article and getting the letter to the editor.

If you are writing for the web, or blogging, or social networking you have to accept the fact that your work is exposed to the opinions of others -- and those others have equally powerful channels to express their opinions.

In conventional journalism one party controls both ends of the conversation by deciding what gets into print. While letters to the editor are theoretically welcome, objections and comments are still filtered by the side with the publication.

As A.J. Leibling famously said: Freedom of the press belongs to he who owns one.

This produces some unfortunate behavior. Newspapers, for example, tend to alibi their mistakes rather than admitting them.

I remember one occasion when, as energy reporter for a major metro daily, I covered the explosion of a pole-mounted transformer belonging to the local power company. Such explosions aren't uncommon since the transformers can overheat in use. In themselves they're pretty harmless. What made this one newsworthy was that the transformer was one of the old ones whose cooling oil was laced with PCBs. That meant it was a hazardous material spill in a residential neighborhood.

Fine, but how much hazardous material? PR guy at the utility told me it was "several quarts" so that's what I used in the story. The next day I got a call from a reader who informed me the actual amount of oil in such a transformer is about five gallons. After confirming that independently, I told my assistant editor, implying that we needed to run a correction.

His response was that five gallons is the same as several quarts so we didn't need a correction.

It was his call, but to this day I think that story was misleading. However the assistant editor wanted to keep from printing a correction. The prevailing theory at newspapers and magazines in those days was that if you didn't admit a mistake you hadn't made a mistake.

While that attitude is still with us -- witness The New Republic's response to the wildly inaccurate stories from Scott Thomas Beauchamp in Iraq -- it works a lot less well even for the print media. Beauchamp's fables were quickly exposed by bloggers with military experience and The New Republic made itself a laughing stock with its lame and dishonest attempts to defend its position.

(Note: Because I'm rushed for time, and because the material in the blogosphere on this is so extensive, I am not going to provide specific links. For the military blogger's point of view I'd recommend 'Blackfive'. For an series of summaries of the controversy, see Michael Goldfarb's articles in the "Weekly Standard". Not an unbiased source, but a lot of links to blogs and other sources. And of course there's "The New Republic's" own statements. For an example of why TNR is a laughing stock, take a look at the investigation into TNR's "investigation" of whether a Bradley Fighting Vehicle could run over a dog in the manner Beauchamp described. For a summation of the culture at TNR that led to all this, as well as an example of the journalistic CYA mentality at work, see "How The New Republic Got Suckered"

The smart new media folks have learned this lesson already and public reaction is teaching it to the others. If you make a mistake, face up to it and deal with it honestly. Because if you don't it will be thrown back at you with hurricane force.

The War On The Wikis

Of all the manifestations of the new media, none has attracted the sheer level of bile aimed at Wikipedia, the online collaborative encyclopedia.

(In future posts we’ll examine some of that bile in detail. For now, let’s accept that it is so and move on.)

Part of that is a profound lack of understanding of how wikis work. Part of it is a knee jerk reaction against something a lot of people see as extremely threatening. Many of those people are right to feel threatened because the existence of Wikipedia and things like it is going to force them to change their ways or fade into irrelevance.

Let’s start with a simple fact. According to nearly every study, Wikipedia is about as accurate (in the journalistic sense of being factually correct) as the major print encyclopedias, such as Encyclopedia Britannica. (The best compendium of the studies, ironically, is on Wikipedia . However with the information there it’s easy to find either the original studies or reports on them.)

While the studies aren’t uniformly favorable (the Guardian’s panel of experts found material disorganized and not always helpful) they generally support Wikipedia against its real-world competitors.

(Writing style is something else again. It varies from graceful to what Nicholas Carr calls “garbage” and “an incoherent hodge-podge of dubious factoids”. While Carr misses the mark with ‘dubious factoids’ – I checked the article he’s criticizing against other sources – he’s right about the hodge-podge style.)

Matching an encyclopedia’s accuracy is not an exalted standard, please note. Encyclopedias are notorious for being rife with errors, some of them deliberately introduced to prevent copyright violation. (The late Fred Saberhagen, who was an editor and writer for the Britannica as well as a science fiction author, confirmed that to me in a conversation at a science fiction convention several years ago.) Many of the errors are simple mistakes.

But no matter what their source, there those errors sit, like flies in amber until the next edition of the encyclopedia comes out, often decades later.

This leads to key point that Wikipedia’s critics consistently miss. Unlike printed sources, wikis have a powerful error correction method built into the process.

In fact Wikipedia is sometimes wrong. This is especially true since another front in the war on the wikis comes from those who want to co-opt them for their own purposes by feeding them false, misleading or slanted information. This has apparently become a growth industry among the rich and powerful, with everyone from the CIA to Microsoft editing Wikipedia articles in an effort to elide inconvenient facts.

If you look at this with the mindset of print scholars it is horrendous. If anyone can edit material, how can you possibly produce accurate, reliable information? Obviously you can’t and the end product must be utterly unreliable.

Except you can and it isn’t.

Not only have examinations shown Wikipedia’s articles are about as accurate as those in conventional sources, repeated tests have shown that incorrect information, even if very subtle, are almost always corrected quickly, often within minutes. There have been a very few highly publicized exceptions, such as the claim that John Seigenthaler, former editor of the Nashville Tenneseean , was involved in the murders of President Kennedy and his brother Robert, but those are both rare and very well reported when they do happen.

Part of the response to wikis is simply the normal human problem of appreciating the different. As one of my cultural anthropology professors was fond of saying, “different doesn’t mean better and it doesn’t mean worse. It means different.” Wikipedia, and wikis in general, use a profoundly different method of insuring reliable information. Because it is so different a lot of people have trouble believing it can work as well as the traditional reliance on experts. Except it does.

Some of the attacks are little peculiar. Lee Peoples, a law librarian at the Oklahoma City University Law Library asked his students to analyze a Wikipedia article on administrative law After they found out the piece had more than 50 authors, none of whom was named, the students displayed a ‘healthy skepticism’.

Skepticism about any source is good, authorship of a source is important, but the exercise seems oddly pointless as a test of accuracy. Administrative law is hardly terra incognita to a law librarian. Surely a better test would have been for Peoples or his students to compare the article to standard sources on administrative law to judge its accuracy. In fact it’s hard to see what Peoples’ example accomplished aside from scoring a rhetorical point.

So, differences aside, what’s going on here?

What is happening I think is that there are a lot of people, especially academics, who feel threatened by the easy access to information provided by the web. As a leading source of online information Wikipedia becomes the focus of that fear.

As we’ll see, one of the striking things about the vast majority of the attacks on Wikipedia is their fevered defense of experts and material “created by scholars” and “published by reputable publishers” as the only true source of knowledge. (The quotes are from Michael Gorman, who we will meet in more detail in later posts.) The defenders have a point, but it is vastly overstated.

Indeed it reminds me mightily of the defenses put forward by Catholic theologians during the Reformation against the chaotic, pernicious and dangerous notion that anyone but an expert could interpret the Bible.

Which is, as Karl Marx was wont to say, no accident. Neither is it an accident that the people who are making these defenses of the status quo are largely the academics, intellectuals, librarians and others who have the most to lose in this epistemological earthquake.

(Not all of the hostility is from academics however. For example the “Weekly Standard” makes a habit of ladling out healthy doses of British snark on Wikipedia and the Standard is by no means a haunt of academics.)

In other words, besides some very real and serious concerns about new information sources, there is a lot of protectionism by those who are proactively protecting their oxen from a severe goring by the new media and new information sources like Wikipedia.