Academic Peer Review
Glenn Reynolds has an interesting post on the differences between the nature of peer review at law journals vice those in the humanities. He points to a book review in Wilson Quarterly which notes,
In 2001 and early 2002, a cascade of professional misconduct charges shook the history profession. The well-known popular historians Stephen Ambrose and Doris Kearns Goodwin both were accused of serial plagiarism. Another highly visible historian, Joseph Ellis, the author of Founding Brothers (2000), admitted telling his students at Mount Holyoke College grandiose falsehoods about being involved in the civil rights movement and the Vietnam War. A much-heralded book claiming that colonial-era Americans owned relatively few firearms, written by Emory University historian Michael Bellesiles, was exposed as containing mythical data about nonexistent records.
Pundits wondered whether the flurry of scandals represented a widespread deterioration in professional standards or just a fortuitous cluster. Ambrose died, Goodwin and Ellis publicly apologized, Bellesiles resigned his professorship, and the outrage abated. Yet the problem recurs: In recent months, two of Harvard Law School̢۪s best-known faculty members, Laurence H. Tribe and Charles J. Ogletree, have explicitly atoned for plagiarized passages that appeared in their books.
To anyone who has taught in a law school, where student research assistants are legion, or encountered the paid researchers employed by commercially successful authors such as Ambrose and Goodwin, the most common pitfall is readily apparent. Both Tribe and Ogletree, and perhaps Goodwin, if not Ambrose, had to apologize not for any wrongdoing they had personally committed, but for the egregious sins of ill-trained assistants whose sloppy handiwork they had carelessly incorporated into their own texts. A book with one name on the cover may turn out to have a team of contributors. Most readers may never have pondered the difference, but a history book whose author alone has carried out all of the research and writing is almost always a more dependable work of scholarship than one whose multiple cooks can easily spoil the broth.
[…]
Hoffer, a history professor at the University of Georgia, states that it’s “almost impossible†for any journal or book editor to “double-check manuscript or archival reference notes†so as to confirm the content, or indeed the existence, of cited records. But anyone who has ever written for an academic law review knows that unpaid student editors at those journals painstakingly review photocopies of every footnoted source. A leading history journal supported by a major university could well do the same, even if a similar practice would be prohibitively expensive for most university presses and commercial publishing houses.
Of course, the nature of the fact checking is rather different here. In the case of a law review, the citation is almost certainly going to be something readily accessible–a ruling in a legal case, a statute, the U.S. Constitution, or the like. Not so in history or the social sciences, where the source may be a one-of-a-kind artifact available only in a museum or achive halfway around the world, available only in a language other than English, or a complicated statistical analysis compiled from numerous gigantic electronic databases. The student editors at a law review simply have a far easier job.
This is especially true when we’re talking about book rather than an article and a commercial press rather than a journal. The student editors at a law review have at least one year of training in the law and are typically in the top 10-15% of their class. They’re hardly the “peers” of a Glenn Reynolds, however. They’re perfectly qualified to fact check his citations but it’s unlikely they’re sufficiently steeped in the debate at hand to grapple with its intellectual complexities. They know ConLaw 101 but it would be a sheer accident if they knew much about nanotechnology or space law, let alone the arcane minutae of the legal debates about those topics. By contrast, an article in a social science journal will typically be screened by three scholars who have published on a given topic, a rather stringent screen that will catch most errors. The reviewers are unlikely to actually check each citation or run each statistical analysis over but they are sufficiently familiar with the literature to spot major problems.
The book world is a different phenomenon altogether. A commerical book publisher is typically staffed with people with an undergraduate degree in English or the like. Their competency to fact check a professional historian is limited at best. They can, and sometimes do, pay money to send manuscripts out to scholars for review, but it’s typically a very small stipend for which little more than a skimming is expected.
