Blogging > Peer Review Publishing?
Robert Farley takes a shot across the bow at the academy from the pages of one of his field's most prestigious journals.
Robert Farley takes a shot across the bow at the academy from the pages of one of his field’s most prestigious journals (PS, ”Complicating the Political Scientist as Blogger“).
If you are reading this article in PS, the article has gone through a vetting and editing process that has probably lasted at least 18 months. This process undoubtedly improved the quality of the article, but it also substantially delayed its entry into the debate. Had I simply posted this discussion as a blog response to Sides, it probably would have taken me three or four days to write and edit it. I would have included multiple hyperlinks, effectively “citing” not only Sides article but a plethora of different pieces on blogging and the academy. The article could have been viewed by some 4,000 regular visitors to Lawyers, Guns and Money, plus another 8,000 or so subscribers. Any one of these subscribers could have responded (helpfully or unhelpfully) in our comments section, likely generating a long debate both on the merits of the article and on the merits of the author. Sides could have responded within a day, and a multitude of other political science bloggers might have chimed in during the ensuing weeks.
Instead, I published the article here in PS, giving up all of that in return for a line on my CV with the “peer review” annotation.The delay of this article, the loss of all of the interactivity that the Internet provides, and the substantial reduction in the number of people likely to read the piece buy me a slightly improved chance at tenure and promotion.
To say that this makes little sense is an understatement.
To the extent that the purpose of academic research is to foster debate and expand the field of knowledge, it’s hard to dispute any of that. The pace of publishing in most academic journals, always slow, now seems positively glacial in the age of blogs and Twitter.
Farley’s article is a response to a very good 2011 piece in the same journal by John Sides of Monkey Cage fame. Given that the two have widely-read blogs—and read each other!—they could have had this debate in real time. Presumably, as early as 2009. And, to the extent that this debate about the nature of the political science discipline is healthy and necessary—and I would contend that it’s both—that we’re now having it three-and-a-half years later seems a colossal waste.
Then again, that “line on [the] CV” is a pretty important thing. Farley is an untenured assistant professor at Kentucky’s Patterson School, so an additional publication in PS could be a difference maker. Sides got tenure and promoted to associate professor at George Washington in 2011, so it’s not inconceivable that having an accepted piece at PS put him over the top in the minds of his committee.
The question, though, is whether that should continue to be the case in a digital age. While Sides promotes blogging and other public intellectual activities as a useful adjunct for the academic political scientist Farley goes further, arguing that it should be front and center. The next paragraph after the earlier excerpt continues:
And so, rather than think in terms of how blogging, tweeting, and other forms of social media could accommodate themselves to the traditional profile of an academic career, let me suggest that we should, as a discipline, think in more radical terms. Effectively, our tenure and promotion system is built around an obsolete social and technological foundation, with career success built around posting a few articles in a few journals subscribed to by a few libraries and read by few people (Cosgrove 2011; Healy 2011). Rather than take the apologetic line that Sides advocates, we should think about blogging as a crowbar to pry open the tenure and promotion process.
[…]
To be sure, the metrics for evaluating the contribution that blogging could make to a tenure or promotion case remain murky. Measures such as traffic, links, and comments are all problematic often to the point of uselessness. The best we can offer, perhaps, is that each blog can be evaluated as part of an academic career, and no clear template for how blogging should fit into career progress exists. This proposal sounds frustratingly amorphous, but in most cases the arguments for and against tenure and promotion rely on fuzzy distinction between journals, publishers, and course evaluations, not to mention the always-important-but-never-concrete quality “will this person make a good colleague?” A more holistic approach to tenure and promotion (Young 2010) would remedy some of the problems of relying on the peer-review system, while also encouraging young scholars to “bridge the gap” by writing articles that people will actually read (Young 2010).
[…]
With state legislatures displaying an increasing reluctance to underwrite political science research that their constituents neither understand nor care about, blogging could become an important avenue for public engagement. Thus, the practice of blogging touches on a core interest of the discipline of political science, even if we have not quite recognized that it is a core interest.
This shouldn’t be an either/or. Dan Drezner, Sides, Farley, Eugene Volokh, and many others have combined a highly visible blogging presence with productivity in traditional scholarly activities. Our own Steven Taylor has been tenured, promoted twice, become department chair, and put out two books over the decade he’s been blogging. But I’m certainly on the side of giving academics “credit” for work as public intellectuals and popularizers.
I think I’d go a bit farther than your “giving academics ‘credit’ for work as public intellectuals and popularizers”. The academy continues to operate based on a 17th century model built on a 14th century framework. A lot has happened since then and change is long overdue.
I recognize that it’s appropriate for scholarship to be conservative in the sense that it’s not unreasonable to capitalize on accepted approaches. But it’s now beyond that to stultification and, frankly, corruption.
Time for an overhaul.
@Dave Schuler: I don’t know where to draw the line, exactly. And, frankly, given that I’ve been a more successful blogger than academic by traditional standards, this is all a bit self-serving.
There’s something to peer review and getting past thresholds. Even as someone who’s been blogging for more than a decade now, I enjoy the validation and visibility that comes with placing what is essentially a blog post at the likes of The Atlantic or The National Interest. But, at the end of the day, I can get just as much feedback and have just as much impact at OTB as at those venues.
How exactly do you peer review a political science article? Isn’t that like peer reviewing an article on rhetoric or philosphy?
Peer reviewing makes a lot of sense if we’re talking about a dissertation on surgical techniques. If a lot of patients are dying on the table that’s a fairly objective indication not all is well. Medical device research. Pharmaceutical research. Bio-pharmaceutical research. Pharmacological research. Engineering designs. Programming. Other STEM pursuits.
Directional, long-short strategies for investing in commodities markets would be great topics for peer reviews. Sharpe ratios don’t lie. Betas. Alphas. Standard deviations. That sort of thing. Of course people who work in those fields don’t want their techniques known, much less reviewed. Too valuable in Realityville.
Related topic: Years ago, I was asked by a friend if I wanted to contribute gratis to a law review article on which she was working. Ultimately the answer was no, but not so much because of the obvious issue with opportunity costs. It was a pointless endeavor. Nobody of consequence would have read it. Nobody would have cared about it. Almost immediately after it would have been written it would have been forgotten.
@Tsar Nicholas: Tsar, you don’t know what you’re talking about. (Like normal.) Ideas put forth in law review articles often get picked up and used in arguments later on in litigation. Law review articles are quite often gedanken-experiments: here’s a problem; what if we solved it this way? Because we’re a common law jurisdiction, it’s much easier for an idea to spread via judicial decisions rather than a statute getting passed incorporating it. One of the present (widely used) methods to allocate liability in water pollution was originally suggested by a law student in a law review article.
And I take it you’ve never compared a well-researched and well-argued political science paper to a badly-written one. A policy paper is more than just pontificating, you know….
@James Joyner:
I’m imagining something a bit more like a wiki with membership limited to people with degrees in the field.
The original form of “peer review” was face-to-face defense. As in when you defend your doctoral thesis. After the printing press came along what we think of as modern peer review evolved and really took off when paper became cheap.
However, the modern system doesn’t take the fax machine into account let alone email, blogs, wikis, or social media. I think it’s possible to have the benefits of the present peer review system but have everything take place on a vastly accelerated basis.
@Tsar Nicholas: I’m afraid @grumpy realist is right here: you’re commenting on something you don’t understand. Most modern political science articles are actually highly quantitative and employ large, rigorous data sets. PS is something of an outlier, in that it has a lot of thought pieces on the state of the discipline and teaching. Pure public policy journals, like Foreign Affairs or The National Interest, aren’t peer reviewed in the academic sense; editors either solicit pieces or accept pitches and the editorial staff is the only hurdle to publication.
@Dave Schuler: Yes, I’m shocked at how slowly the mechanisms have evolved. There’s no reason we couldn’t do something like that to subject pieces to more rapid peer review. Theoretically, there’s no reason blog pieces couldn’t evolve into journal articles; the tradition that the research should appear in the journal essentially de novo is one of vanity for the journal, not practical service to the discipline.
@James Joyner:
Shocking!
A few points:
1-) Running a blog like OTB or Volokh is a pretty tough job. You have to comment daily subjects in the news and you have a bunch of very smart people ready to point it out to you if you write something wrong or incorrect.
2-) My experience here in Brazil is the following: there are no Academic journals that have large following outside the university here in Brazil. Most professors write papers and the only people that cares to read these papers are sycophants that have an open interest in finding a professor that can get him a place in the doctorate programs.
And I discovered that it´s in their interest that no one reads these papers because most of them suck. Several of them have no originality, they have inaccuracies and even factual errors. Academically speaking, the US university environment is a much tougher place than here in Brazil, but in the US being a tenured professor in a large professor is a pretty good job, combining safety and good earnings that you are not going to find anywhere else.
I´m not saying that there are not exceptional thinkers in the US academics, and I admire lots of them. Many of them faces daily scrutiny on blogs like these. But I also think that for many professors the fact that very few people reads their papers is an asset, not a liability. That´s why many professors do not complains about the pay walls in JSTOR and other academic journals.
😉
The whole Ward Churchill affair(A tenured professor from a top state university writes dumb things on the internet and then everyone that´s was offended by that can uncover tons of academic malpractice to fire him) is a example of what I´m trying to say.
@Tsar Nicholas:
What you’re talking about is typical research for about 85% of Finance academics (the other 15% is behavioral finance) and about 70% of Accounting research (the other 30% is behavioral in nature – either based on behavioral economics/finance or, more commonly, based on psych/sociology theories applied to topics like auditing and investor judgment & decision-making). It’s all out there in journals like The Accounting Review, The Journal of Accounting and Economics, Contemporary Accounting Research, etc., and it is all peer-reviewed. And if it hasn’t been peer-reviewed yet, its typically sitting on SSRN for anyone to look at.
You just really have no clue at all about anything…
@Andre Kenji:
What you’re pointing out is explained neatly by Sturgeon’s Revelation: 90% of everything is crap. That pertains to academic publications as much as it does to what appears in blogs.
An interview with “the quotable” Alan Kay cuts at these revolutions a bit differently:
@Dave Schuler: My point is not regarding crap. But that being a tenured university is a pretty good job and that the professors have an incentive that people do not read their work, not the opposite.
@Andre Kenji @Dave Schuler: The crap part is likely true. The lack of desire for being read, not so much. Aside from ego, the currency of success as an academic researcher, aside from simply being published, is being cited. Ideally, you want your piece to be “must cite” in the area.
The reason pieces aren’t widely read is because the nature of academic publishing, at least in the disciplines I’m familiar with, is that they’re hyper-specialized and thus of interest to a relatively narrow sub-sub-subfield of one’s discipline. Popularizing, broad, cross-disciplinary works are viewed suspiciously by most academics.
There’s also the difference between publishing in a journal devoted to cutting edge stuff vs. journals that do reviews of the fields. Anyone can submit to the former; the latter are more likely to be invited articles from people known in the fields. (In physics, this would be the difference between publishing in Phys. Rev A and in Reviews of Modern Physics.)
Interdisciplinary journals are usually not treated with the same amount of respect, mainly because it’s easier to bluff your way through peer review. It’s only when something stops being “interdisciplinary” and considered a new field in its own right that the quality of the journals in that field can be appreciated…
I’ve also seen journals that used to be the seminal journal for the field, but then refused to specialize and ended up as the dumping ground for anything connected with the field that wasn’t suitable for the specialized journals. Again, the cachet of publishing there went down…
@James Joyner:
That´s to the people that WANTS to be a star academic. Many professors are satisfied with six digit income and a steady source of income.