Seth Mnookin laments a series of embarrassing failures in science writing in recent months but rejoices in the rich dialog that followed.
After a rehashing of the much-discussed Jonah Lehrer saga and pointing to some absurd pseudo-science that has made its way into the highest levels of mainstream American journalism, he points to what to me is a far more damning problem:
Wednesday, a team of researchers based in France published a paper in PLOS ONE titled “Why Most Biomedical Findings Echoed by Newspapers Turn Out to be False: The Case of Attention Deficit Hyperactivity Disorder.” (The paper’s authors were intentionally evoking the title of John P. A. Ioannidis’s groundbreaking 2005 piece, “Why most published research findings are false,” which built off of his earlier JAMA paper, “Contradicted and Initially Stronger Effects in Highly Cited Clinical Research.”) After examining every newspaper report about the ten most covered research papers on ADHD from the 1990s, the authors were able to provide empirical evidence for a troubling phenomenon that seems to be all but baked in to the way our scientific culture operates: We pay lots of attention to things that are almost assuredly not true.
That might sound crazy, but consider: Because it’s sexier to discover something than to show there’s nothing to be discovered, high-impact journals show a marked preference for “initial studies” as opposed to disconfirmations. Unfortunately, as anyone who has ever worked in a research lab knows, initial observations are almost inevitably refuted or heavily attenuated by future studies — and that data tends to get printed in less prestigious journals. Newspapers, meanwhile, give lots of attention to those first, eye-catching results while spilling very little (if any) ink on the ongoing research that shows why people shouldn’t have gotten all hot and bothered in the first place. (I have a high degree of confidence that the same phenomenon occurs regardless of the medium, but the PLOS ONE study only examined print newspapers.) The result? ”[A]n almost complete amnesia in the newspaper coverage of biomedical findings.”
So, to summarize: one of our biggest stars was revealed as a fraud; publications that should be exemplars of nuanced, high-quality reporting are allowing confused speculation to clutter their pages; researchers and PIOs are nudging reporters towards overblown interpretations; and everything we write about will probably end up being wrong anyway — not that we’ll bother to let you know when the time comes.
But he notes that the Internet generally and blogs and social media in particular make it easier than ever before for the good science to get out there:
And yes, the ENCODE coverage highlighted some of deep-rooted flaws in how we value and communicate about science - but the snarled, labyrinthine debate also highlighted the incredible opportunities available to anyone interesting in reading, or writing, about complex scientific issues. Genetics is a subject I know precious little about — and one I hope to write about in the future. Five years ago, it would have been difficult to know where to start. Today, I turned to Princeton genomics and evolutionary biology professor Leonid Kruglyak‘sTwitter stream. Among the many places that directed me was biochemist Mike White’s posts at the Finch and Pea and evolutionary biologist T. Ryan Gregory’s posts on ENCODE at his blog, Evolver Zone: Genomicron. Once I began pulling on those threads, they lead me to computational biologist Sean Eddy’s “ENCODE says what?” post at Cryptogenomicon, the 4,900-word, “My own thoughts” post that Ewan Birney, the lead scientist on the ENCODE project, put up simultaneous to the ENCODE papers’ publication, and Birney’s response to the reactions/backlash that ensued.
It wasn’t until I sat down to write this post that I realized that those are alldocuments written by people who are not only working scientists but also experts in the fields in question. When I began searching out work by science writers, I found subtle, sedulous pieces like Ed Yong’s “ENCODE: The rough guide to the human genome” and Brendan Maher’s “Fighting about ENCODE and junk.”
The end result of all of my reading was manifold: I now have a good grasp of the ENCODE project; I’m aware of some of the big issues facing genetics; I understand why the initial coverage proceeded the way it did, why that coverage was criticized, and how to avoid similar mistakes in my own work in the future; and I have learned of, and in some cases made contact with, a range of dynamic scientists dealing with these issues.
The same can be said for most any field of endeavor: A lot of crap gets out there but virtually all of it is thoroughly debunked by experts who simply had no access to the public debate a little over a decade before.
Alas, I would argue that the latter in no way makes up for the former. For every Seth Mnookin out there joyously seeking out the truth, there are 10,000* consumers who never go beyond the “Good Morning America” coverage. Once stupid factoids get out there, they become permanent truths for some significant portion** of society. So, while this is the greatest time in the history of mankind for wonks, we seem to be firmly into the post-truth era for the vast bulk of information consumers.
__________________
*I’m sure there’s a study out there with this very ratio, but I simply made it up for the purposes of this essay.
**I’m too lazy to make up a percentage here. Go with 27% if you require more specificity; it’s not implausible.
