Peer review and the new media
I attended a Macintosh Users Group recently. (Yes, I am a Mac user.) This meeting was unusual. Natali Del Conte, a tech writer for CNet and CBS, was the featured speaker. She talked about being authentic, social networking, and how technology has changed how we get information. She argued that gone are the days that information is simply pushed to us and that Walter Cronkite is the most trusted name in news.
A detail I have come to think more about is what we know. I remember that stomach acid was thought to cause ulcers, but Marshall and Warren have received a Nobel prize for their discovery of the role of Heliobacter pylori and its role in peptic ulcers.
Mitch (Feb 08) and azmanam (Feb 02) have posted on congressional misunderstanding of science and false or poor science reporting, respectively, but I don’t think chemists are as cognizant of the accuracy or correctness of textbooks or peer reviewed papers. There have been a few cases in which errors have entered the chemistry world.
I wrote the the archivist at the Oregon State University Library inquiring about whether there was any correspondence regarding a paper Pauling published (there wasn’t). I wondered what the referees may have said. Now, I have been thinking how this is like the comments to a blog post. One of the really interesting things about the new media, is errors can be pointed out. They can be argued and open to everyone.
I have been thinking about how our ideas of atomic theory have evolved. In doing so, I have been reading a fascinating series of transcripts from recordings deposited at the Center for History of Physics of the American Institute of Physics.
It was interesting that some people thought Niels Bohr had confused the literature with his papers. However, overall, what I liked was how these transcripts contained the personalities of the scientists, their interests, and in some cases their ideas (or biases) about topics being discussed. I felt these transcripts from leading scientists were like our modern internet (although generally without the details of the science).
Our modern internet has no rules. It can be difficult to sort out the wheat from the chaff. We need to learn who to follow and who to ignore. Peer reviewed journals only give the filtered result. The referees reports are confidential. Comments are not published except in blogs. Now that science is moving toward electronic publication, would a new model for scientific publication improve the scientific world?
Just to note that this is not unusual in chemistry. Organic Synthesis has long provided a kind of review for a select set of procedures. I don’t foresee H.C. Brown’s papers becoming ignored, but independent reports could prove useful. Similarly, critical steps to improving yields would be helpful. You can find examples of this in the Organic Chemistry Forums.
Should online journals allow comments? Would it be useful? How could it be done? Would it be science?
PLoS allows for commenting on their journals.
http://www.plos.org/
Sure. Why not? Rules would develop gradually, via the community. e.g. if an author does not respond to a posted question that should probably not (yet) be taken as a sign of evasion. If there are lots of questions, a ranking system may have to be adopted (which many non-science sites have) to help weed out the crazies. PLoS One has been courageous setting this up so quickly.
A little self-promotion but anyone can comment on the latest chemistry articles from ACS, RSC, some Wiley at:
http://www.chemfeeds.com
Mitch
Nice, Mitch – that site has come on a long way since I checked it last.
I’ve managed to forget where I found this one the other day, I’m sure those following Mitch, Mattodd, or Egon will have seen it:
http://www.jcheminf.com/content/1/1/2
“Chemistry publication – making the revolution” which lays out pretty clearly the issues involved in open access and review … 1+ mil. articles per year. Check out the graph of publications vs patents:
http://www.jcheminf.com/content/1/1/2/figure/F1
Speaking broadly, chemistry is the most nebulous community to organize. Which is why OA probably has had little to no impact in chemistry,
If the point is to have access to the literature then authors should put their papers on their group websites. Hosting your own paper I hope falls within established copyright law and that way there isn’t any major publisher upheaval; everyone is happy, and chemists can go about their business. It would be an odd day indeed if you emailed the author of a paper for access to it and they didn’t send a copy to you. That being said I do have a paper in Nature Chemistry that I don’t have access to, as Berkeley doesn’t subscribe, and that is an odd experience.
Mitch,
I got a stern talking to on this issue a while back…unless things have changed and I’m misreading the JACS policy:
http://pubs.acs.org/page/copyright/learning_module/module.html#twenty4
“# May I post papers I have authored on my Web site, even though ACS holds copyright?
Only the title, abstract, and graphical material may be used without further permission of the ACS. You may provide a link to the paper using the Digital Object Identifier link (see Q. 26 below).”
“# Suppose I download a PDF file of an article from the ACS Web site in accord with permission listed on the site. May I then email that file to a colleague, or post it on a file server for my students/colleagues?
As a general rule, copying and distributing electronic material (be it a PDF file or other type of data file) that is protected by copyright is limited to the same extent that photocopying the article would be. Given the extensive use of institutional licensing for ACS materials, your colleagues can often access the same material, and giving them the appropriate journal reference is as easy (and accomplishes the same end) as giving them the file itself.”
I agree that things should work the way you suggest … I wish science in general was still a gentleman’s sport.
@Verpa I wonder how difficult it would be to change ACS’s stance on hosting your own papers? I can assure ACS it is in their best interest to have the young chemists not anti- large publishing houses. I’ll passively start looking into it.
My apologies to @orgopete, as we did kind of jack this thread in a different direction than his post.
@Mitch: Like you said before “On a long enough time line, the survival rate for everyone drops to zero.” or, to how difficult would it be, the great open source answer, ‘Not very, if you help.’
Though, in seriousness, as someone who has thought about this stuff more than most, how would one go about trying to sway the ACS? Facebook group them? A bombard of tweets? A thread in the non-existant JACS forum? A well written letter to the editor?
First we have to do what you’re doing and build the tools to change things.
@Mitch – meant that you and your posse on this site had thought about the future of the chemical community more than most. *failing friday*
@Verpa, which is why I said passively. It probably will take a combination of finding the right governance committees to be on, getting as many like minded individuals in ACS to agree, developing alternatives so ACS sweats, passive threats from distinguished chemists. It’ll likely take at least 5 years before ACS starts making some copyright exemptions, I trust the organism to sort itself out. A tweet/facebook bomb doesn’t seem like the best way.
10 years from now ACS will think it always gave authors the right to host their own papers and will be confused why people thought they couldn’t.
I looked at the website of some of the most famous organic chemists across the country. Of the 14 pages I looked at, 6 had direct links to PDFs and 8 did not.
Phil Baran at Scripps even notes on his publications page that they cannot provide the manuscript online due to copyright restrictions. However, other Scripps professors do provide direct links to PDFs.
http://www.scripps.edu/chem/baran/html/publications.html
@Verpa, thanks for the jcheminf note. That is what I was thinking about.
Re:other commentary options, I was thinking of how the new media is able to operate. If you print a paper, additional commentary becomes impossible without reprinting. As we get more and more information electronically, it does become possible to “reprint” and thus comments or corrections could be made.
@Mitch, I think I had encountered the http://www.chemfeeds.com, but I had forgotten. None the less, I always find it difficult to be aware or to spend the time to search for comments on any reaction. Even for simple typographical errors that are published a few issues later, they can easily be missed.
In grad school, one of the cumulative exams was to review a paper from a peer reviewed journal. No one passed this cum. The essence of the review was to read the data and to determine independently whether the authors interpreted the data correctly (they hadn’t). Later in my industrial career, I encountered an erroneous structural interpretation. Since I was working in that area, I had the resources to write a paper proving the correct structure. However, in doing so, the editor of the originating journal did not deem my paper worthy of their journal and when I did find another journal to publish, I was subjected to a much higher standard of proof than the original authors. Thus, I argue a fairly high barrier to printed commentary exists in printed journals and I doubt a correction of the paper in the cum was ever made.
Here, I agree with Verpa, that the ACS at some time in the future, should include commentary. This does trigger a different thought. If browsers or a reader script had a plugin to find chemfeeds, then something closer to what I was asking about would result. Perhaps the commentary would evolve its own guidelines.