opinion

‘Anonymous’ peer-review – a cautionary tale

Some time ago I received a grant proposal to review. Its was from an eminent professor.

I was extremely disappointed, on numerous accounts, with the quality of the proposal. In short I thought it relied very heavily on the authors standing in the community and only paid lip service to the science. And so (naively) safe in the knowledge that my review would remain anonymous, I gave a frank assessment of it.  I advised the panel to reject the proposal.

Satisfied that I had done my job I gave it no more thought until, a few months later, I received an email from the proposal’s author. It started …

I have had a review – see below – for a pending application with [research council/charity]. It is of the “unhelpful” type the community gets, and looks as if it from an unsympathetic, uninformed, prejudiced non-expert – the other reviews were all very positive, but I now have to answer the points.

What followed was a spurious query about some detail in one of my papers, which he/she apparently needed to help address the comments I’d given as an ‘anonymous’ referee.

It seemed to me that the real reason for the email was to tell me off.  But how could this happen? How could the author have found out that I had refereed the proposal? So I sort advice from senior colleagues who had sat on research council/charity committees. And I was shocked by what they told me. It seemed that it wasn’t unusual for the committees to know who the referees are, but even worse committee members were often happy to share this information with eminent grant applicants.

Since this episode I’ve been in a quandary.  What should I do when asked to review a proposal from a high profile scientist? Should I be truthful about what I think, risk being found out and so make enemies? Should I lie and write glowing reviews based on who the applicant is and not what is proposed? Or should I refuse to review applications from the higher echelons of community? To be honest I’m still at a loss.

There’s more. The email went on …

Also, I am getting personal comments from others in the field [lists of other eminent names] to ensure that the [research council/charity name] panel does understand the importance of the approach, to help all of us doing [technique], and not create a negative and highly damaging impression of [technique] at their panel from this kind of review – it playing right into the hands of the [another technique] community which is well represented on this panel.

So it seems that, if you have the connections, its acceptable to seek one’s own referees and lobby the panel with them.

The whole episode has left me with a very bitter taste and a complete lack of confidence in the validity of the  peer-review system.

Finally, this all happened sometime ago, and since then I’ve thought long and hard about whether I should make the incident public. After all, pointing out these goings on is hardly going to make me any friends. But I’m convinced that giving an honest assessment of an application in a peer review has already damaged my chances of getting grants. Therefore I wanted to share this cautionary tale with early career scientists who may still believe in the system.

P.S. The proposal was funded.

By March 28, 2014 24 comments opinion, science policy

Bloggers are time wasters

Because we are aren’t we? Really what’s the point? We should be off writing grant proposals, research papers, and (if there is anytime left) maybe do a spot of teaching. Blogging’s not going to get us anywhere, nobody takes it seriously so ,frankly, let’s all just pack it in and get on with some proper work. And if by chance we do stumble across something important (possible plagiarism or fraud maybe) then really we’d best tell the big boys and they’ll sort it out. If we really feel the need to write something about it then we can report the situation once its all sorted out.

Message received and understood.

At least that’s what some members of the community would have us do. Take the commenter (going by the name of bloggersaretimewasters) who responded to Chembark‘s eloquent rebuttal of ACS Nano editorial attempt to put bloggers in their place.

I took the liberty to check out your website at Saint Louis University. I am not sure what the requirements for tenure are in your department, but I can assure you that if you keep up the meager publication output you have had so far (despite working in excellent labs at outstanding universities) you would find it difficult to obtain tenure at a serious and reputable University (though you would likely have a good shot at a community college).
I have also cross referenced the publication output and credentials of several other “bloggers”, like yourself. What you all have in common, besides the self-proclaimed notion that you are doing the community a huge favor by uncovering so many frauds left right and centre is… a modest publication output and an overall “low” scientific profile. By low I don`t mean that you keep a low profile out of modesty, mind you.
Bottom line. How about you people try to forge your careers through your own discoveries and original scientific inquiries, rather than try to get ahead by attempting to undermine the work of others? Ah wait, I guess many of you don`t do it because you are not able to. That`s too bad.
You are wasting a lot of time and resources… I guess it will not really hit you until you are denied tenure. It seems you have started your tenure track position very recently, so maybe you are still in time. Please consider this post as a wake up call, because the day your tenure is denied (through peer review, mind you, not by declared or anonymous bloggers) it will be too late… game over!

 

I’m sure this attitude is far from uncommon. The majority of the people who think along these lines probably don’t bother reading our “time-wasting” outputs so certainly wouldn’t bother commenting on them. But, in my experience, this is an increasingly archaic view point. I have a permanent academic position in a UK university. I was recently promoted to the Senior Lecturer (UK equivalent of Associate Professor) not despite my blogging and other engagement activities, but, in no small part, BECAUSE of them. And increasingly, certainly in the UK, the need for scientists and academics to effectively communicate their views and work to a wider audience is being recognised: Nature Chemistry highlights bloggers outputs in every issue and take a look at the success of The Conversation, a news site written entirely by academics and sponsored by top research Universities such as Warwick, UCL, and Bristol as well as agencies such as the Wellcome Trust. Or on a smaller scale Guru Magazine again written largely by academics and funded by The Wellcome trust.

Yes my publication list would probably be longer if I did less blogging. But the same goes for spending time with family. Maybe I should stop wasting my time with them as well?

By October 25, 2013 13 comments opinion, science news

In defence of DHMO

Chemophobia and the use of satire such as the old DHMO joke, to tackle it seems to be the topic of the week. I’m guessing (and maybe I’m flattering myself) that my ironic piece in the Guardian, in response to the now notorious Buzzfeeds article, may have added fuel to the flames. So here’s my tuppence worth.

The latest round of the chemophobia debate started with Chris Clarke’s post Do you know douchbags are full of dihydrogen monoxide? Chris clearly doesn’t like the use of the DHMO ‘joke’, as it

 ..mocks alleged “gullibility” in a way that dissuades the corrected from learning.

plus its as old as the hills (or glaciers), as he puts it,

I first heard the joke back in the end-1980s, back when kilometer-thick sheets of solid-phase dihydrogen monoxide occupied the Northern Hemisphere as far south as present-day Kentucky, it got old fast.

Janet Stemwedel at Doing Good Science doesn’t like it either.

Really, all the target of the joke learns is that the teller of the joke has knowledge and is willing to use it to make someone else look dumb.

and she goes on to say

 ..there are instances where the dihydrogen monoxide joke isn’t punching down but punching up, where educated people who should know better use large platforms to take advantage of the ignorant.

I deployed DHMO in the Guardian (a large platform)  in an attempt to quench chemophobia and so even if Chris and Janet’s comments aren’t levelled directly at me, then very similar ones have.  Here’s my defence (and thanks to  for his).

The DHMO joke (and similar) is a clear example of reducto ad absurdum.  It use makes for a powerful argument. And the fact that the joke is old serves to flag up any satire. I thought that this plus the multiple links throughout and the increasingly ludicrous statements I made should also have made it clear that my piece was satire. I hold up my hands now and admit that it may not have been obvious enough (but then even more extreme examples of satire, in the Guardian, have recently been taken seriously).

I did not set out to punch down, up or any other which way. My aim was merely to demonstrate that through over extrapolation and application of the wrong spin any chemicals can be made to sound dangerous. I thought this was obvious. Admittedly it might not have been immediately so, that much is apparent from the comment’s thread which is littered with people who were taken in but then later realised the irony. What is also apparent was that the majority of those that did not realise it was a parody DID release that the arguments I appeared to use were garbage (I received plenty of abuse from this crowd). There was also a fair amount of concern that I was punching down (although that phrase wasn’t used). This criticism I take seriously. BUT after a trawl through the comments (donning my thickest skin first) there appeared to be little evidence of the punched.

In short, there were those that got it and thought it funny, or not, either way no harm done. Then there were those that didn’t get the joke, but got the science, no harm done here either (except to me who had to deal with some fairly vocal trolls). And those that thought it an in-joke which did more harm than good, but I don’t see much evidence of the harm.

Which all in all leaves me to conclude that we may be underestimating the intelligence of the audience. They aren’t abused and ignorant, they get the science, if not the joke.

 

 

By July 16, 2013 2 comments opinion, Uncategorized

When is a thank you just not enough?

I bet there is an interesting back story to this little episode. In January a neat communication appeared in JACS  describing “Small-Molecule Inducer of Beta Cell Proliferation Identified by High-Throughput Screening“. Basically the authors have induced the growth of cells that produce insulin, so opening up a possible route to cure Type 1 diabetes. Intersting enough, but not the subject of this post. I’m more concerned with the correction published last week. It seems that 20 authors wasn’t quite enough. Eric C. Peters probably wasn’t happy about being left languishing in the acknowledgements and having to be content with a thank you for experimental support.  So come March he got promoted to the author list, leaving John Walker left all alone in the acknowledgements. You’ve got to feel sorry for him (unless the authors are referring to a whisky at the end of the day), how come he’s the only one left with a hat tip and no place on the front page? Especially given that its difficult to imagine what sort of contribution would warrant an acknowledgement on a communication’s worth of work (although granted that in this case there is and additional 12 pages of supporting info) as opposed to less than a 5% share of the author list.

Anyway this rather unusual correction got me thinking. What do you have to contribute to a study before you are entitled to a place on the paper’s author list? Or when does an acknowledgment suffice? My thinking is that anyone who’s made a vital contribution should be named as an author, this includes all sorts of technical support. For example I’ve worked on plenty of projects where lab technicians have prepared protein samples from bacterial cultures. In this case I’d say its pretty clear cut, the technician spent several days making samples for me to do my measurements on, so they get to be an author. However, I know of plenty of places where the same technician would have to make do with an acknowledgement (if that). Maybe the argument is that he/she hasn’t made an intellectual contribution and are merely conducting a routine role, but the same thing could be said for many of the procedures carried out by grad students.

Then what about the flip side? When is a spot on author list too much?

By March 9, 2013 1 comment opinion, Uncategorized