Having enjoyed Faz’s discussion on the current situations of scientific publishing arms everywhere I think it is important to also restore a little faith in the community regarding Peer Review processes that do in fact work.
Last month I had the pleasure of meeting Nature’s Kendall Powell and discussing a few topics facing the scientific community at large. Most of the convorsation evolved around the current PhD glut and how to combat it but we briefly discussed peer review and she outlined the preventative steps that Nature takes to prevent publishing inaccurate or false information.
This largely goes back towards finding qualified groups of individuals (read; previously published and distinguished scientists) who a.) have the time to meet for peer review decisions and b.) are knowledgeable in the specific field that the publication is related too. For instance, a paper which may have found a new process in ribosome regulation would go to one of the Molecular cell Biology peer review sub groups [there are several].
This is very similar to the sort of peer review processes that occur for researchers who are applying for grants from a funding agency like the NIH, NHGRI, NCBI etc.
During this process, the application will go to the peer review department where conflicts of interest are identified by looking into their academic history to see if there is some sort of overlap - i.e. did the reviewer attend the same institution as the applicant? First the applications are identified to each sub group of ~20-30 established scientists in their field, mainly tenured professors or highly decorated scientists. At this time the potential application is made available to each of the members of the sub-committee and they read it and give it an initial scoring or will outright mark it ‘Not Feasible.’ For those applications that have some issues in language or need clarification these comments are made and are usually relayed back to the applicant so that they may correct it and progress through the peer review process. During Peer Review meetings in person, this person is escorted out of the room. Let’s be realistic, this is the modern age and researchers who volunteer or are appointed to peer review donn’t have time to fly to a centralized location to discuss the applications or publications. In the modern age we utilize technology and use teleconferencing. The pre-identified conflicts are moved out of the conference by a moderator and brought back in when the discussion for the publication or application is over and it is time to move on to the next one. If there is no immediate conflict of interest, the reviewer is honor bound to volunteer that they have a conflict so that they do not create bias in the process and will ask to be excluded from having an impact on the decision.
Reviewers will discuss how ambitious the application is, if they are on the right track, if they are making some additive improvemets to the field and if there may be an unseen problem with the application from the initial scoring event. Those applications that do not make it may be ripped to shreds at this time but don’t worry - you’ll never see this as the applicant. What happens is that constructive criticism is advised and then returned back to the author so that the next time they have a submission they’ll be better prepared and can get published / awarded the funds.
While discussing the similarities with Mrs. Powell, we agreed that the similarities are there in the thoroughness at least between funding agencies and a reputable journal such as Nature, however on the funding side additional metrics are used when looking at the applications.
Here’s a hint for those of you applying for funds, we’re all human. Show your ambition and knowledge of the project or the publication you are writing. When applying for a grant, the funding agency wants to know if you are going to go on and be an esteemed researcher in the field, if you’re planning on going into industry... don’t write that in your application!
This leads to the segway in that getting into academia with a PhD is getting more competitive since we have so many highly educated people entering career stage in life. Tenure track positions are getting more and more exclusive and there are the horror stories from Australia of PI’s holding their post-doc’s captive because they are quality researchers and not allowing them to progress. But this is a topic for another time.
Have faith fellow scientists, peer review may be broken in some publishing institutions but it should make you work harder for your suubmission to be published in a journal of high excellence! Don’t settle for a weaker publication just to ‘stat pad.’ Remember you’re colleagues are trained scientists and can spot the fluff from a mile away.