planet.donarmstrong.com

Can I use comments from other people's prior reviews when reviewing a submission to a new journal?

I just had the dispiriting experience of receiving a paper to review from Journal B, that was unchanged from a prior submission to Journal A. The "dispiriting" part of the experience was that the paper was completely unchanged, despite a host of minor and major comments on the paper from all three reviewers for Journal A.

I ended up writing that I was disappointed that the authors had not seen fit to confront the bigger issues in any way, much less correct even the smallest and easiest errors; and then pasted in my previous review. What I wanted to do was paste in the expert reviews from the other two reviewers for Journal A, but I didn't feel like that was OK.

(If I get the paper back with some revisions, I'll reevaluate it in light of the Journal A reviews, too.)

I think the behavior of the authors is very questionable, too, and I hope they rethink this strategy. If your paper is desk-rejected by a hoity-toity journal without review, that's one thing; if reviewers put in hours of effort and give you detailed comments, you goshdarn well should put in an hour or two of your own time before resubmitting.

Why don't all journals always send all the reviews to all reviewers?

David Koslicki visited my lab yesterday, and I was reminded of the mash and MetaPalette situation from a few years back. Briefly:

I was a reviewer on both the mash paper (Ondov et al. (2016)) and the MetaPalette paper (Koslicki and Falush (2016)) and in my final review of MetaPalette I mentioned the mash paper enthusiastically. (Both were already up on biorxiv.)

At some point later on I sent David an e-mail to follow up on some suggestions I'd had, and we realized that he'd never received the text from my review of MetaPalette. He later told me that he thought that receiving my comments would have accelerated his research by a few months, by pointing him at a new area.

So why didn't mSystems send him the review text?!

(There are plenty of journals that are guilty of this. Nature Biotech is one that I've noted in the past.)

Isn't it irresponsible not to make some portion of the reviews public when the paper is published?

Peer reviews often provide important context that can help people understand why the paper is important and interesting. It's fine and dandy to say that that should all be in the final paper, but that's a hard task and often papers are space constrained (...for some reason).

I think journals should make reviews public along with the article.

The biggest argument against this is that it might take some work by someone to properly adjust reviews for fixes from earlier versions. A short term fix might be to have a box for "this is the part of the review that I would like to make public if this paper is accepted".

Why don't journals behave as if reviews belong to the reviewer?

I no longer review for PNAS, because they started including a provision that I couldn't make any part of my review available in any form, even anonymously. I can understand that they don't want reputation laundering (e.g. my previous behavior in posting reviews, which boosts my own reputation while also being a sign of my own privilege), but I see little harm in allowing it to be posted anonymously.

Journals sure are proprietary about work they didn't pay for. That's a bigger theme here, I guess :)

There is no conclusion other than that peer review seems really broken.

Anyway. Those are my ranty off the cuff comments for today.

--titus