Behind the scenes: How a journal editor reaches a decision on a manuscript after peer review

Behind the scenes: How a journal editor reaches a decision on a manuscript after peer review

The previous part of this series of posts details the process by which an Associate Editor (AE) selects reviewers and sends out a manuscript for peer review. In this post, I will detail the sequence of events that lead from peer review to editorial decision making: how the AE procures the finished reviews, how he weighs the reviews, and ultimately comes to a decision on whether to accept or reject the manuscript or request a revision. The post also describes the process that a revised subordination goes through.

Dealing with delayed reviews

When I very first commenced working as an AE, I imagined that the bulk of the effort was in weighing up the reviews for a paper, and synthesizing these to come up with a careful, considered decision and rationale for it. However, this turned out to be a much less significant part of my work than I had thought. Indeed, it seems that much of the effort of the AE is in dealing with reviewers who have agreed to provide a review, but who fail to fulfill their promises; this involves reminding, cajoling, and even menacing sometimes. In the ideal situation, reviewers will perform their task within the allotted time (typically, six weeks to a few months), and supply a cautiously thought-out, clearly voiced review. Indeed, most reviewers do an excellent job in this regard, and I am truly grateful to them. However, there are many cases where things do not go after this outline, and more active involvement is required. Some journals may include a “due date” for each review (which can be set by the AE), and may automatically remind the reviewer as the deadline approaches and is passed. In addition, around the time of the deadline, I send a personalized reminder, as this is stiffer to disregard than an automated message. I do not keep detailed statistics, but while many reviews are received on time, it is a sad fact that a large number are late. A little tardiness is forgivable, but after more than a week, it starts to become a problem. Many journals strive to have a rapid turnaround time for submissions, and delayed reviews are the fattest obstacle to achieving this objective. In such cases, I have to keep an eye on which reviews are late, and send reminders to reviewers, requesting that they make good on their promise, and supply their review. However, the pressure that I can put on the reviewer is limited: I can send increasingly plaintive requests, or express my dissatisfaction or anguish at the continued delay; I can attempt to provoke guilt or regret in the reviewer; but there are few direct deeds I can take against the tardy reviewer. Persistence is my only weapon. In a few cases I have given up on receiving a review when the other reviews received were sufficient to reach a decision.

Checking the quality of reviews

When I do receive a review, I read it cautiously, and check that there are no visible problems with it. Problems in reviews are uncommon, but periodically it may be clear that the reviewer’s standards are not calibrated for the target journal (too harsh, or too lenient); or that the recommendation does not align with the content of the review (e.g. many major flaws highlighted, but an “accept” recommendation). Reviews can sometimes be improved by clarifying what is expected from a revision, and ensuring that the discussion is as objective as possible. The AE can ask a reviewer to revise or elaborate their review. Very infrequently, there may be drastic inconsistencies across the assessments of different reviewers, and these are usually resolved through an email discussion initiated and mediated by the AE.

Making the final recommendation

When there are sufficient reviews for a paper, I can make a recommendation. The typical number is three, but more or fewer is possible. I am glad to recommend rejection for a paper on the basis of two reviews which agree on this outcome, or even one in extreme cases. For a positive recommendation, I choose to have received three reviews, even if they are not unanimous. Collecting four reviews is reasonable (and acts as insurance against one reviewer failing to submit the review); more than four is unusual except for very selective journals. I usually find it fairly swift to make a recommendation: reviewers often agree on the general level of quality and interest in a conformity. Some normalization is needed based on the standards of the journal, but in general, it is a fairly quick task to weigh the comments and scores of the reviewers and reach a recommendation. The process is guided almost exclusively by the reviews–my opinions of the paper carry almost no weight at this point.

The very first question is a binary one: Is there any prospect of publishing this paper in the journal? Does it demonstrate enough potential and interest? If not, then the recommendation is to “reject” the paper. This recommendation is accompanied by a justification, summarizing the reasons for rejection: I identify the main reasons from reviews that led to the recommendation. My recommendation may or may not include encouragement to submit to another journal, depending on whether the subordination was ultimately judged out of scope or below threshold for my journal. The authors may appeal a reject decision, either to the AE or the EiC, but without evidence of serious unfairness this is unlikely to alter the outcome. A rejected paper is sometimes resubmitted to the same journal, after some revisions. Most journals will attempt to catch this and either reject outright, or assign it to the same AE to treat. If the paper is not rejected, there are three possible recommendations: “accept (as is)”, “minor revision”, and “major revision”. The exact semantics of this vary depending on the journal, but as a rough guide, the resubmitted manuscript for a “major revision” decision will be returned to the same reviewers for their opinion on the fresh version; a minor revision will be scrutinized by the AE; and an accept will stir straight into the publication queue.

However, the AE has a lot of leeway: a minor revision may be sent out to reviewers; and a major revision may be sent only to a subset of reviewers, or fresh reviewers may be added. I won’t spell out all the situations that can arise, but the underlying issue is the same: before providing an “accept (as is)” recommendation, I want to be certain that the paper represents a sufficient contribution for publication in the journal. When the reviews indicate some notable questions or concerns, I want to be assured that these are suitably addressed before recommending the paper for publication. Sometimes I can do this myself (based on the revised subjugation, and any cover letter or list of revisions, and comparing these to the original reviews); or I may seek the opinion of the original reviewers on such questions.

Difference inbetween a recommendation and a decision

You may notice that an AE makes a “recommendation”, not a “decision”. This is deliberate terminology: it is the Editor-in-Chief (EiC) who makes the decision, whereas the AE merely recommends an outcome. I will let you into a secret: I have not encountered cases where the EiC’s decision did not go after the recommendation of the AE, albeit this does happen. I find that this is a useful way of thinking about the process. It reminds me that I have to justify my recommendation both to the authors and to the EiC; I am not making decisions at my caprice. Once I submit my recommendation on a paper to the EiC, I can again sit back: my work – for now – is done.

What happens to papers that need revisions?

For revisions, the process starts over again – selecting reviewers, obtaining reviews, and making a recommendation. Typically, the AE invites the same set of reviewers, albeit there is the option to add fresh reviewers (if extra input is needed), or drop some (for example, if they were entirely pleased with the previous version). There can be numerous rounds of revision, but if major issues remain after a very first revision, it is common to budge towards a reject. Once a reject or accept recommendation is reached, the AE’s involvement with the paper is concluded.

With this post, the description of the AE’s role and involvement in the editorial decision making process is accomplish. In the final part of this series, I will provide a few recommendations for authors and reviewers, which can help improve journal processes and make the publication process a smoother practice for all parties involved.

This post is a modified version of the article What does an Associate Editor actually do? originally published on the website of the Association for Computing Machinery, Special Interest Group On Management of Data.This post has been modified and republished with the permission of the author, and is the third part of a series. Stay tuned for the next segment of this series.


Related video: Job Interview Tips: Tell me about yourself. Example of a good answer.

Leave a Reply

Your email address will not be published. Required fields are marked *