At NEJLT, the review process has the following top-level steps.

  • An anonymised article is submitted for review in the journal template.
  • Editors are assigned to manage contact with the authors, and the review process.
  • Expert reviewers are found, to submit their reviews.
  • Once all reviews are in, a decision is made on the article.
  • Depending on the outcome, the article is published, sent for revisions, or rejected.

The process is explained in detail below, as well as how NEJLT manages anonymity.

Author anonymity

Author identity should be hidden from anyone responsible for the decision on an NEJLT manuscript. This is achieved through accepting only anonymised submissions, and obscuring author identity from reviewers and action editors. A separate managing editor handles contact with the authors where necessary. This is triple-blind reviewing.

This is based on both peer-reviewed research revealing a number of biases in academic review when facts about authors are revealed, and also experience in the NLP field with blind and non-blind area chairing.

Triple-blind can impose unreasonable constraints on resource-type papers, and so in this case, blinding requirements are relaxed a bit; there need be no “undignified mental gymnastics” in describing work on language resources that one has done. The goal is to protect, and not to punish, the author. However, a reasonable effort should be make to conceal names of authors from reviewers who may be unfamiliar with the resource or its context, including e.g. hiding some URLs, keeping author names off the title page, and avoiding referring previous work with pronouns like “my” or “our”.

Reviewer visibility

While during reviewing authors and other reviewers will have their identities concealed from each other, NEJLT adopts an open policy later in the decision process. After reviewing is done, reviewer names may be revealed both to other reviewers and also to authors. This follows indications that malicious reviews are less prevalent when reviewer identity may be revealed.

At NEJLT, we hope to foster constructive dialogue between authors and experts in the field, and remove the shields of anonymity is a step towards improving that dialogue, especially important in a field such as computer science where reviewing can be hypercritical. Revealing reviewer identity serves to improve review tone, while also removing biases during reviewer discussions.

Review workflow

Submissions are assigned a managing editor (ME), who desk-checks and deals with author contact, and allocates an action editor (AE) to make the decision and to find reviewers.

Papers must reach “minor edits” by the end of the 2nd review cycle. Possible cycle outcomes are:

  • reject (implicit 12-month embargo at NEJLT) - when there are flaws in core/most contribs;
  • major edits - flaws in contribs / missing fundamental parts;
  • minor edits (<15% of the content needs to change);
  • accept / polish+accept (<1% needs to change)

NEJLT maintains a large standing pool of reviewers, drawn from the NEALT community and from invitations to those who have reviewed responsively for the field in the past. We expect AEs to be responsive; in return, we limit their load, and will find new AEs if manuscript load increases. The journal should have as much transparency as possible: this means notifying authors as manuscripts move through the pipeline.

We give an estimated minimum reviewing time of about seven weeks, if everything runs perfectly. From 2020, the intended process, after receiving a submission, is:

  • Allocate managing editor
  • ME filters for desk reject, using BMJ procedures as guidelines if needed
  • ME checks paper type and if needed talks with authors about this
  • ME waits for an available AE
  • ME allocates AE and passes on anonymised reviews to AE, if given
  • AE accepts
  • AE finds reviewers, keeping the manuscript anonymised
  • Two reviews are received over the next month
  • AE finds additional review(s), if needed
  • AE writes meta-review on all reviews available, revealing (if supplied) past reviews to other reviewers
  • ME contacts authors with outcome
  • Authors given fixed window to revise (15 weeks for major/minor revisions; no fixed window for accepts)

The way that prior reviews are used is that they may be considered by AEs for the first decision, and are released to reviewers only after the first round of reviewing (details below). This is to avoid biasing our reviewers’ initial impression and thus compromising our quality.

Sometimes a manuscript should be rejected without review. This is called a desk reject. Desk rejects should be quick. These will be issued for manuscripts that are clearly out of scope; that are much below the required quality standard; that are duplicates or plagiarised; or that openly reveal author names. These are generally unpleasant to deal with, and we would prefer not to send any, so please read the guidelines before submitting anything, and contact us if you are in doubt.