It is that time of year again. No, not spring, its the beginning of the semester. As the semester is kicking in, so does the ever present flow of journal paper review requests. I get 2-3 review requests a week and often end up delegating them to my students, when they are of mutual interest. In this summary I provide some details about the reviewing process for journal papers.
Why should you review a paper? In brief, reviewing a paper provides:

  1. A window into the review process. When you review a paper you read a manuscript critically, and hopefully start to realize potential criticisms of your own manuscript.
  2. A privileged insight into state-of-the-art in research, new research tools, methods, etc. Remember that reviews are confidential - nothing learned can be used until the paper is published. If you find something new, write it down and put it away until the journal or conference version appears.

Reviewing is a duty of every active researcher. If you submit a paper to any journal, you are, karmic-ally speaking, obliged to review one or more papers in return. Note, this does not mean that you are obliged to respond positively to every review request. To be fair to the authors and editors, you should only accept papers that you can confidently review, either on topics you know or are willing to get to know. Feel free to say no if you have a good reason. Keep in mind that you should try to review papers in your research area as a means of keeping in touch with the research world.

Reviewing also gains visibility. Editors are potential sources of employment, for academic jobs, or reference letters. Writing good detailed reviews may lead to a postdoctoral position, job, or collaboration later one.

Now the hard part, actually reviewing a paper. This is a matter of personal taste. Many may not agree with my suggestions but, as an editor, this is what I would like to see from a reviewer.

Review guidelines

  1. A serious review should be at least 1-2 pages of single spaced text. Longer is generally better. In extraordinary cases, there will be a really good paper and you just can’t find enough to complain about. In most cases though there will be many issues.
  2. Your review is preferably written in plain text and pasted into the appropriate manuscript central window (or sent to the editor in an email). You can also use a pdf but sometimes pdf files have tags that can identify you. If possible avoid pdfs and this pitfall of being identified.
  3. Plan to spend anywhere from 2 hours - 2 days on a paper review. It depends on how well you know the area. If you don’t know the area you may have to do some background reading to see if the proposed research is novel.
  4. Always do your own check for novelty. Often authors do not cite important papers. Do not rely just on the list provided by the authors, especially if it is quite thin, i.e. less than 20 references.
  5. The most important ingredient of a review is critical insight. You must demonstrate a clear understanding of the paper and form a strong opinion. Your written review must reflect this.
  6. Typical composition of a review
    • Summary of the review. In this case you summarize the review in your own words and provide a recommendation. Typically this includes a summary of the key contributions of the paper (your opinion of the key contributions, not a restatement of the abstract) and some justification for your recommendation. Recommendations include
      • Accept - This is super strong and should only be applied to absolutely stellar papers on the first round of reviews. Can be used on subsequent rounds if paper has evolved to an acceptable level.
      • Minor revisions - This is quite strong. Usually used for the case where the technical content is sound but there are some issues that you believe can be addressed by the authors. Paper is typically accepted after minor revisions.
      • Major revisions - This is weak. Used for cases where you have some concerns about the manuscript that you don’t know if can be addressed, or if you have a lot of comments and you want to make sure they are all addressed.
      • Reject - This is the worst outcome for the authors. Used in cases where there are major flaws in the paper or it requires a major revision. Sometimes rejected papers are (and should be) resubmitted. Other times not.
    • List of issues in the paper with discussion. Typically this will take the form of a bulleted or numbered list of paragraphs. Numbering helps the authors respond to your comments in their reply. In each of these cases, I suggest that you include a summary of the issue and some discussion. For example “I don’t believe the derivations in equation (XX). A previous paper [ref to paper] showed that …” Examples of major issues include
      • Problems with writing
      • Problems with derivations
      • Errors in equations
      • References missing
      • Simulation results missing
      • Comparisons missing
      • Paper organization needs improvement
      • Problems with references (number, formatting, etc)
    • List of less critical concerns. This includes typos, formatting issues, grammatical errors, etc.
  7. Be careful to maintain your anonymous status when you review a paper. Be careful about citing your own work - it is often a dead giveaway. It is tempting when you read a paper related to your research but be very careful. It is especially important not to cite your own preprints, or preprints of others, that are not widely available on the web. I have had several excited reviewers cite many of my papers in their reviews for me. It puts me in an awkward position. Cite what is relevant. Only cite your paper if absolutely essential and then with some other papers as well.
  8. Avoid conflicts of interest. Never review papers co-authored by your advisor. Be careful about reviewing papers of close collaborators. If you can’t provide an unbiased review then you should pass on it, or at a minimum disclose to the editor.
  9. Sleep on the review and reread before submitting. Even if the paper is quite bad, don’t be too harsh. A review that is offensive to the authors is just that - offensive. It makes it harder for the editor and shows a lack of refinement. If the paper is poor, simply document this as best you can. Do not get emotional.
  10. Submit your review on time. It helps authors and reviewers if you can submit the review in the timeframe agreed.
  11. Check as much of the paper as possible. It is hard to rerun the simulations yourself but you should check the math in the paper and all other statements. Often steps are missing, proofs not quite correct, etc. Check everything. If you can’t check it, disclose this to the editor / author. Often the authors need to provide a better explanation to you, especially if this is your area.
  12. Don’t be afraid to reject papers. Often first-time reviewers write a very harsh review then proceed to accept the paper. This is hard on the editor. On the other hand, don’t be afraid to accept good papers. You will review papers that are quite good. It happens.
  13. On paper revisions, make sure the authors addressed your comments in the manuscript. Too often authors simply respond to the reviewer without changing the manuscript. Read all the reviews and check new parts of the manuscript. If your comments have been satisfied then accept the paper. If not then ask for another revision.

That’s all for now! Please post questions or comments and I will revise as appropriate.