Comments on: Reviewing a Journal Paper - Guidelines http://windowsil.org/2006/09/11/reviewing-a-paper-guidelines/ Your window into the Wireless Systems Innovation Laboratory Sun, 27 May 2012 13:08:42 +0000 http://wordpress.org/?v=2.0.3 by: WNCG Blog http://windowsil.org/2006/09/11/reviewing-a-paper-guidelines/#comment-80482 Tue, 25 Jan 2011 02:06:58 +0000 http://windowsil.org/2006/09/11/reviewing-a-paper-guidelines/#comment-80482 [...] Jeffrey G. Andrews, adapted and in some cases outright plagiarized from Prof. R. Heath’s guidelines .  The following should be interpreted as the opinions of Prof. Andrews and his alone. [...] […] Jeffrey G. Andrews, adapted and in some cases outright plagiarized from Prof. R. Heath’s guidelines .  The following should be interpreted as the opinions of Prof. Andrews and his alone. […]

]]> by: Jeff Andrews http://windowsil.org/2006/09/11/reviewing-a-paper-guidelines/#comment-6451 Tue, 08 May 2007 22:24:24 +0000 http://windowsil.org/2006/09/11/reviewing-a-paper-guidelines/#comment-6451 Nice blog, very detailed and I agree with all of it. I would, however, add one following opinion. As an editor, I have found that a significant minority of papers (1/3 or more) aren't even at the standard that I think they deserve a serious review by 3 experts in the area. They are poorly written and have no or virtually no novelty, and show naivety about research. Usually, it is easy to identify such papers by reading the abstract and introduction and seeing that there really is no contribution over prior work, many key references are missing, English is really bad, etc. In these cases, I do not believe a student should spend more than an hour on the review. The rest of the paper should be glanced through, and problems with the paper still need to be documented, but you don't need to spend a whole day tearing the paper apart. This is a waste of time. In these cases, a 2 paragraph decision is sufficient. Nice blog, very detailed and I agree with all of it. I would, however, add one following opinion. As an editor, I have found that a significant minority of papers (1/3 or more) aren’t even at the standard that I think they deserve a serious review by 3 experts in the area. They are poorly written and have no or virtually no novelty, and show naivety about research.

Usually, it is easy to identify such papers by reading the abstract and introduction and seeing that there really is no contribution over prior work, many key references are missing, English is really bad, etc. In these cases, I do not believe a student should spend more than an hour on the review. The rest of the paper should be glanced through, and problems with the paper still need to be documented, but you don’t need to spend a whole day tearing the paper apart. This is a waste of time. In these cases, a 2 paragraph decision is sufficient.

]]> by: Robert Heath http://windowsil.org/2006/09/11/reviewing-a-paper-guidelines/#comment-73 Sat, 16 Sep 2006 00:16:16 +0000 http://windowsil.org/2006/09/11/reviewing-a-paper-guidelines/#comment-73 Regarding the normalization issue, I think you have to go for your gut instinct for journal papers. For conference papers, usually I get 5-20 to review at a time so I have a good feeling about the top versus the bottom and can calibrate accordingly. You have to be careful. Sometimes there is the temptation to give papers that are well written and easy to understand a low score because it is "obvious" when a paper with the same technical content but more obscure would be more highly rated. Regarding the normalization issue, I think you have to go for your gut instinct for journal papers. For conference papers, usually I get 5-20 to review at a time so I have a good feeling about the top versus the bottom and can calibrate accordingly. You have to be careful. Sometimes there is the temptation to give papers that are well written and easy to understand a low score because it is “obvious” when a paper with the same technical content but more obscure would be more highly rated.

]]> by: Bob Daniels http://windowsil.org/2006/09/11/reviewing-a-paper-guidelines/#comment-72 Thu, 14 Sep 2006 21:22:45 +0000 http://windowsil.org/2006/09/11/reviewing-a-paper-guidelines/#comment-72 One of the problems that I have with reviews is that the review scale is not normalized. Most review forms have criteria which you are supposed to rate 1-4 where 4=groundbreaking research, 3= solid contribution, 2=average work, 1=no contribution. Personally I rarely award a 3 and have never rewarded a 4. In my opinion 4's should be saved for contributions of the magnitude of Alamouti's diversity scheme (probably wouldn't ask a student to review such a paper). 2 would represent your typical incremental paper (what I tend to give) and 3 lies somewhere in between. 1 would be for all papers whose contribution is not worthy of publication. However, after reviewing papers for conferences you'll see many 4's and 3's. Sometimes I think that papers are accepted mainly because the reviewers' scales were more generous than the reviewers of other papers. I don't think there is anything you can do about this, but it's frustrating! One of the problems that I have with reviews is that the review scale is not normalized. Most review forms have criteria which you are supposed to rate 1-4 where 4=groundbreaking research, 3= solid contribution, 2=average work, 1=no contribution. Personally I rarely award a 3 and have never rewarded a 4. In my opinion 4’s should be saved for contributions of the magnitude of Alamouti’s diversity scheme (probably wouldn’t ask a student to review such a paper). 2 would represent your typical incremental paper (what I tend to give) and 3 lies somewhere in between. 1 would be for all papers whose contribution is not worthy of publication. However, after reviewing papers for conferences you’ll see many 4’s and 3’s. Sometimes I think that papers are accepted mainly because the reviewers’ scales were more generous than the reviewers of other papers. I don’t think there is anything you can do about this, but it’s frustrating!

]]> by: Edward http://windowsil.org/2006/09/11/reviewing-a-paper-guidelines/#comment-71 Thu, 14 Sep 2006 06:26:43 +0000 http://windowsil.org/2006/09/11/reviewing-a-paper-guidelines/#comment-71 >Do not rely just on the list provided by the authors I totally agree with you! My experience is that some authors usually do self-reference, i.e. cite their own works for the topics / areas of the manuscripts only ... Sometimes ... there is even an overclaim on the title of the manuscript, which is very bad ... >Do not rely just on the list provided by the authors

I totally agree with you! My experience is that some authors usually do self-reference, i.e. cite their own works for the topics / areas of the manuscripts only …

Sometimes … there is even an overclaim on the title of the manuscript, which is very bad …

]]> by: Seijoon http://windowsil.org/2006/09/11/reviewing-a-paper-guidelines/#comment-69 Tue, 12 Sep 2006 19:02:58 +0000 http://windowsil.org/2006/09/11/reviewing-a-paper-guidelines/#comment-69 I think that this guideline will be also helpful to write a paper because authors can see through the reviewer's viewpoint. I think that this guideline will be also helpful to write a paper because authors can see through the reviewer’s viewpoint.

]]>