Blueroselady's Weblog

I wish you abundant happiness, health & wealth

How to be a responsible reviewer

leave a comment »

Disclaimer: All notes here were ctrl+c & +v from many resources, please see references for more.

# Start the review with one or two sentences summarizing the paper.
# Communicate clearly the strengths and weaknesses of the paper.

Allan Newell: When we get a paper to review, at the beginning we should always have as the default that we accept the paper. While reading the paper, we may start raising specific objections along the issues in the review form, namely the goals are not stated, the system is not well described, the approach is not novel or not validated, etc. etc.

Authors deserve detailed feedback in a non-hostile tone, regardless of whether you recommend rejecting, revising or accepting their work. Do the quality of review that you wish others would do for you. By agreeing to review, you take on the responsibility of doing a thorough job.

Good reviewing does not go unnoticed by editors, and can establish a useful reputation for you over time.

Plan to read the submission three times: the first to get a feel for the it, the second reading the paper in depth, and the third to actually mark it up.

Do not be afraid to reject a bad paper; i.e., do not feel obligated to accept a paper which you believe needs more work.

Authors may not explain themselves clearly, so the concepts, motivation, background, actual results and contributions do not come across well. The paper may require drastic proofreading. The research may be of low quality or not a significant contribution. The research may be premature to publish. The paper may have been “thrown together” (unsuccessfully) at the last minute to meet a deadline. It may not fit the characteristics of the intended journal or conference. It may too similar to other publications the author has made (though some journals will publish slight modifications of papers that have appeared in conference proceedings—check with the editor if you have any questions).

Does the author cover everything promised in the introduction?
Does the author provide adequate motivation?
Does the author provide adequate background information?
Are all descriptions clear? Are the tables and figures clear? Do they make sense on their own or only if one has read the text carefully? Are there too many? Would an additional table or figure help? Would an example help?
Are the research contributions clear? Are the contributions significant?
Is the approach clearly explained and well laid out? Does the author justify each of the points made?
Are the equations, algorithms, methods, experiments, and conclusions: correct, robust comprehensive and sensible?
Is the research properly ground in the literature?
Has the author expressed the limitations of the research and the author’s approach?
Has the author performed a complete analysis and drawn insightful conclusions?
Has the author described his or her future research plans? Is it clear where the research described in this paper will lead and what the next step will be?
Is the conclusion significant? Is it just a rehash of the paper? Does it provide new synthesis or insights? Does it leave the reader excited about the research, the research domain or the future?
Does the author use important, current and adequate citations? Are there too many citations? Are any irrelevant or insignificant? Are they at an adequate level for the publication (e.g., research oriented vs. trade journals; technical reports vs. journal articles)? Are there too many citations to the author’s own work? Can you suggest any missing citations the author may have overlooked?
How is the author’s writing style? Is it too “dense” to make sense? Does it keep the reader’s interest? Is it too informal? Note that an informal style in itself sometimes is very effective in getting a paper’s ideas across. Similarly, many authors use humor very effectively in research papers. Only if the informality or humor gets in the way, should it be discouraged. (On the other hand, there are certain fields which enforce very formal writing styles, in which an informal style is deemed inappropriate.)

general comments on the paper; and specific comments on individual passages in the paper.



Written by blueroselady

July 22, 2009 at 2:19 am

Posted in sciences, study

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: