Tips on how to do a good journal review for a research paper

A. In general...

1. Don't be nasty or chastising (e.g., "the author is under a terrible misassumption…"; "the author failed…")

2. Imagine that you will be reading your review to the author in a face to face meeting

3. In general, do not criticize papers on the grounds of external validity (e.g., "does this generalize to all managers?")

4. Make 3-4 main points and then make a list of your small, picky items

5. A good review is between 1-3 pages

6. Things to look for:

1. Is there theory in this paper? Or, is it a fishing expedition?

2. Is there information gain from this paper or is it a conceptual replication? Obvious extension of theory?

3. Are the results meaningful and worthwhile? Or so obvious as to be manipulation checks?

4. Are the operationalizations true to the conceptual variable?

5. Are there confounds?

6. Are there any process measures to support what the authors believe are the psychology?

7. Are there demand characteristics?

7. Never say, this should be published or not, etc. (that is the editor's job)

B. Morals and ethics

1. It is NOT appropriate to cite papers that you are reviewing

2. It is not cool to let someone know you are reviewing his/her paper at any time before or after, no matter what your current relationship with this person

C. Informal reviews (for colleagues, etc.)

1. Writing in margins is OK

2. But don't just say, "be clear"

3. On the cover page, make 2-4 main points; put details in margins

4. Get to the point without a lot of fluff and apology