Don't be nasty or chastising (e.g., "the author is under a terrible misassumption…"; "the author failed…")
Imagine that you will be reading your review to the author in a face to face meeting
In general, do not criticize papers on the grounds of external validity (e.g., "does this generalize to all managers?")
Make 3-4 main points and then make a list of your small, picky items
A good review is between 1-3 pages
Things to look for:
Is there theory in this paper? Or, is it a fishing expedition?
Is there information gain from this paper or is it a conceptual replication? Obvious extension of theory?
Are the results meaningful and worthwhile? Or so obvious as to be manipulation checks?
Are the operationalizations true to the conceptual variable?
Are there confounds?
Are there any process measures to support what the authors believe are the psychology?
Are there demand characteristics?
Never say, this should be published or not, etc. (that is the editor's job)
It is NOT appropriate to cite papers that you are reviewing
It is not cool to let someone know you are reviewing his/her paper at any time before or after, no matter what your current relationship with this person
Writing in margins is OK
But don't just say, "be clear"
On the cover page, make 2-4 main points; put details in margins
Get to the point without a lot of fluff and apology