Does Reviewer 2 deserve that terrible reputation?
As Jonathan Weisberg (Toronto) notes:
Reviewer 2 is accused of a lot. It’s not just that their overall take is more severe; they also tend to miss the point. They’re irresponsible and superficial in their reading. And to the extent they do appreciate the author’s point, their objections are poorly thought out. What’s more, if they bother to demand revisions, their demands are unreasonable.
First, he looks at whether there is any connection between the order in which referees are invited and their severity. It turns out the answer is no. Then he looks at whether the speed with which reviewers complete their reviews affect the severity of their reviews. Here there is an effect, but not one that matches up with the reputation: referees who take longer are less severe in their refereeing.
For further analysis, see Weisberg’s post here.