Getting Credit for Peer Review
It is a great service to the profession to peer review articles, and service to the profession counts at most institutions towards tenure and promotion. But how much does peer reviewing count?
My sense is that the credit one gets for peer reviewing is disproportionately small compared to how important peer reviewing is for the academic enterprise, but it would be good to get more than just my sense of things. If you have a moment, please write in to describe how peer reviewing enters into the evaluation of members of your department (for merit raises, tenure, promotion). How does it enter into your department’s evaluation of job candidates? Are there minimum expectations for the quantity of peer-reviewing one does? Does the quality of the venue for which you are reviewing count? Does the review of whole book manuscripts count for more than the review of article manuscripts? Etc.
One possible obstacle to giving credit for peer review is a lack of a common understanding of how much is normal. While it’s possible for departments to keep track of the numbers internally, I don’t know if any do. Does yours?
In any event, local tallies do not provide a picture of what the norms for the discipline are. Enter Publons. The site was brought to my attention by Simon Evnine (Miami). According to its website, it “allows you to record, verify, and showcase your peer review contributions in a format you can include in job and funding applications (without breaking reviewer anonymity).” And from their About page: “We collect peer review information from reviewers and from publishers, and produce comprehensive reviewer profiles with publisher-verified peer review contributions that researchers can add to their resume.” Reviewers appear to have the option to post the text of their referee reports for published papers.
You can read an interview with Publon’s founder and some users at Nature.com. The site seems to be used mainly, but not exclusively, by academics in the sciences—so far.
Publons lists many philosophy journals, but a quick look revealed that only a handful of them had submitted reviews listed in their profiles. Are any philosophers using this? If so, please share your thoughts and experiences with it. More generally, I am curious whether philosophers think it would be good if Publons or something like it were used widely in the profession.
This is a really interesting and potentially valuable initiative. Thanks for calling it to our attention.
At my institution, peer-review counts toward one’s “Service” score, which forms part of the basis of merit-review and thus, in the long run, can make a small, indirect contribution to tenure-and-promotion decisions. I don’t think that there are “minimum expectations” for peer-review here, because it’s viewed as one component of “Service,” which comes in many varieties. If, say, someone did a lot of good committee work or community outreach, then the fact that they hadn’t reviewed any manuscripts this past year probably wouldn’t matter at all. (This is to say that service is fungible in a way that teaching or research is not, which I expect is true in most departments.) It’s up to individual faculty members to keep tallies of their reviewing activity and report it. I’ve never heard a candidate praised or criticized based on the part of their CV where they list their peer-reviewer activity.
I wanted to suggest a slightly different perspective about the opening assumption of the post. Peer-reviewing can be a great service to the discipline. But it can also be a great disservice. Good peer-review is a service. Bad peer-review is a disservice. A system that gives “credit” to people for peer-review should also be capable of giving them “discredit” too. This is hard to do in the context where nearly all peer-review is anonymous and most people unquestioningly assume that is how it should be.
Unfortunately, in my experience, much — perhaps most — peer-review falls into the “disservice” category. The discipline lacks a common understanding about what counts as “publishable,” which creates a lot of uncertainty, misunderstanding, confusion, and frustration in the peer-review process. What’s worse, many people in our discipline abuse the peer-review process because they adopt the severely misguided assumption that they, as reviewers, are entitled to act as a covert co-author or research supervisor. (Some people who do this might *think* that they’re being helpful.)
Without a serious, sustained conversation in the profession about the norms of peer-review, the situation is unlikely to improve. And as long as peer-review remains anonymous, and human nature remains roughly as it is, an alarming amount of “peer review” will continue taking the form of “peer abuse.”
In my experience, peer-reviewing is of marginal importance in tenure and promotion reviews. At best, it might count as service to the profession, but what most universities want to see by way of “service” is service to *them*. They want evidence that you are pulling your weight in faculty governance and they care very little about what you do for your profession. John Turri rightly notes that service is fungible, but no amount of peer reviewing will compensate for a candidate’s reluctance to do any of the chores in their department or on faculty committees.
This does not mean that people should give up on peer-reviewing. The reason for doing this is that other philosophers are doing the same for you when you are submitting articles to journals. A comparable case are letters for tenure and promotion cases, which are significantly more work than journal referee reports and for which people do not get compensated, either. The reason why you should agree to write tenure letters after you get tenure is that other people did the same for you when you were coming up. Of course, there is the question of how much you “owe” the profession, but that is a question that each of us gets to decide on our own. Nobody is being forced into peer-reviewing or tenure letter writing. As with requests for excessive university service, you can always say no.
I don’t think there is a problem here that Publons would solve. They probably just want to sell more advertising.
“How does [peer reviewing] enter into your department’s evaluation of job candidates?”
In my experience, peer reviewing has never come up as a consideration, either positive or negative, in reviewing job candidates at my university.
“Are there minimum expectations for the quantity of peer-reviewing one does?”
No. Faculty are expected to do service of some kind–peer reviewing counts as “service”–but there is no minimum expectation for peer reviewing.
“Does the quality of the venue for which you are reviewing count?”
Sure, I guess, insofar as it makes something that counts for very, very little count just a little bit more.
“Does the review of whole book manuscripts count for more than the review of article manuscripts?”
I’ve done book manuscript peer reviewing. My sense is that peer reviewing of manuscripts and peer reviewing of articles both count for very little.
I agree with Justin’s original point that peer reviewing is extremely important work for the profession while it is also work for which we receive very, very little credit. I am fairly certain that my university, which does not provide us with its criteria for merit raises, does not actually take peer reviewing into consideration when making such decisions. When we make decisions on tenure and promotion, it is taken into consideration as part of a faculty member’s overall service record, but it is not the sort of work that receives as much consideration as, say, membership on a university committee. This is despite the fact that there might be some peer reviewing assignments that take more work than being on some university committees.
I would advise junior faculty who are interested in becoming senior faculty, or graduate students or part-time faculty who are interested in becoming full-time faculty, to focus on their own research and avoid doing peer review. One receives very little credit for it. This is unfortunate, because it is often the young researchers in a field who will know the current research the best, and who would as a result be the best peer reviewers.
One of the many things I like about Thought is that we list our regular referees on our webpage. This gives much-deserved credit to those who serve. Perhaps it also provides an incentive to take the responsibility seriously, though I haven’t thought much about that.
I’ve been getting the Publons emails asking me to join. I just can’t think of a compelling reason I would need to do it.
Peer review activity is one of the things faculty at my institution are expect to track in our faculty activity reporting database. To my knowledge, it doesn’t count for much, but I suspect it is considered part of the somewhat vague tenure requirement that one is on the way to “establishing a national reputation.”
I can understand how it might be attractive to receive credit for peer review. However, given my hard work peer reviewing (I peer review over 20 or so articles a year, on average!) and furthermore the time I’ve spent on Publons (at least 30 minutes or so for each article I’ve reviewed, if you include the time I spend checking to see who has viewed that I’ve peer reviewed things), I feel like I have done more to get credit for my peer reviewing than others, but than I’m not getting credit for THAT. Specifically, and put as simply as possible, I am feeling a bit disenchanted by the fact that I’m putting in a lot of effort on Publons, to do exactly what they say to get credit for my peer reviewing, but I’m not getting credit for this. What I propose is a meta-Publons site that allows people to get credit for getting credit for peer review–that way, I wouldn’t be failing to get credit for what I’m doing to get the credit for the papers I’m peer reviewing.