The following is a guest post* by Aaron Garrett, associate professor of philosophy at Boston University. Professor Garrett recently became editor of the History of Philosophy Quarterly and asked if we could open up a discussion about reforming various aspects of article refereeing. I encourage people to contribute to the discussion and share their experiences and concerns as authors, referees, and editors.
(Some previous related discussions at Daily Nous: A Closer Look at Philosophy Journal Practices, Guarding the Guardians, Getting Credit for Peer Review, Deciding Which Papers to Referee, Reasons You Rejected a Paper, Making Journal Statistics Publicly Available.)
Ideas for Reforming Refereeing
by Aaron Garrett
I am currently the editor of the History of Philosophy Quarterly and in my few months running the journal I’ve been thinking a lot about how to make refereeing more efficient and more rewarding. I really liked Eric’s discussion of the upsides of refereeing. I would add another upside, implicit in Eric’s discussion. You learn a lot from refereeing papers, proposals, and book manuscripts (although not from every paper, book or proposal you referee).
Eric proposed an idea for rewarding referees – publishing referees’ names with a successful paper. This could be done formally or informally, asking referees if they want to reveal themselves to the author in order to be thanked in a note. My worry is that it rewards referees accepting papers when some of the most important work is done rejecting papers. If the response is “no need to worry, it’s a marginal reward” then is this the sort of reward that would have any effect?
I wonder what people think about sending the referees all the reports when the process is completed. And whether this should be anonymous or not? I’ve really found this helpful when I’ve refereed at journals that do it to get a sense of whether I’m in step with the other referees. Relatedly should the referees be apprized of the decision? Should they be sent offprints of pieces that are published? This all seems like a good idea to me – often one feels like the report goes into an abyss and one has little sense of whether or how it had an effect.
Also how do people feel about desk rejects? The advantage of the desk reject to the author is quick turnaround. The advantage to the journal is maximizing its limited and most important resource (along with good submissions) – the time spent by expert referees towards strong publications. The disadvantage is arbitrariness. Relatedly, do people feel a few comments are necessary with a desk reject?
Finally, authors are likely aware that some referees take a long time to turn in their reports (or never turn them in). A referee who goes AWOL is a disaster for the author, in particular when they are attempting to get publications ready for tenure or the job market. This is ramified when the paper is highly specialized and it is difficult to find an appropriate referee. Fortunately this is rare.
Authors may not be aware, though, that editors spend a lot of time waiting for potential referees to respond to their requests. Sometimes potential referees never respond at all and a week or two may be lost waiting to hear back before sending a request to another referee. One way of dealing with this is an automated system that asks referees to press a button in a link as opposed to responding to the editor. The hope is that this will aid potential referees in deciding whether to referee or not quicker. The disadvantage is it is impersonal and doesn’t give potential referees the honest sense of how much their contribution to the journal and to the profession is appreciated. Impersonality may be an advantage as well. Submitters may also prefer an automated and impersonal system. Another disadvantage to journals is that the systems are expensive.
I should stress that most potential referees are not like this at all! Indeed in my few brief months I have been impressed and heartened by how serious, thoughtful, quick, and non-vindictive the referees who volunteer their time to our journal are. And for the most part the reports are models of philosophical thoughtfulness and care, real windows into expert knowledge in a subject area. Reading many of them is an education. I wish there was some way to publicly praise some of the authors of these exemplary reports! There are a few I wish I could publish!
Anyway, any thoughts in general people have about ways to make refereeing beneficial for all involved are most welcome.