Stakeholder Refereeing for Controversial Ideas: Replies to Some Criticisms

I appreciate the responses, here and elsewhere, to my idea of using stakeholder refereeing as an alternative to the pseudonymous authorship policy planned by the Journal of Controversial Ideas.

Below are some criticisms of the idea and my responses to them.

Criticism 1. Justin’s idea is intentionally aimed at preventing the publication of controversial articles.

No it’s not. As I’ve said many times here on Daily Nous, I favor an intellectual community characterized by thoughtful disagreement.

Here are some positions I’ve taken, quite publicly, here on Daily Nous:

  • arguing in favor of demographic diversity in the profession in order to bring new perspectives and experiences into philosophy;
  • supporting an expansion of what counts as canonical philosophy, to learn about earlier differences and to see how disagreement may have been suppressed by cultural chauvinism;
  • fighting attempts at mob rule in academia, because the academic enterprise is about argument and evidence, not social media popularity;
  • skepticism about the “culture of fear” narrative in academia, in part because it inhibits minority voices;
  • opposing the “universities are overrun with political correctness” narrative in part because it encourages dismissive attitudes rather than thoughtful engagement;
  • criticizing obnoxiousness in the profession, in part because it discourages people from participating in public conversations.

All of these are about promoting more and better disagreement.

I’m not opposed to the publication of controversial scholarly articles. I’m just doubtful that creating an academic journal based around offering authors the cover of pseudonymity in order to publish such articles is a good idea.

Criticism 2. Stakeholders should not have a veto over whether an article that may offend them gets published.

My suggestion does not give stakeholder referees veto power over publication. Publication decisions would, as with other journals, rest with the editors. And as with the assessments made by ordinary referees, it is up to the editors to decide what weight to accord the reports of stakeholder referees.

If an editor judges that a stakeholder referee is using inappropriate criteria (such as agreement with the thesis) or is being purposefully obstructionist, and this cannot be resolved in a way that allows that particular stakeholder referee to continue to contribute to the process, the editor might reasonably weight that referee’s assessment less.

Some may feel that this gives the editor too much discretion, or too much responsibility. Perhaps there are better ways to strike a balance between stakeholder referee input and editorial power. Just keep in mind that any arrangement is going to have its flaws. The goal is not to find a problem-free solution, but to find a solution that has problems we can live with.

Criticism 3. Stakeholder referees would not favor the publication of high-quality controversial articles.

I envision a stakeholder referee as having several characteristics: (a) belonging to one of the stakeholder groups, (b) being an academic or an expert, (c) willing and able to approve for publication articles that defend or rely on theses they disagree with, provided the articles are of sufficient quality.

Most versions of Criticism 3 appear to operate as if (c) is not part of the understanding of who should be selected as a stakeholder referee. That may be owed to me not making that explicit enough in my original post. My apologies.

If we assume that Criticism 3 is made in light of (c), though, then it seems to be saying that despite (c), the stakeholder referees would be too restrictive. That may end up being the case. I don’t know. I suspect whether one thinks that’s so depends a lot on who one talks to, and how well one knows the types of people one’s talking about. I would guess that some commenters are inferring people’s refereeing tendencies from their comportment on social media, and that seems like a bad inference to me.

Criticism 4. There are no (or not enough) people who would be willing and able to serve as stakeholder referees.

The criticism here is that while there may be enough people who fit the criteria of (a) and (b), there aren’t enough people who fit those two criteria and (c).

I would bet that people who assert something like Criticism 4 believe that were they themselves asked to be stakeholder referees, they would be willing and able to approve the publication of articles that defend or rely on theses they disagree with, provided the articles are of sufficient quality. But if they can do it, why can’t others? If people who agree with them on certain matters can do it, why can’t people who disagree with them on those matters do it, too?

This will come as no surprise to regular readers of mine but I suspect that the availability heuristic is at work here. When thinking of controversial views and those who might be bothered by them, people are thinking about political opponents, and when we think of our political opponents, who comes to mind? Generally, the loudest, most vocal, most visible, most dogmatic of them. Such people might indeed make lousy stakeholder referees, and editors would probably be wise to avoid them. But such people are not properly representative of the stakeholder groups of which they are members. Are you a reasonable person? Do you know other reasonable people? Of course. Why believe there aren’t such people in various stakeholder groups?

Criticism 5. People who would be willing to serve as stakeholder referees are not the people authors are fearful of, and so getting them on board does nothing to solve the problem.

It is true that the people likely to, say, attempt to instigate a social media pile-on against an author—call them “mobbers”—are unlikely to be well-suited to be stakeholder referees. I never took my proposal to depend on denying this. What the stakeholder refereeing process can do is: first, send a message about the paper that makes it less likely for mobbers to pick that paper’s author as a target, second, provide good examples of how people who are similar to the mobbers (and those who would follow them) in some relevant respects might take a non-mobbing approach to reacting to papers they do not like, and third, improve the quality of the paper in various ways, including helping it jettison unnecessary distractions and provocative missteps.

Criticism 6. For any given paper, how do we decide who counts as a stakeholder?

This is a good question to which there are probably several reasonable answers. It may be that none of these reasonable answers is completely unproblematic. But that’s okay. In policy matters, the absence of problems is an unreasonable standard.

Criticism 7. Stakeholder refereeing will bring papers to the attention of the very people authors are likely to be fearful of.

I think publishing the articles in something called The Journal of Controversial Ideas will probably do that, regardless of the refereeing process involved.

Criticism 8. If this process had long been in place at every journal, so few interesting ideas would have gotten published.

My proposal is not a proposal for all journals. It is an attempt to help provide a solution to a problem some people think is worth addressing: how can we ensure that defenses of worthwhile yet politically unpopular theses by authors fearful of the professional repercussions of being associated with such theses get published?

Jeff McMahan, Francesca Minerva, and Peter Singer offered one solution to this problem: create a Journal of Controversial Ideas and protect the authors by hiding their identities with pseudonyms. My proposal took the creation of that journal for granted, but offered an alternative mode of protection that I argued was better for a few reasons. Nothing about my proposal involves subjecting all articles to the process.

That said, the process formalizes something that is generally advisable for anyone wading into particularly controversial waters with their work: have others, especially those who disagree with you and those who are more expert than you on evidence or methods you rely on, give you their assessment of  your work prior to publishing it.

*  *  *  *  *

Further feedback is welcome.

Takehiro Kishimoto – carving of an apple

Your email address will not be published. Required fields are marked *

Please enter an e-mail address