Recently, mainstream philosophy journals have tended to implement more and more stringent forms of peer review (e.g., from double-anonymous to triple-anonymous), probably in an attempt to prevent editorial decisions that are based on factors other than quality. Against this trend, we propose that journals should relax their standards of acceptance, as well as be less restrictive about whom is to decide what is admitted into the debate.

We start by arguing, partly on the basis of the history of peer review in the journal Mind, that past and current peer review practices attest to partisanship with respect to philosophical approach (at least). Then, we explain that such partisanship conflicts with the standard aims of peer review, and that it is both epistemically and morally problematic. This assessment suggests that, if feasible, journals should treat all available and proposed standards of acceptance in philosophy as epistemically equal, and that philosophical work should be evaluated in terms of the novelty and significance of its contribution to developing thought in ways that are of value. Finally, we show, in a programmatic way, that improving the current situation is feasible, and can be done fairly easily.

That is the abstract of “Pluralism and Peer Review in Philosophy,” by Joel Katzav and Krist Vaesen, forthcoming in Philosophers’ Imprint. It is worth a read.

In arguing that the profession, through its journal editing, adopt a pluralistic approach to philosophical methodology, the authors recognize that their central opponents will be philosophers who think that the mainstream “analytic” Anglo-American philosophical method is superior to alternatives and that their proposals would mean a lowering of philosophical standards. Katzav and Vaesen argue that the distinctiveness of the analytic philosophical method should not be mistaken for its superiority.

They begin by describing how a factor contributing to the dominance of the analytic style in mainstream philosophy was the preferences of journal editors: Moore’s and Ryle’s editorships of Mind are the central examples.

They then note that this mainstream (“m”) style is characterized by what the authors call “m-rigour” and “m-clarity”:

M-rigour predominantly characterises argumentation that aims to uncover the correct answers to substantive philosophical questions [and] argumentation that is adversarial…

The standards of m-clarity can plausibly be partially characterised in terms of the requirement that an author’s work be relatively easily understood by other mainstream philosophers, especially by those having the work as their area of specialisation. The standards of m-clarity also plausibly involve the requirement that concepts, propositions, inferences and larger-than inference level argumentative structures that make up written philosophical work be presented using one of a limited repertoire of standardised language forms; at the most general level, these forms will borrow from symbolic logic and/or regimented, informal reasoning, but they will also borrow from more domain specific forms such as the terminology of recent analytic metaphysics or recent meta-ethics.

Both m-rigour and m-clarity could be elaborated upon but, the authors say:

What matters is that the mainstream philosophy journals we are concerned with are, in employing m-rigour and m-clarity as criteria for assessing submissions, partisan with regard to which approach to philosophy they serve and thus which papers they are willing to publish.

Readers will no doubt differ over whether this sounds like unwarranted partisanship or just the application of reasonable standards. The authors think that a look at the proper functions of peer review will show it is unwarranted partisanship. Peer review has epistemic and non-epistemic functions:

The epistemic role of peer review is assessing the quality of research. From the hundreds of thousands, or even millions, of manuscripts submitted on a yearly basis, it weeds out those which are not sound, significant or novel… The authoritative nature of peer review is widely supposed partly to rest on the fact that work is to be judged on shared standards of excellence and expediency…

As regards non-epistemic certification, peer review serves as an evaluative standard in hiring, conferring tenure, promotion and grant committee decisions. Peer-reviewed articles are taken to demonstrate a scholar’s ability to (continue to) contribute to a field, in a way recognized by that field and, at least in comparison to invited contributions, independently from the scholar’s personal and professional networks.

So when Moore, around 1924-1925, changed editorial policies at Mind “so as to favour some of the approaches to philosophy that were still thriving in Britain over others that were doing so,” the authors argue that he was undermining the epistemic role of peer review “because the epistemic standards of evaluation employed during these editorships were not shared among the scholars served by the editorships.”

The problem is not merely historical, argue Katzav and Vaesen:

It is natural to think that the community currently served by, say, Mind, just comprises contemporary, mainstream philosophers. This community does, it might be suggested, share standards for evaluating philosophical work. Thus it might be thought that, although the journal has recently been partisan about how philosophy should be carried out, its peer review practices have been in accord with the standard requirements of peer review.

Such a line of thought is, however, too quick. Mind and other prominent journals collectively play an important role in determining how resources such as jobs, funding and research time are distributed among academics working, and seeking to work, in academic philosophy in the United States of America, the United Kingdom and other countries. Accordingly, the community these journals serve extends well beyond those who accept the standards of mainstream philosophy; it includes, for example, many working in African philosophy, feminist philosophers critical of mainstream philosophy, many continental philosophers and many of those who see themselves as trying to bridge the analytic-continental divide. The community in question also includes those who are not quite a good fit for any particular, widely adopted approach to philosophy. The journals being considered are thus, deliberately or not, partisan about whom they support from among the community of actual and potential academic philosophers in countries they influence. [emphasis added]

What is especially interesting about this paper, in my view, is how Katzav and Vaesen then raise skepticism about the philosophical enterprise (based on the argument from disagreement) to argue against the bias in peer review in favor of mainstream analytic philosophy. This part of the paper resists an easy summary, but here is one excerpt:

Thus far, our discussion recommends responding to the argument from disagreement by accepting that no philosophical approach has been established to be a reliable means of answering substantive philosophical questions and trying nevertheless to legitimate some standards of acceptance in philosophy. We have also suggested that, once the reliability of mainstream philosophy is held to be an open issue, there appears to be no good reason to prefer its standards… 

We do not deny that there is widespread agreement among mainstream philosophers about how to do philosophy. Our point is that, nevertheless, mainstream philosophers who have examined the issue of the reliability of philosophy do not provide agreed grounds for supposing that any available approach to philosophy is reliable…

[W]e argue that, in the absence of intersubjectively agreed grounds for taking available approaches to philosophy to be reliable, there are good epistemic and moral reasons not to prefer any particular available approach to philosophy in peer review.

The authors discuss a variety of objections to their arguments, and claim that review processes in philosophy need to be changed.

Two desiderata for adequate review processes in philosophy journals are suggested by the previous section. The first desideratum is that these processes should treat all available and proposed standards of acceptance in philosophy as epistemically equal (irrespective of who puts them forward). This desideratum is suggested by the thought that pluralism about philosophical approaches makes sense given the lack of an established to be reliable philosophical approach. The second desideratum is that review of philosophical work should include evaluating such work in light of the novelty and significance of its contribution to addressing philosophical questions that we need to address as humans and indeed, more broadly, in light of its contribution to developing thought in ways that are of value. This desideratum is suggested by the observation that, plausibly, addressing questions that we need to address and developing thought comprise an important part of what is of societal value in philosophy apart from its potential ability to answer substantive philosophical questions in a reliable way.

Both of our desiderata clearly recommend that approaches to philosophy according to which it has something other than the correct answers to its questions as a goal—e.g., literary value, suspension of belief, a purely heuristic role in discovery—should be included among the repertoire of approaches to philosophy. The second desideratum also recommends pluralism in approaches to philosophy as the historical record shows that a wide variety of philosophical approaches, including the proposal of novel approaches, have been of value. 

What changes do they recommend? Here are some quotes from the paper illustrating some possibilities (see the paper for details on them):

1. With regard to fostering pluralism of approaches in philosophy, the easiest option is for journals to adopt editorial policies that are pluralist about approaches to philosophy and, correspondingly, for them to diversify their editorial boards and review committees so that each paper is assessed by standards that match its approach. Mind has, very recently, partly gone down this route.
2. A further change that retains the basic set-up of the current peer review system would be to lower the bar for acceptance. Journals could set rejection rates at, say, around 60% rather than at above 90%.
3. Additionally–also easy to implement–journals might disclose, at the time of article publication, relevant review reports (with optional blinding of reviewer names), author responses and editorial decisions.
4. More inclusive policies regarding who participates in peer review, or even the elimination of peer review. A feasible way of broadening the participation in peer review involves following [a] public reviewing practice [in which] any paper which passes a basic quality check is put online and made available for public review and discussion.
5. [Editorial bias] could be avoided by allowing authors to curate their own papers. Authors, rather than editors, would thus decide whether and how to revise in light of comments received, when to consider their paper to be final or to retract.

Read the whole thing.

Julie Mehretu, “Sehkmet”