Do Journals Favor Affiliated Authors?
“Do academic journals favor authors who share their institutional affiliation?” That’s the central question of a recent study, which finds evidence that suggests the answer is “yes.”
The study, “Academic In-Group Bias: An Empirical Examination of the Link between Author and Journal Affiliation.” by Yaniv Reingewertz and Carmela Lutmar (both of the University of Haifa), looks at journals in the field of international relations, but the findings may raise concerns about bias in any field.
The motivation for the study, according to an article about it at Inside Higher Ed, was the thought that “journals published by specific universities might have a slightly lower quality bar for authors who either worked at or earned a Ph.D. from those institutions.”
Here’s how the authors describe the study:
we examine citation counts for articles published in four leading international relations journals during the years 1995-2010. We use a difference-in-differences methodology to compare citation counts for articles written by “in-group members” (authors affiliated with the journal’s publishing institution) versus “out-group members” (authors not affiliated with that institution). Articles written by in-group authors received 9 to 19 fewer Web of Science citations when published in their home journal (International Security or World Politics) vs. an unaffiliated journal, compared to out-group authors. These results are mainly driven by authors who received their PhDs from Harvard or MIT. The findings show evidence of a bias within some journals towards publishing papers by faculty from their home institution, at the expense of paper quality, as measured by citations.
How relevant are these findings to philosophy? Philosophy has some in-house journals, e.g., The Journal of Philosophy at Columbia University, The Philosophical Review at Cornell. Questions had been raised about bias at Philosophy & Public Affairs, which is affiliated with Princeton, in a comment thread on a post here a couple of years ago. The key comment was this:
Philosophy and Public Affairs seems to me to have a real problem with bias. Because I suspected that it had such a problem, I went through back issues last year, looking at volumes 37, 38, 39, 40 and 41, in which there are 57 papers. Of those 57 papers, 17 had at least one author who received their doctorate from the same institution as the current Editor, Oxford. Another 13 were written by authors whose doctorates were from Harvard, where a number of editorial staff work or studied. The next most common place for authors to have received their doctorate is Princeton, where the journal is based. Of the ten papers in those five volumes published by authors who were last year employed by institutions in the western United States, four are by members of the editorial staff, three are by former visitors at Princeton, where the journal is based, and two are by people who were supervised by members of editorial staff. Only one of the ten has an author with no obvious links with editorial staff. It is impossible to be sure, simply on the basis of that sort of data, that there is anything untoward about its editorial policy, let alone anything deliberately so. I suggest though that it places the burden of proof on those who claim that it assesses all submissions in the same way.
(A response to this by Alan Patten, then the editor of Philosophy & Public Affairs, was added in Update 3 to that post. That response is also discussed in the comments.)
If people know of relevant work of this sort on philosophy journals, please share it.
Of related interest: “A Closer Look at Philosophy Journal Practices,” “Guarding the Guardians (or Editors),” “How Journal Capture Led to the Dominance of Analytic Philosophy in the U.S..” “Are Women Philosophers Underrepresented in Top Ethics Journals?” “Diversity in Philosophy Journals,” “Citation Patterns Across Journals,” “Philosophers Don’t Read and Cite Enough,” “Citation Problems in Philosophy—and Some Fixes,” and Kieran Healy’s “A Co-Citation Network for Philosophy.”
(Thanks to Joseph Shieber for suggesting a post on this study.)
Celia Smith, “Swallow” (study)
there are three obvious ways this would occur in philosophy:
1. schools that believe themselves to be prestigious favor other philosophers from there, and the more they publish affiliated authors the more they boost their perceived quality or prestige
2. the same thing but with advancement of a specific ideology or school of thought. see: Princeton and Chicago.
3. in state schools perceived as backward, there could be an automatic negative perception. here I am thinking of people having a negative bias against me because they associate me with Texas.
For what it’s worth, Phil Review has something of a policy of not accepting submissions from Cornell grad students and recent (within 5 years or so) Cornell PhD-s. I don’t think there’s an official ban, but we have been strongly discouraged from submitting out work there, and – as far as I know – we are all complying. My impression is that the faculty apply the same norm to themselves and refrain from submitting their work there. I might be wrong, but I cannot recall a paper recently published in Phil Review by Cornell faculty.
I believe that Philosophical Quarterly has a similar policy. St Andrews graduates are discouraged from submitting for the “first few years” after graduation.
I can see work published that was submitted when the author was a faculty member.
Note that there is a minimally pernicious explanation that may account for some bias: the bias may be toward topics or approaches (which it is an editor’s prerogative to favor) rather than people or institutions. That would explain some of the bias if those who have an affiliation with an institution are more likely to have the same biases toward topics or approaches.
Right–and in order to help lessen the suspicion that it’s due to a pernicious bias, I think it’s important for the author’s identity to be hidden from the editor too, not just the reviewers.
For this and other reasons, I wholeheartedly agree.
This hypothesis would also explain why names from the same circles seem to dominate Oxford Studies in Metaethics and Oxford Studies in Normative Ethics (and I’m sure there are plenty of other examples too, e.g. I have often heard this gripe about the Bellingham (now Vancouver) summer conference) — not because in-group members are favored *as such* but because members of the in groups know how to pick topics and write abstracts that will get their work into the associated workshops and then into the pages of the journals. That’s problematic insofar as it perpetuates inequality in the discipline, but it’s not evidence of anything untoward on the editors’ part.
Right. But the fact that the submission process for oxford studies in metaethics isn’t fully anonymized has to be playing a role, too.
That’s one factor. Another factor is that it’s difficult to pay for conference travel and the like. I can’t swing it so I don’t submit to the conferences that I’d need to to get my work published there. I wish they’d decouple the conferences from the volumes.
This hypothesis is plausible only if both the people evaluating the proposals are ignorant of (what you call) their fellow ‘in-group’ members current research projects and the people submitting these proposals write about and approach things in such a generic way that there is no way to identify the author of the abstract on the basis of similarity in style or approach to other already known work.
One thing to keep in mind: This doesn’t have to be a conscious thing by any means. If you know someone you might try harder to be charitable to their work than you would if it was something by someone you didn’t know. Further, the association with the school you went to or the school you work for might also bias one in subtle but real ways. I’m sure it’s relatively rare that anyone deliberately puts their thumb on the scale for a contributor (though unfortunately from my experience refereeing papers I do know that it happens) but I’d imagine that it’s very common that people act in biased ways without ever realizing it. (I’m late to the party on this I suppose, but at the moment I’m reading Dan Ariely’s work on how subtle bias and unconscious for those benefit us creeps in to judgments and it’s quite eye opening). And to second Tim’s comment, that’s why there’s absolutely no excuse for journals, especially journals that liberally use the desk reject, to not have a triple blind refereeing policy.