“Anything can happen in a small sample, but it was enough to suggest to me a hypothesis: There is no such thing as a generalist philosophy journal.”

The following is a guest post* by Brian Weatherson, Marshall M. Weinberg Professor of Philosophy at the University of Michigan. It originally appeared at his blog, Thoughts, Arguments, and Rants.

Citation Patterns Across Journals
by Brian Weatherson

I’ve been interested for a while in the different things that get attention in different philosophy journals. Part of what got me interested in this was looking at the ways that different well known articles get cited, or not, in different journals. Below is a table showing how many times (according to Web of Science), six prominent turn of the century articles were cited. The articles are:

1. Elizabeth Anderson, What is the Point of Equality, Ethics, 1999
2. Peter K. Machamer, Lindley Darden & Carl F. Craver, Thinking about Mechanisms, Philosophy of Science, 2000.
3. David Lewis, Causation as Influence, Journal of Philosophy, 2000.
4. Michael G. F. Martin, The Transparency of Experience, Mind and Language, 2002.
5. James Pryor, The Skeptic and the Dogmatist, Noûs, 2000.
6. Jason Stanley & Zoltán Gendler Szabó, On Quantifier Domain Restriction, Mind and Language, 2000.

Here are the 21 journals these six papers have (collectively) been most cited in, along with a count of how often they were cited in each.

A few quick notes about this data.

• It all comes from Web of Science, and as I’ve discussed previously, there are flaws with that data. I think the flaws don’t make a huge difference to the points I’m making, but they exist.
• I’ve tried to sort the citations to Nous from the citations to Philosophical Issues and Philosophical Perspectives, which Web of Science sometimes (but only sometimes) collapses.
• These are the journals that most cited the six articles in Web of Science. If a journal that Web of Science doesn’t index cited the six of them 15 or more times, I wouldn’t know about it. And Web of Science added a lot of journals in 2008, so even some journals it now indexes won’t appear there.

The last point affects one thing that might jump out at you from that list, or at least jumped out at me. These are almost all pretty high prestige journals. And, conversely, almost all the high prestige journals are on the list. (Philosophical Quarterly is probably the most notable omission, relative to my personal ranking of journals.) You can imagine a world where the really significant articles of 15-20 years ago are discussed much more in lower ranking journals, perhaps because they are mimicking what was going on in elite journals some time before. A priori, I wouldn’t have been surprised to find we were in such a world. But this is some (weak) evidence against it.

But what I really want to stress is how uneven the pattern is here. The two most cited articles of those six, by a lot, are the Anderson, and the Machamer et al. Here are the citations each has on Google Scholar, and on Web of Science as of today. (These numbers are sure to change in the future.)

So in terms of their impact on the journals collectively, those two articles have each been something like three times more influential than the other four articles. And those ‘other four articles’ rank somewhere between essential and field-defining in their importance to their subfields. And both the Anderson, and the Machamer et al, articles are in fields that I thought were pretty central to contemporary philosophy: political philosophy and philosophy of science. So they should be cited a lot in the ‘generalist’ journals, right? Right? Well, here’s the table above, this time with articles 1-2 collapses and articles 3-6 collapsed, and just the ‘generalist’ journals listed, and with the sort by the number of times articles 1 + 2 were cited.

It’s only six articles, and I guess anything can happen in a small sample, but it was enough to suggest to me a hypothesis:

There is no such thing as a generalist philosophy journal.

How could we test this? Here was one way that I thought of. Philosophy has journals that are recognised as specialist journals. Some of them, like Ethics and Philosophy of Science are roughly as high status as the generalist journals. We could measure how often articles in each journal cite other journals. If there was a generalist journal, it should routinely cite the high status specialist journals, and it should be routinely cited in both generalist and specialist journals.

So to that end I decided, again using Web of Science, to make a giant database of which journals cited which other journals. To avoid the problem that Web of Science can be a bit erratic in when it updates, I decided to focus on citations where both the cited article and the citing article were published between 1976 and 2015. And I focussed just on journals that Web of Science had indexed throughout that range, or (if they started after 1976) which Web of Science indexed from their foundation.

I chose 30 journals that were a mix of generalist journals and specialist journals. In part the choice was influenced by what Web of Science had available. And in part it was based on a deliberate over-sampling of specialist journals, because what I really cared about is the interaction between generalist and specialist journals. Here is the list of 30 I ended up with.

In part because it was easier to collect the data this way, and in part because I think it is a more meaningful measure, I didn’t count cites of more than one article from a journal in a given year, in a particular citing article. So if an article cited both my Intrinsic Properties and Combinatorial Principles, and David Lewis’s Redefining ‘Intrinsic’, both of which are PPR 2001, that would count as citing PPR once. I think that’s the right way to measure things; otherwise we’ll end up treating engagement with a symposium as a bigger deal than it really is.

And, as mentioned above, I separated out the Nous citations from Philosophical Issues and Philosophical Perspectives.

I fed this data into the (amazing) Circos Table Viewer, and here’s the first of the outputs. (Click on any image to get a higher res version.)

Between each (distinct) pair of journals, there are two lines, showing how often they cite each other. The lines go from the outer circle to the inner circle, and represent how many times the journal on the outer circle is cited by the journal on the inner circle. So you can see at a glance, for instance, that among this group, the Journal of Philosophy is cited many more times than it cites other journals, and the Canadian Journal of Philosophy cites other journals (in this group) much more often than it is cited by them.

The graph is a bit of a mess, so I tidied it up in two ways. First, I removed 8 journals that weren’t adding a lot to the graph, because they had very similar patterns to other journals that I included. (The 8 are BJA, PHR, KS, JPP, EP, RSL, RM, and EPI.) We’ll come back to those 8 in a bit, but for now I want to focus on the 22 remaining. Second, I used Circos’s feature that lets you cut out the thinnest 50% of the lines, so we can focus a bit more on what’s remaining. Here’s the result:

And we see a large scale version of what the six articles above suggested. With perhaps the exception of Philosophical Studies, there are very few journals that have strong connections to both the leading political philosophy journal, Philosophy and Public Affairs, and the leading Philosophy of Science journals, Philosophy of Science and BJPS. What really surprised me was how weak the links to Ethics are. I’m not surprised that some generalist journals don’t publish much aesthetics, feminist philosophy or history of philosophy: I knew those were gaps in coverage. I was surprised at how little philosophy of science, political philosophy, and even ethics, there seems to be in some leading journals.

I’ll write more about this in future posts, including deeper dives into which journals do and don’t interact with which specialist journals, and some important caveats to the no generalist journals conclusion. (Spoiler alert: Past performance is not necessarily a guide to future results.)

And I’ll do a long look at how the journals have changed over time. The four takeaways will be:

1. The Journal of Philosophy was unbelievably influential in the 1980s, but is now merely one leading journal among many.
2. Stewart Cohen has done an incredible job with Philosophical Studies. (This is a bit of a recurring theme actually.)
3. David Lewis gets cited a lot. It ends up being important to remember when the Lewis articles appear when getting a sense of the impact of various journals across time.
4. The interactions between generalist and specialist journals don’t change a lot over time, but they do change a bit, and some of the differences are worth pausing over.