Philosophers Don’t Read and Cite Enough (guest post by Marcus Arvan)

Philosophers Don’t Read and Cite Enough (guest post by Marcus Arvan)


The following guest post* is by Marcus Arvan (Tampa). Marcus runs The Philosophers’ Cocoon, a helpful blog aimed at early-career philosophers. Last week saw the posting of a report on philosophers’ citation practices by Kieran Healy. Marcus has written on this topic a few times over the years at The Philosophers’ Cocoon (the latest is here), and so I asked him if he would share his take on these issues. As you’ll see, he thinks the problem is more troubling than one might first imagine. Discussion is welcome, as usual.


There has been a great deal of discussion on social media the past couple of days about Kieran Healy’s report on citation data culled from the 2,100 articles published in four of the most highly ranked philosophy journals (Nous, PPR, Philosophical Review, and Mind) from 1993-2013. In brief, Healy reports that although there were no statistically significant gender differences between how often the typical article by men and women in these journals is cited, the top-1% cited articles by men were cited far more often than the top 1% by women. Finally, Healy suggest this indicates there are significant gender differences in who—men or women—are taken to be “agenda setters” in philosophy (hint: it’s all men).

Most of the discussion I have come across on social media thus far has focused on issues of gender bias. I want to suggest there is evidence, however, that the real problem may be much deeper than this. In a number of posts at The Philosophers’ Cocoon the past several years, I have asked readers about their general reading and citation habits. A couple of trends emerged. A number of people said—and indeed endorsed—the following practices:

  • Only reading a handful of top-ranked journals (viz. “why should I read bad journals?”)
  • Only citing articles they bothered to read and draw influence from.

Both of these practices/norms appear to be quite idiosyncratic to philosophy. I know that in some other fields, people generally expect themselves and others to read and cite all recent work in the areas they publish in. Reading and citations, in other words, are not considered in these fields to “honorific”: one does not merely read and cite journals or work one considers “good.” Rather, one is expected to read and cite everything recent as a matter of basic, sound scholarship.

With this in mind, let us return to Healy’s data. In addition to the aforementioned gender differences, Healy reports several other facts of interest:

  1. Only 12.5% of articles published in Nous, PPR, Mind, and Philosophical Review from 1993-2013 were by women.
  2. Almost 1/5 of articles cited in the above journals have never been cited at all.
  3. A little more than 1/2 of all articles in the above journals have been cited fewer than 5 times.
  4. A very small proportion have been cited over 25 times.

Now let’s think about the math here. Suppose all you did was read these four top journals and cite articles in them in proportion to these citation practices. Next, suppose we define articles cited more than 25 times as “agenda setting” (which actually seems too weak. True agenda-setting work is presumably cited far more than that). Finally, suppose that only, say, 10% of articles in the data set are cited more than 25 times (in line with Healy’s statement that “Getting cited just twenty five times is enough to put a paper in the top decile of the distribution”). This means–prior to implicit or explicit biases having any opportunity to influence readers’ citation patterns—the probability of an article written by a women in one of these journals being recognized as “agenda-setting” (in terms of citation counts) is .125 x .10 = .0125, or approximately 1%.

This suggests to me, again, that the problem goes much deeper than mere bias. The problem is that philosophers’ reading and citations habits more generally are problematic. If one only reads top-ranked journals and you only cite papers you think are “agenda-setting”, then—prior to any gender bias on your part in selecting citations—you will tend to cite women as “agenda-setters” approximately 1 time in 100. As such, if we want to make citation counts more equitable, correcting for implicit and explicit biases alone won’t suffice. We must fundamentally change norms in philosophy about what to read and what to cite. One should not merely read authors or journals one takes to be “good” or “agenda-setting.” Our reading and citation practices should not function as honorifics (to recognize “good work”), for—as we see above—these practices alone suffice to mathematically/probabilistically exclude people from being cited (disproportionately, women) prior to implicit or explicit bias even entering the citation picture. Not only that, if anything enables implicit bias to make things worse (after calculating the 1% number above), surely—I want to say—it is the very same practices and norms. If, as a discipline, we are (A) in the habit of only reading and citing articles we take to be agenda-setting and (B) a significant number of people are in turn biased to treat men as “agenda-setting” but not women, then (C) the number of women who are recognized as agenda-setting is likely to be lower still than the pre-bias 1% figure (.o1%?).

As such, I want to suggest that if we really want to fix the problems Healy’s report describes, we need to replace the norm of only reading and citing journals and articles people consider “good” with an alternative norm utilized in many other fields: the norm of citing everything recent on your topic, good journal or no, good article or no. This alternative norm, as I see it, exists in these other fields for a reason. It exists so that the kinds of problems that Healy’s data illustrate do not arise. If you have to read and cite everything, it becomes much more difficult to systematically exclude people from citation networks, and indeed, from “agenda-setting.”

(image: “Paper Owl” by Irving Harper)

Horizons Sustainable Financial Services
Subscribe
Notify of
guest

64 Comments
Oldest
Newest Most Voted
Inline Feedbacks
View all comments
Walto
Walto
9 years ago

This is a strange amalgam. A bit about paucity of citations, some stuff about journals that should be cited more, and a couple of stray thoughts involving (I think) gender discrimination.

Sticking to the first topic, I believe that philosophy papers do indeed cite less than papers in other fields. And not merely “academic papers.” When I left teaching philosophy to become a bureaucrat and got involved in administrative hearings, reading briefs and administrative decisions it was a culture shock for me. Nearly every substantive statement–whether involving fact or law–had an associated citation; often more than one.

I’m not suggesting here that one approach is better than the other, or that either is perfect. But it’s hard to deny that philosophy papers contain many fewer citations than those in many other fields.

David Wallace
David Wallace
9 years ago

I agree with the diagnosis but I’m less sure about the proposed remedy.

The primary point of citation is scholarship: it is to identify, for any idea, concept, content or result in your paper, the original source of that idea. That’s not a subjective notion: if some idea has previously appeared in the peer-reviewed literature you are obliged to cite it, whether or not you yourself learned of the idea from that source. And ignorance is no defence: you have an obligation to search the peer-reviewed literature to see if anyone has come up with any part of your idea first. (I don’t think you have an obligation to search grey literature – unpublished talks and the like – though of course you do have an obligation to cite grey literature that you actually know about.) Citation also has a pedagogical role – you point readers new to a field towards survey articles and the like – but it’s decidedly secondary to scholarship.

But I don’t think that means that you need to cite everything recent on your topic, at least if “on your topic” is sufficiently broadly construed. If I have a proposed solution to Problem X that doesn’t overlap with your proposed solution, and if I don’t discuss your work and haven’t drawn ideas from it, I don’t think I have an obligation to cite it. (And fairly clearly the literature in other fields doesn’t deploy something that broad-brush either, else citation counts would vary much less than they do – the papers that get cited are the ones which make actual contributions on which others depend.)

Matt
9 years ago

A quick question on the presentation here: It’s said:

“2. Almost 1/5 of articles cited in the above journals have never been cited at all.
3. A little more than 1/2 of all articles in the above journals have been cited fewer than 5 times.

My understanding was that this study was only looking for citations of articles in the journals that also appeared in the journals, so 2) and 3) are possibly mis-leading as put. If I understood correctly, we only know that 1/5 of the articles cited have never been cited _in those four journals_ and that more than half have been cited fewer than five times _in those journals_. Is that right? (I’m sorry if I’ve misunderstood the claim.) That’s important, since for many areas of philosophy, these journals are not at all the places you’d expect to see most of the discussion taking place.

This doesn’t distract from the main point, which I agree with – philosophers tend to read too narrowly and cite too sparingly, and should do better on that.

Neil Dewar
Neil Dewar
9 years ago

“The problem is that philosophers’ reading and citations habits more generally are problematic. If one only reads top-ranked journals and you only cite papers you think are “agenda-setting”, then—prior to any gender bias on your part in selecting citations—you will tend to cite women as “agenda-setters” approximately 1 time in 100.”

Perhaps I’m missing something, but I don’t see how this claim follows from the above. The 1.25% figure is the probability that an article is cited more than 25 times and was written by a woman (on the assumption that these are statistically independent). The probability that an article is cited more than 25 times and was written by a man is (under the same assumption) 8.75%. In other words, by virtue of the independence assumption, the gender ratio of agenda-setting articles is precisely the same as the overall data set; the fact that agenda-setting articles only make up 10% of the total is irrelevant. Hence, what would follow is that, if I resolve to cite only agenda-setting articles (and that I’m equally likely to cite any one agenda-setting article as any other), the probability that I will cite an article written by a woman is 12.5%, not 1.25%.

Of course, this assumption of independence is problematic: the fairly clear worry is the one that Arvan discusses towards the end, that because people are systematically failing to recognise women as agenda-setters, agenda-setting articles written by women are likely to constitute less than 12.5% of the total number of agenda-setting articles. (Also, it might be that even when articles are recognised as agenda-setting, they are less likely to get the further citations than agenda-setting articles written by men.) But that problem is then presumably due to implicit bias and associated effects, rather than being a consequence of citation practices.

Richard Yetter Chappell
9 years ago

Marcus — I think your probability claims here are mis-stated.

You write: “the probability of an article written by a women in one of these journals being recognized as “agenda-setting” (in terms of citation counts) is .125 x .10 = .0125, or approximately 1%”

As I read it, what you have calculated here is instead the proportion of all articles in these journals that are both (i) written by women, and (ii) agenda setting. Pre-bias, the proportion of articles written by women that are agenda setting is the same as the proportion of all articles that are agenda-setting, i.e. 10%.

This error is repeated when you write: “If one only reads top-ranked journals and you only cite papers you think are “agenda-setting”, then—prior to any gender bias on your part in selecting citations—you will tend to cite women as “agenda-setters” approximately 1 time in 100.”

Again, I don’t see how you got this figure. If 12.5% of articles in the top journals are by women, then — pre-bias — if you only cite “agenda-setting” articles then we should expect that 12.5% of those will be by women. The 1 in 100 figure is instead how many of your citations would be to agenda-setting women if you cited all articles in these journals (so, for every 100 citations, you would also be citing 11 or 12 non-agenda-setting articles by women, etc.)

Have I misunderstood you?

Jacob Archambault
9 years ago

On the proposed amendment (leaving aside whether it really conforms to what other scholarly disciplines do), citations in a philosophy paper would become the analogue of shout-outs on a rap track: they would serve the purpose of a kind of group identification of those who do and don’t count as members of the discipline, only now cheapened by the essentially promotional requirements that people cite everybody on their record label, as it were. At worst, this exacerbates the problem by promoting the further ossification of the areas of research themselves (anecdotally, it seems to me many of the women whose work I find so impressive are more likely to do edgier work that doesn’t fit well within currently drawn disciplinary boundaries). At best, it merely widens the circle of those who get to belong within what remains an in-crowd mentality.

Beau Madison Mount
Beau Madison Mount
9 years ago

One of the purposes of citation is to aid the reader in researching the topic. By isolating the works in previous literature that are directly relevant to the question at hand, the author offers an expert opinion on how one should allocate the limited amount of reading time one has. Obviously, this opinion can be wrong, and it’s subject to all the biases that judgements of merit and relevance in general are subject to; no one should take the list of references cited in a particular article as the final word on what to read. It is nonetheless often a very useful — in some cases, an indispensable — guide. But if an author habitually string-cites dozens of articles, burying those she has used and those she takes to be of general merit, amid dozens of others included only for completeness, then this information is lost, and the reader is left without a guide. Pointless overcitation is a bad habit unfortunately prevalent in the social sciences and law; if we can do little to push it back there, we can at least keep it from spreading to philosophy. Anyone can print out the results of a Google Scholar search; what’s useful to the reader is a *selective* guide to further investigations, reflecting the author’s judgement.

anon
anon
9 years ago

I came to recall my past experience reading lab reports and research papers in engineering; they tend to cite numerous papers even without sparing any evaluative comment.
I thought it was no more than a interruption to the logical flow of an argument, but I should have been wrong.

Marcus Arvan
9 years ago

Hi Richard: Thanks for your comment. Yes, I think you have misunderstood me. You write, “As I read it, what you have calculated here is instead the proportion of all articles in these journals that are both (i) written by women, and (ii) agenda setting.” This is precisely what I meant to calculate: the proportion of articles that are both written by women and *recognized* as agenda setting. The point is simply that, following current citation standards, the pre-bias probability that any article in the above journals from 1993-2013 will be both written by a woman and recognized as agenda-setting (+25 citations) is 1%.

Marcus Arvan
9 years ago

Jacob: Thanks for your comment. However, I think you misunderstood the proposal I’m advancing. Of my proposal, you write, “citations in a philosophy paper would become the analogue of shout-outs on a rap track: they would serve the purpose of a kind of group identification of those who do and don’t count as members of the discipline, only now cheapened by the essentially promotional requirements that people cite everybody on their record label, as it were.”

This is exactly what I think occurs *now*, and it is what my proposal (viz. one should be expected to cite everything) is intended to address. Under prevailing norms, people tend to cite who they want to cite (i.e. papers of people/journals they read). This predictably leads to people tending to cite friends, etc., leading to precisely the kind of ossification you mention: a relatively small group of people in the literature constantly receiving “shout-outs” from everyone else, and a much larger body of researchers being cited hardly at all.

My proposal follows that of the hard sciences. In many STEM fields, one is expected to cite *everything* recent on a topic. This prevents ossification precisely because it *widens* the body of literature people are expected to read and cite (I say this with some experience: my wife works in a STEM field).

Here’s another way to look at it. Requiring everyone to give a “shout-out” (e.g. citation) to all recent work on a topic might appear on the surface to be merely pro-forma–that is, a formal requirement that is unlikely to do much to include people in philosophical discussion (“Great”, you might say, “women might be cited more often. But would their work be discussed any more?”). But, I think this gets it exactly wrong. People tend to get included in philosophical discussion *because* they were cited, and for two reasons. First, when a person is cited, it draws attention to their work, encouraging at least some people to read it who wouldn’t have read it if it had never been cited–indirectly leading more people to take the work seriously (and perhaps discuss it in their own work). Second, in my experience, people tend to be included/excluded from philosophical discussion directly on the basis of whether their work is cited. I’ve often heard people say things like, “There’s no need to discuss so-and-so’s work. Nobody reads that journal”, and, “You really need to discuss so-and-so’s work. Everyone discusses it!”. In other words, although requiring people to cite other people’s work might *appear* to do very little on the surface (viz. it’s just “shout outs”), in reality this very practice–giving shout-outs!–will tend to (A) increase the visibility of people’s works, and (B) decrease the probability that someone will be excluded from discussion because “no one discusses them.”

Marcus Arvan
9 years ago

Matt: I think you’re confusing Healy’s earlier co-citation analysis with the new one where *all* web of science citations were collected. The data here isn’t the former (i.e. how many times articles in these four journals cite one another). Rather, it is the total # of citations articles in these four journals have received in ALL journals.

Marcus Arvan
9 years ago

anon: Thanks for your comment. You write, “I came to recall my past experience reading lab reports and research papers in engineering; they tend to cite numerous papers even without sparing any evaluative comment.
I thought it was no more than a interruption to the logical flow of an argument, but I should have been wrong.”

It may seem like an “interruption”, but giving people credit for their work is good reason for interruption! STEM fields do not have the same citation problems that we do in philosophy (e.g. gross gender-imbalances, people publishing “new” philosophical arguments without knowing that someone else already published it years prior and not citing that earlier work, etc.). Ensuring that people are properly credited for their work is worth the interruption.

Marcus Arvan
9 years ago

Neil: Thanks for your comment. I think you’re misunderstanding my point in a way similar to Richard. You write, “Hence, what would follow is that, if I resolve to cite only agenda-setting articles (and that I’m equally likely to cite any one agenda-setting article as any other), the probability that I will cite an article written by a woman is 12.5%, not 1.25%.”

Yes, but the problem is *recognizing* which articles are agenda-setting! The point is, of any 100 articles in the four journals, 10% will in fact be recognized as “agenda setting” (+25), and only 12.5% of those are likely to be by women: hence, of any 100 articles, the pre-bias likelihood that a given article will both (A) be written by a woman, and (B) be recognized as agenda-setting is 1%. That’s the only point I was trying to make. Take any copy of those journals you like, the pre-bias probability of an article being both (A) and (B) is 1%.

Matt
9 years ago

Thanks, Marcus – I’ll look again. I appreciate the correction.

David Wallace
David Wallace
9 years ago

Marcus, on Neil’s/Richard’s point: yes, agreed that (on your assumptions) a given article, selected at random from a Healey-top-four journal, has a 1% chance of being both (a) written by a woman, and (b) agenda-setting. But I’m not really sure what I should infer from that, and it’s not how you later use the 1% figure later. To quote:

“If one only reads top-ranked journals and you only cite papers you think are “agenda-setting”, then—prior to any gender bias on your part in selecting citations—you will tend to cite women as “agenda-setters” approximately 1 time in 100.”

I don’t know how to disambiguate that other than “1 time in 100, when you cite, the article you cite will be by an agenda-setting woman”. But that’s plain false: on that citation policy, it will be one time in 8. (Or put another way: if you cite women as agenda-setters approximately 1 time in 100, then you will cite men as agenda-setters approximately 7 times in 100. What happens the other 92 times out of 100? 92 *papers* out of 100 don’t get cited, but that’s not salient to what *I* do N times out of 100.)

You then say:

“If, as a discipline, we are (A) in the habit of only reading and citing articles we take to be agenda-setting and (B) a significant number of people are in turn biased to treat men as “agenda-setting” but not women, then (C) the number of women who are recognized as agenda-setting is likely to be lower still than the pre-bias 1% figure (.o1%?).”

The only ways I can see to disambiguate this are (a) pre-bias, the fraction of women who are agenda-setting is 1%; the post-bias level will be 0.01%; (b) pre-bias, the fraction of agenda-setters who are women is 1%; the post-bias level will be 0.01%. But neither pre-bias number follows. On (a), the data as given just don’t provide any route to estimate the faction of women who are agenda-setters (consistent with the quoted data, every single paper in the Healer top four could be by the same two people, both of whom set the agenda). On (b), unambiguously the pre-bias fraction of agenda setters who are women is 12.5%.

I’m not (just) being pedantic: I (and I suspect Neil and Richard) are failing to see why there is any statistical link between low citation rates and issues of underrepresentation of women that goes beyond the (obviously very significant, but already known) fact that only 1/8 of Healey-top-four papers are by women. Provided the means by which I select papers to cite from Healey-top-four journals are not themselves correlated with gender, 1/8 of my citations will be women, whether I cite 1% of papers, 10% or 100%.

(As a more pedantic point, rounding 1.25% to 1% is fair enough, but I don’t think it should then be described as 1 in 100 rather than 1 in 80.)

David Wallace
David Wallace
9 years ago

Small correction, sorry: on my (b), the pre-bias fraction of agenda-setting *papers* that are by women is 12.5%. We don’t have the data to estimate the fraction of agenda-setting *philosophers* who are women (defining an agenda-setter as someone with at least one agenda-setting paper).

Sacco
Sacco
9 years ago

Marcus isn’t giving his proposal enough credit, since it could also (1) aid journals and (2) make it less likely that someone would be awarded for an idea that was already published in the literature. Paper 1 doesn’t sufficiently cite the literature but is otherwise solid? Desk rejection. Same goes for Paper 2? Desk rejection. And Paper 3? Well, the author of Paper 3 cites just about everything, thus not only does it not exhibit any preferential citation patterns, but it almost certainly doesn’t repeat what has already been said by someone else in the literature? Pass it along to a referee.

Walto (@WalterHorn)
Walto (@WalterHorn)
9 years ago

One problem with the suggestion that EVERYTHING be cited involves the flower-in-the-crannied-wall. In philosophy, it can be (and often is) argued that nearly every topic is involved in nearly every other topic. If one has to cite Davidson on concepts to write about non-cognitivism in meta-ethics, or mention every paper on color perception written in the last century whenever one ponders whether “red” is a rigid designator, every philosophy paper will be busy, boring and long.

Joe
Joe
9 years ago

Marcus, on your blog, you used to write quite a lot about the need for bold, fresh, interesting ideas (http://philosopherscocoon.typepad.com/blog/2012/10/safe-and-a-little-boring.html). Indeed, there is a common sentiment, often echoed by senior philosophers, according to which younger philosophers are increasingly incapable of “thinking outside the box”. What more effective method could possibly be designed for *maintaining* that state of affairs then requiring of them that they should explore (and cite) literally every single thing inside the box?

I know you think that ‘rigor fetishism’ is the real culprit here, but from this armchair psychologist’s perspective, the need to read and cite absolutely everything else on one’s topic is plainly just as stifling. By definition, time spent scouring the literature for missed citations is time *not* spent in creative original thought. My point is not that your proposal is necessarily mistaken, but rather that it would surely have consequences which you yourself have worried about in the past.

Marcus Arvan
9 years ago

Hi David: Thanks for the follow-up. Sorry for rounding off. Anyway, as you put it, hypothetically speaking, if you were citing papers in those journals from 1993-2013, the chances you would cite an article written by a woman is 1 in 8 (12.5%). Now, you’re certainly right about the next point: that we cannot know from the data alone what proportion of papers by women are agenda-setters (i.e. in the top decile of papers with 25+ citations). However–and I should have been clearer about this–I meant to be making a ceteris paribus argument. *If*
(1) 1 out of 8 papers in the journals are by women, and *if*
(2) 10% of those papers (by women) in turn fall into the top decile (which seems plausible, ceteris paribus, since there is no reason offhand to think, pre-bias, that men are more likely to be agenda-setters than women, *then* if follows that, before citation-bias,
(3) Only 1.25% of articles published in those journals during that time period written by women will end up in the “agenda-setting” top decile.

I think this is a good, ceteris paribus argument. Of course, all things might not be equal. The point, though–the point I was trying to make–is that if we simply combine (A) the low rate of articles by women in those journals, with (B) low-citation rates all around, it is a tautology (ceteris paribus!) that articles by women will tend not to recognized as “agenda setting” (so defined as 25+ citations). Although again this is only a ceteris paribus argument, it certainly seems to me worth worrying about.

Finally, I’d like to address your (very important, in my view) earlier comment. You write: “The primary point of citation is scholarship: it is to identify, for any idea, concept, content or result in your paper, the original source of that idea. That’s not a subjective notion: if some idea has previously appeared in the peer-reviewed literature you are obliged to cite it, whether or not you yourself learned of the idea from that source. And ignorance is no defence: you have an obligation to search the peer-reviewed literature to see if anyone has come up with any part of your idea first. (I don’t think you have an obligation to search grey literature – unpublished talks and the like – though of course you do have an obligation to cite grey literature that you actually know about.)”

I entirely agree. The problem, though–and a lot of people have reported on this at the Philosophers Cocoon and elsewhere–is that there seems to be a distinct habit in philosophy *to* treat ignorance as an excuse. I’ve had more than a few people tell me, for instance, that although their publication on Argument X comes up at the top of a very simple philpapers search, their article has been systematically ignored/not cited in the literature. I’ve also had more than a few commenters explicitly attempt to rationalize these exclusory practices by saying that (A) they don’t have time to read everything, or (B) they should only cite articles that they have in fact read and which in fact influenced them. I see no reason to be satisfied with either rationalization. People in STEM fields have at least as little time as us–probably less–and yet their citation standards are typically much more strict, and the idea that one should only cite articles that in fact influenced one is anathema to sound scholarship.

Jacob Archambault
9 years ago

Dear Marcus,
Thank you for your reply. I will address it below in two separate comments: the first directed toward your response to the parenthetical in the first sentence of my original post; the second directed toward your take on the analogy more broadly.

On the current practices of STEM journals: if I understand your proposal correctly, then the result of its implementation would be that the journal that receives the most citations on a topic will just be the journal that publishes the most articles on that topic (regardless of the quality of those articles). Though articles in STEM fields cite far more articles than is typical in philosophy, the practice in STEM fields doesn’t seem conform to your proposal, either.

Jacob Archambault
9 years ago

On the relation between citation practices now and under the proposal:

I think you missed an important part of the analogy: the phrase “only now cheapened, etc.” implies that even if we *grant* (as I do, agreeing with you) that present citation practices often serve to promote an in-crowd out-crowd structures, your proposal doesn’t change that essential orientation: it merely expands it. That’s why I say “at best, it merely widens the circle of those who get to belong…”

It is an injustice for work worthy of citation to go unnoticed, especially if this is related to gender- or other forms of bias. It is not an injustice for work to go uncited simpliciter. Correcting for the former by correcting for the latter either a) just undermines the merit involved in being cited in the first place, or b) just relocates it to the question of whose work counts as belonging to the discipline. This relocation is not essentially different from and potentially just as problematic as the current practice.

Marcus Arvan
9 years ago

Joe: Thanks for the comment, and good question. Although we can only speculate about this, I see little reason to think that reading stuff undermines a person’s ability to think outside of the box. On the contrary, I think *not* reading a lot plausibly invites inside-the-box thinking, and for a variety of reasons.

First, I think there is a tendency–one sees it quite often–for smallish groups of people to specialize/focus on small, insular batches of literature (I don’t think this is controversial; many debates are clearly had by smallish groups of people enmeshed in very narrow debates!). In contrast, reading a lot can expose one to ideas *outside* of one’s “wheelhouse”, which can in turn enable one to question the assumptions that underlie small, provincial debates.

Second, I think not reading a lot can invite a whole lot of “reinventing the wheel” (something one also sees a lot of). It is all too easy for someone to think they have a terribly original idea because, well, they haven’t bothered to see if anyone has published on it. I don’t want to pick out particular cases (for obvious reasons!), but I can think of many cases right off the top of my head of articles defending “new arguments” that were in fact presented by other people years earlier. In order to know whether your idea is original/outside the box, you need to (A) know what the box is, and (B) whether anyone has already published your “original” idea.

Richard Yetter Chappell
9 years ago

Marcus – “(3) Only 1.25% of articles published in those journals during that time period written by women will end up in the “agenda-setting” top decile.”

Again, this is simply not true. Pre-bias, trivially, 10% of articles published in those journals during that time period written by women will end up in the “agenda-setting” top decile. You mean to say that only 1.25% of articles published in those journals will be both written by women and in the agenda-setting top decile. But this is not the same as claiming that only 1.25% of the articles written by women will end up being agenda-setting articles. To say the latter, when you mean the former, is extremely misleading.

(Apologies to Neil for my original comment effectively just repeating his — it hadn’t appeared when I was writing mine.)

Anon Grad Student
Anon Grad Student
9 years ago

Marcus, I’m sympathetic to the idea, but I find the appeal to STEM practices uncomfortable. STEM reading is an entirely different animal than philosophy reading. You can read a STEM piece in a matter of minutes. I’ve done it. You can take longer, and you occasionally should take longer, but it is not the norm to spend time thinking hard as you read.

You cannot (responsibly) do that with philosophy articles.

Marcus Arvan
9 years ago

Richard: You are absolutely right. Thanks for clarifying/setting me straight!

Dave Baker
Dave Baker
9 years ago

Although I don’t agree with the statistical reasoning in the post, for reasons pointed out by Dewar and Chappell, I do think it would be an improvement if the average philosophy paper cited more of the literature.

Here’s something editors can do to facilitate this: don’t count a paper’s bibliography toward your journal’s word limit! Many of the best journals have draconian word limits for submissions, and one of the easiest ways to reduce length without damaging an article’s content is by cutting a citation or two. I’ve done it before, although I wish I hadn’t.

New Asst Prof
New Asst Prof
9 years ago

I am concerned that if the number of citations in philosophy were to drastically increase, then the prestige-value of being cited would drastically decrease.

Right now being cited is a big deal. But, in a world where each paper has hundreds of citations, it would not do much to boost a person’s profile. Right now, being cited makes it likely that several people will pick up your article and read it. But, if we are all reading hundreds of articles to write our own papers (on Marcus’s proposal), and each of these has hundreds of citations, it may just become a blur. Or, some new metric of prestige might come in to play. For example, I could imagine “survey articles” becoming vogue, where more selective citing is permitted and where the real stand-out articles are summarized.

None of this denies that scholarly progress might accelerate if philosophers start citing more. My concern is just that this would do little to address concerns about equity and representation in the philosophical community.

David Wallace
David Wallace
9 years ago

@Marcus: If you’re right (and I’ve no reason to doubt that you are right) that ignorance is taken as an excuse not to cite, that’s really quite worrying. That suggests an undergraduate attitude to citations. It isn’t something I’ve come across myself – I wonder if it’s less true in philosophy of physics, maybe because we’ve inherited physics citation practices? (But Dave Baker’s comment suggests that’s optimistic.)

@Anon Grad Student@25: In my experience reading a paper in mathematics or theoretical physics is at least as demanding as reading a paper in philosophy, but those disciplines seem to manage proper scholarship.

algol
algol
9 years ago

Seems to me the fundamental problem is not too few citations but a kind of elitism or cronyism in citation practices.

Vincenzo Politi
Vincenzo Politi
9 years ago

I know of at least a couple of cases of people who had their papers rejected by ‘top journals’ because (among other things, of course) those papers did not engage enough with (aka, cite) the ‘relevant literature’. In both cases, the ‘relevant literature’ mentioned (in full bibliographical details) in the reviewer’s comments was constituted by the papers and books authored or co-authored by a philosopher in particular. (In one of the two cases, the ‘relevant literature’ kindly ‘suggested’ was by a female philosopher.) Not only I really do wonder who the ‘anonymous referees’ may have been (yeah, right), but I also wonder whether the ‘agenda’ is set during the referencing process – i.e., ‘if you want to publish this paper you must cite X’s papers and book’.

Marcus Arvan
9 years ago

New Asst Prof: There are other, better ways of gauging prestige and influence–namely, people actually writing articles *on* a person’s work (e.g. developing it, criticizing it, etc). Citations alone should be understood as giving credit for ideas, not prestige. Citations-plus-engagement is a better of prestige.

Marcus Arvan
9 years ago

Jacob: Thanks for your follow up comments.

You write, “It is an injustice for work worthy of citation to go unnoticed, especially if this is related to gender- or other forms of bias. It is not an injustice for work to go uncited simpliciter.”

I reply: it depends on how we’re understanding “worthy” here. It *is* an injustice for a “bad” article defending argument X not to be cited by a later, better article also defending X. In this case, although the former article is not very “worthy” (its defense of X is poor), the paper still ought to be cited–as it defended X before the later, good paper did. The first, bad paper should still be cited simply because it actually defended the idea first. The later author is free to say the earlier paper was bad, but they still have a duty to cite it, simply to draw attention to the fact that they are not the first to defend the idea.

You also write: “On the current practices of STEM journals: if I understand your proposal correctly, then the result of its implementation would be that the journal that receives the most citations on a topic will just be the journal that publishes the most articles on that topic (regardless of the quality of those articles). ”

I reply: I actually think this is the case with STEM journals. The journals that are cited the post do appear to be those that publish the most material. But this isn’t a problem, for reasons I give in my reply to New Asst Prof. In STEM fields, prestige is not gauged primarily in terms of citations but in terms of *influence*–where this is something like a combination of citation and active engagement with (e.g. active development of) a person’s work.

Dave Baker
Dave Baker
9 years ago

David, I do think things are better on average in philosophy of physics, although I think philosophers of physics could be much better about reading and citing relevant work from other areas of philosophy. But most papers do cite all the relevant phil physics literature. Books are less consistent about this; you sometimes get a book written on the Kripke model where you mention a bunch of ideas without citing the people who thought of them.

Ben
Ben
9 years ago

I don’t think every article should include a comprehensive literature review, and I don’t think science citation practices are anything to aspire to in philosophy. I rather dislike reading social science articles with their proliferation of irrelevant footnotes and constant parenthetical interruptions that consist of nothing but long lists of references.

Reading is another matter. You should of course do your best to avoid “reinventing the wheel,” but this is actually pretty hard to do, especially for those of us who do work at the intersection of several disciplines. Often, practically the same issues are discussed in very different terminology by communities that don’t know about one another (I think this happens even within subdisciplines of philosophy). So I think it’s inevitable – and forgivable – that a certain amount of reinvention of the wheel is going to take place.

Gordon
Gordon
9 years ago

I find the defense of not citing very odd, and I suspect that anybody who works at the intersection of philosophy and another field will too. I know legal literature better than STEM, so I’ll stick with that. I think we can all agree that law review papers sometimes go too far in the other direction, footnoting obvious statements of fact or producing seemingly irrelevant lists of articles. But that doesn’t mean it’s a bad idea for philosophy to cite more things. First, well-executed lit review footnotes will tell you something about the literature (“many people have talked about this, but most have said x: see, for example …” or “for an exception to this, see xx (arguing that the moon is made not of green cheese, but blue)”). Those sorts of citations are really helpful to readers because they both help one to see where a paper’s contribution lies, and what the right further reading to do is. Second, it would put a stop to the practice referred to in the OP of only reading “top” journals. It only takes a moment’s thought to realize that there’s no way that those top few journals publish the only interesting work – they don’t publish enough papers to do so. Third, I suspect (totally unscientifically!) that this business of under-citing is part of a disciplinary ethos that tends to think that academic or other creative work is something that is the product of an isolated genius. Part of why I suspect this ethos is part of the problem is the defense of under-citation: the idea that if you read too many things you’ll stop being creative. That’s just not true.

Walto
Walto
9 years ago

I just felt the need to note my outrage at Ben’s failure to site my earlier “busy, boring, and long” comment here. Will the marriage of justice and good scholarship never occur??

;>}

Jonathan Ichikawa
9 years ago

I think that Marcus is running together two thoughts I’d rather keep separate.
One thought, with which I completely agree, is that as a discipline, we need to be better about reading more widely and engaging with our colleagues’ work in a fairer way.
Another thought, with which I have much less sympathy, is that we ought to have much longer lists of references in our papers—indeed, that we should cite everything that’s ever been published in the area. This second thought strikes me as both unhelpful and problematic.
It seems to me unhelpful because I don’t see what it would do to address the first thought; the issue is how we decide what is essential reading and agenda-setting, not whether we drop a long list of papers we don’t engage with. We could have exactly the same problem with the distribution of work’s perceived importance, even with giant bibliographies that obscure this fact.
It seems to me problematic for two reasons. One is that I think we have a responsibility to read, or at least skim, everything we cite, and there is far too much published work in each field for anybody to be expected to read or skim it all. So publication would be too onerous. Another is that, as Beau mentioned at #7, Marcus’s policy would destroy one of the central functions of citation—for a researcher to express one’s own attitude about what others interested in the area should be reading. If everyone followed Marcus’s advice, then citation counts would tell us nothing except which fields were popular. I can look at the PhilPapers index page for a topic if I want to know everything anyone’s written on the topic; I don’t need to replace every bibliography with the same list.

Marcus Arvan
9 years ago

Jonathan: Thanks for your comment. However, I think you may have missed some of my comments explaining why I think the two things go together. You say that “the issue is how we decide what is essential reading.” My point, though, was that implicit biases are very hard to change–and once only a small subset of work is cited, *that’s* the work that tends be read. If you want to change people’s reading habits, you need to change what’s *cited*–precisely because (A) people tend to read what’s cited and (B) ignore stuff that’s not cited. In other words, I don’t think we can reasonably expect to change people’s reading habits without an intervention in the citation domain.

I also still don’t get the idea that the proposal would “destroy” a central function of citation, namely, registering readers’ attitudes towards work. My very point is that this shouldn’t be seen as one of its essential functions. All that does is (A) allow people to systematically exclude others from the literature, on the basis of (B) explicit or implicit biases. On that note, one of my friends on social media reported to me today that one person told her he didn’t cite her work on Topic X because he believed her argument to be unsound. This is *precisely* what is wrong with this approach, and it it is why other fields do not adopt this approach. The job of a citation should not be to register approval. It should be to give credit for the fact that an idea is out there–that someone published it–and came before you! If you think the paper is bad, that is no reason to pretend that it does not exist. Cite it and say in your footnote or something you do not accept its argument.

Marcus Arvan
9 years ago

Ben: Thanks for your comment. You write, “I rather dislike reading social science articles”.

I reply: sound scholarship is not about what we like or don’t like. We may not like engaging in sound scholarship, but for all that, we have an obligation to do it: an obligation to give *credit* for ideas. Social science articles have lots of citations because it’s sound scholarship.

Beau Madison Mount
Beau Madison Mount
9 years ago

Marcus wrote: “The job of a citation should not be to register approval. It should be to give credit for the fact that an idea is out there–that someone published it–and came before you! If you think the paper is bad, that is no reason to pretend that it does not exist. Cite it and say in your footnote or something you do not accept its argument.”

I’m a little confused at this point. I thought that everyone was in agreement that if a paper anticipates your ideas in any way, then you have a responsibility to cite it, whether or not you think it’s good and whether or not it directly influenced you. Also, as David Wallace pointed out, you have a responsibility to make reasonable efforts to search through the literature (at least the peer-reviewed literature) to find such papers.

I thought the question was whether you *also* have a duty to cite work which *doesn’t* anticipate your ideas, which *doesn’t* provide tools or intermediate results that you use, and which relates to your paper only insofar as it discusses, in some way or another, the same topic. (I admit being very unsure what this would mean in practice, since “topic” seems irremediably vague.)

I’m having trouble seeing why the latter thing is a moral duty, or any part of “sound scholarship”. Certainly the ordinary explanation for the duty to cite — on which it follows from a more general not to arrogate to oneself credit for what is partially due to the efforts of others — doesn’t help here.

anon
anon
9 years ago

Wow, having just now stumbled onto this thread, I’m astounded by the resistance to more complete citation practices in philosophy. E.g.: “I rather dislike reading social science articles with their proliferation of irrelevant footnotes and constant parenthetical interruptions that consist of nothing but long lists of references.” The ‘nothing but long lists of references’ strikes me as odd. Authors whose works contain these ‘long lists of references’ are not including those lists for no good reason. They’re embedding their points in a wider literature and giving their intellectual antecedents their due. This is part of any meritocratic scholarly practice, so there is a very important role these ‘long lists of references’ serve.

I am likewise surprised by Jonathan’s claim that one point of citation is “for a researcher to express one’s own attitude about what others interested in the area should be reading.” I strongly reject this view. Citation should be an as-value-laden-free-as-possible practice designed to acknowledge antecedents. These antecedent scholars should be acknowledged whether or not one thinks their work amounts to ‘what others interested in the area should be reading.’

No doubt my own interdisciplinarity–I sometimes work in cognitive science–influences my own attitudes about citation practices (to echo Gordon’s point). I am personally embarrassed by the under-citing and underspecificity of citing that philosophers so often engage in (an example of underspecific citing would be when an author cites an entire book when the relevant point is confined to a short passage in the book). I think it borders on intellectual dishonesty. Of course citing is a major annoyance as it is and more complete citation practices will be a bit *more* annoying. But that’s part of being a scholar, if one of the least enjoyable parts.

Finally, do I really have to mention the obvious power issue here? I think we all know of at least one case where: (i) a point was originally defended by a woman and/or philosopher of color and/or very junior person and/or otherwise marginalized member of our community, (ii) that point was subsequently defended by someone more empowered, and (iii) the subsequent literature cited only the second defender and not the first. Normative citation practices, such as citing ‘what others interested in the area should be reading’ will no doubt tend to perpetuate these sorts of pre-existing power dynamics, whether intentionally or not.

For my part, I am incredibly grateful to Marcus for starting a blog devoted to identifying citation gaps in literatures and to Jessica Wilson for being the first person to contribute to it: http://philosopherscocoon.typepad.com/the_campaign_for_better_c/

Jonathan Ichikawa
9 years ago

I agree that it’s hard to change implicit biases. I disagree that demanding the kind of completion is even a way, let alone the best one, to effect the change we need, for the reasons I explain here and in the comments to your older post.

I do think that you and I have a rather deep disagreement about what to do with bad arguments. I think that some published views or arguments are not worthy of attention, and that the fact that an argument is not worthy of attention is a great reason not to cite or discuss it. (I agree with you, of course, that the fact that an argument was published in a less prestigious journal by someone you’re not buddies with is a poor reason.)

anon
anon
9 years ago

Jonathan, this is anon from 6:44. I take it that your remarks were directed at Marcus and not me, but I want to address this: “I think that some published views or arguments are not worthy of attention, and that the fact that an argument is not worthy of attention is a great reason not to cite or discuss it.” I unequivocally reject this claim. That an argument is bad is a great reason not to discuss it, that I grant. But I deny that an argument’s being (in your opinion) bad is no reason not to cite it. Rather, on my view any article in a peer-reviewed philosophy journal that fits the relevant thesis/distinction/argument ought to be cited. What matters for citation is *topical relevance*, not how good you think it is.

I also want to note that I fully agree with some previous commentators that certain fields go too far in the direction of over-citing. On my view, the error these fields commit is not one of citing too much ‘bad’ work. It’s the error of citing too much ill-fitting, i.e., topically irrelevant work. E.g., citing orthogonal or only marginally relevant literature is unnecessary, but it’s *because of a lack of topical fit* that it’s unnecessary, *not because it brings in too much ‘bad’ work.*

It’s an interesting further question how we might adjudicate between this ‘topical fit’ view of what ought to be cited (which I take it is also close to Marcus’) and your ‘normatively good work’ view of what ought to be cited. One might look to downstream consequences of each view for the profession, for individual contributors, and so on. But, as my previous post suggested, I’m inclined in a different (though compatible) direction, on which we defend the ‘topical fit’ view on the grounds that to exclude relevant work itself *constitutes* a harm to the excluded author, the harm of not having one’s work recognized by one’s peers. (I wonder whether there’s an interesting connection to Miranda Fricker’s work here, esp. to her claim that testifiers can be harmed in virtue of not being believed).

Sacco
Sacco
9 years ago

Ichikawa writes: “…publication would be too onerous [on Marcus’ proposal].”

I’m sorry, but this has to be one of the poorest excuses running through this thread. That your job would be more difficult if you cited more topically relevant publications from the literature is not a good reason to refrain from doing it.

Marcus Arvan
9 years ago

Anon: Thank you for answering Jonathan on my behalf. That is precisely what I want to say. Jonathan is conflating one legitimate thing (not *discussing* bad work) with an illegitimate thing (not *citing* bad work). A scholar had every right not to discuss work they don’t like. They do not have the right not to cite things they don’t like. I may think all of Hegel’s arguments are bad, and if I do, I do not need to discuss Hegel in my own work. But this does not give me a right to talk about broadly Hegelian ideas and not cite him simply simply because I think he did a bad job with them. Good or bad, he had the ideas first and deserves to be credited for them. So do our colleagues who have published arguments we don’t like. We may not like them, but they *exist*, and therefore should be credited *as* existing.

Clayton
9 years ago

I just wanted to register agreement with most of what Ichikawa said above. Anon above wrote, “Rather, on my view any article in a peer-reviewed philosophy journal that fits the relevant thesis/distinction/argument ought to be cited. What matters for citation is *topical relevance*, not how good you think it is.”

I disagree. In fact, I want to register strong disagreement on this point. Suppose you wanted to write on the lottery paradox. Just a quick check on google scholar yields 94 results for this search: “lottery paradox” & “rationality”. Since 2014. That number climbs over 1000 if you drop the restriction to articles written since 2014. If you’re going to write on that paradox, you’re probably going to want to discuss its significance for some way of thinking about rationality, justification, or knowledge. If you wanted to cite, say, everything of topical relevance to a paper on the lottery paradox, rationality, and, say, sensitivity theories of knowledge, Google Scholar yields another 10,000 papers when you search: Nozick, sensitivity, knowledge. Ought implies can.

Having said that, I agree that it shouldn’t matter to citation whether the author -believes- a paper on a topic is good. What matters is whether the paper is good in the sense that it makes a good contribution to the relevant discussion. Topicality matters as does originality. I think we should aspire to citing good contributions to a discussion with an eye to giving the people who deserve credit for good ideas that credit while monitoring for things like bias. I think the idea that you cite everything that’s topically relevant is an incredibly silly idea.

Roger Clarke
Roger Clarke
9 years ago

Marcus, can you say something about the difference (if there is one) between citing *everything on your paper’s topic* and *everything relevant to your paper*? Maybe an example? Say, if I’m writing about epistemic contextualism, do I need to cite everything published on epistemic contextualism in the last n years?

I agree that “X is a bad paper” is not a good reason not to cite X, but I think I agree with Beau that “X is irrelevant to my argument/thesis” is a good one.

@Sacco (45): “publication would be too onerous” is not the same as “publication would be too onerous for me.” If you have to read everything before you can say anything, people new to philosophy won’t say anything (not to mention people with obstacles to reading everything).

Walto
Walto
9 years ago

I think it ought to be fairly clear from a look through the original post and all the comments resulting from it that (i) many philosophers don’t cite sufficiently; and (ii) one can’t actually cite every relevant paper (and shouldn’t if one could). A parade of possible refinements and counterexamples to a PROPER CITATION PRINCIPLE has, naturally, ensued, and my sense is that it could be continued for at least several years.

My own suggestion is that those interested in this matter give up attempts to define things like “derived from” or “sufficiently relevant to a topic at hand” here, and instead provide an example or two of what they take to be appropriately cited papers or books. We can then get a sense of what people are actually shooting for and whether there is even agreement on that.

I will take up this suggestion myself and hereby offer up Rawls, A Theory of Justice. I think Rawls amply covered both recent and classical versions of his associated theses in his cites, provided (generous) pointers to people and works that had influenced him , gave copious examples of previous attempts that he disagreed with etc. But the book is nevertheless not crammed with everything ever written on the matters of justice, goodness, utility, Pareto, social choice, value theory, etc. Anyhow, it seems to me about optimal on that front (if not on all others). [Caveat: It’s a very long book and has a lot of footnotes.]

Do others think that’s about right? Are you looking for more? Less?

Cathy Legg
Cathy Legg
9 years ago

Marcus thank you for putting forward these ideas. One reason I think they are important is that the process of “canonization” in Philosophy is typically much more gender biased than other professional recognitions, such as hiring and being accepted for publication. This can be seen in challenges and discussions that have taken place over the past few years concerning textbooks. The issue of who gets to be seen as “agenda-setting” is key here.
I disagree with those in this thread who have argued that adopting Marcus’ proposals would overwhelm the readers of Philosophy articles with unmanageable quantities of citations that don’t add anything to the argument of the paper. It’s more about choosing more thoughtfully what one does put in. This might also improve the paper itself. It has been my observation that in the published literature in Philosophy – “off Broadway” work by relatively unknown (i.e. uncited) people is often just as interesting and valuable as that produced by the rock stars of the profession, who are overwhelmed with engagements with their work.
I would love for our discipline to become more rich and varied in its discussions, and more inclusive and fair in its institutional rewards, and I think that these two things are not unconnected. For that reason, for the past few years I’ve been making a conscious effort to read, engage with, and cite the work of people who are not hugely cited but who I enjoy reading and learn from. As Marcus suggests, this is one small thing that we can all do as fellow-participants in this institution of Philosophy, without too much effort, on our own initiative, to bring about change.
p.s. This is now a very long thread but I can’t help noticing that mine seems to be the only recognisably female name on it – ?? This makes me a little nervous, but I will weigh in anyway.

Charles Pigden
Charles Pigden
9 years ago

I think that Marcus Arvan’s proposal is a silly one and that, if it were put into practice, it would have a deadening effect on at least SOME kinds ‘off-Broadway’ and ‘out-of-the-box’ philosophy, confining philosophers to their pre-established areas of specialization. Let me explain with a bit of methodological autobiography. Here’s how I often work. I come across a book or an article which gives me an idea for a paper, in an area that I have not studied (much) before. I read around a bit until I am reasonably confident (though not a 100% certain) a) that I have not been anticipated and b) that the reason I have not been anticipated is not that my idea is either stupid in itself or open to obvious objections. Then I write it up and submit. What I DON’T do is trawl through the recent and often rather boring literature looking for vaguely relevant papers to cite. And though I am often (though not always) a relatively generous citer with lists of references running to between twenty and fifty items, most of those citations are not the kinds of citations that Marcus Arvan wants me to make, that is citations to recent journal articles by young or youngish philosophers. Instead they are citations to the mighty dead or to histories, newspaper articles, novels, biographies, plays autobiographies and court memoirs or to scientific studies of one kind or another (mostly in psychology). Now if I had forced myself to follow Marcus’s prescriptions most of these articles would have been indefinitely deferred as I ploughed my way through the journals looking for relevant stuff. ‘Small loss’ you might think. But actually the eleven out-of-my-bailiwick papers that I have published between 1985 and 2015 (four of them co-authored, one with a woman) have an average citation rate of seventeen, including citations by lawyers, political theorists, classicists, psychologists, communications theorists, historians of religion, literary scholars, ethnologists, climate scientists, sociologists, theologians and semioticians. They represent just the kind of engaged and interdisciplinary philosophy that others have been pleading for elsewhere on this blog. So deterring such publications would not be a Good Thing. Now the fact is that you simply cannot write the kinds of articles that Marcus wants you to write unless you are thoroughly steeped the relevant literature, since what you haven’t read you cannot (honestly) cite. Which means in practice that it is difficult if not impossible to write on topics outside your areas of specialization, since that is the only literature in which you can expect to be steeped. Marcus Arvan’s proposal is therefore a recipe for dull, unadventurous, self-involved philosophy, of no interest to anyone outside the profession.

It is not just that the suggestion, if followed, would have unfortunate effects – it would not have the good effects that Marcus hopes for. What really gets his goat is that Philosophy as a discipline is a winner-takes-all-economy as defined by Frank and Cook (1995) . The rewards in terms of fame and attention are distributed in a convex sided pyramid with a very wide shallow base and a narrow lofty steeple. Most papers are not cited at all whilst a lucky few have tens or even hundreds of citations. He thinks this is due to the moral failings of famous philosophers who wantonly disregard the young and struggling. Not so. For the fact is that ALL academic disciplines have a winner-takes-all structure. Furthermore, this is largely due not to the nefarious citing practices of famous old fogies but to scholastic acts between consenting adults, scholastic acts of which he himself is almost certainly guilty. What most of us want, in our heart of hearts, is to be, and to be recognized as being, brilliant philosophers who make a major contribution to the discipline. But this is an ambition that most of us will never achieve. Why? Well books and articles pour off the presses at a rate than no single individual can possibly keep up with. The attention that we can pay to others is finite. Ditto the attention that others can pay to us. There just isn’t world enough and time to read more than a fraction of what might be relevant and even good. (And the problem is compounded if you happen to have wide interests.) So when it comes to what we buy with the currency of attention we all have to be viciously selective, especially as reading a philosophy paper, and still more a book, represents a sizable investment in terms of time and effort. Nobody wants to waste their time on a dud. We are all like the buyer in the airport bookstore who wants a good book to entertain herself on a long-haul flight. She doesn’t not want to be bored on the flight across the Pacific (since that would make a bad thing worse) so she plumps for the latest Kay Scarpetta novel from Patricia Cornwell. And this is not because she thinks that there aren’t better books in the same genre but because she does not have time to do a search and because she knows that Patricia Cornwell is a reliable purveyor of the right kind of literary product. As it is with thrillers, so it is with Philosophy. A few top names get the lion’s share of the attention and citations, whilst a great many people not very much less talented (or maybe even more so), languish in relative obscurity. We tend to read papers by names that we recognize or which have been recommended to us by people that we trust or admire, because then we can be reasonably confident that we won’t be wasting our time. Furthermore there is a tendency to read the paper that everybody is talking about even if you privately think that there are better papers in the offing, simply because everybody is talking about it and you don’t want to sound like an ignoramus. All this does not mean that you are in direct competition with your colleague down the corridor. On the contrary, her fame can contribute to yours and vice versa if you cite her in your successful paper and she cites you in hers. The competition is rather at the collective level. If somebody somewhere is reading one of your papers then somebody somewhere else is not getting read. When it comes to attention and fame, philosophy at large is a limited-sum game.

So even if the profession at large tried to follow Marcus’s precepts it would achieve at best a slight mitigation in the winner-takes-all effect. The benefits of his proposal are slight and the costs heavy. I would shout ‘Don’t do it’ did I not know that most people are not going to do it anyway.

Charles Pigden
Charles Pigden
9 years ago

CONCAVE! I should have said that the pyramid is concave.

Turdstile
Turdstile
9 years ago

I believe that citation should be strongly value-laden in this sense: there is an obligation not to cite papers hidden behind the paywalls of for-profit publishers who engage in exploitative pricing. The moral obligation to adhere to certain accepted canons of scholarship perpetuates a system that uses the scholar’s sense of duty against itself.

Turdstile
Turdstile
9 years ago

Charles Pigden, whose work I greatly admire, acknowledges that philosophy–and all of academia–is a winner-take-all game. If so, what obligation do the losers have to support the winners? Should the winners not be satisfied with what they have already won? Why should the losers support the winners and further compound their losses?

Grad Student
Grad Student
9 years ago

Academic philosophy is a professional discipline wherein academic philosophers are paid to perform certain services. The primary two services are pedagogy and research, with others (e.g. institutional service) coming somewhere after these two. Philosophers are not hired and paid in virtue of their research because the university wants to help them develop their ideas and become beautiful unique snowflakes. Academic research is a collective enterprise. Period. Philosophers are being paid so they can build up a collective body of knowledge that is of some value to the rest of humanity. If you (generic reader) think that this is a mistake and that philosophy simply cannot be done in this way, then the inevitable conclusion is that philosophy must be pursued outside academia. In this case either one of three things must happen. (1) Academic philosophy should be abolished in its entirety. (2) The reward structure of academic philosophy must change so that academic philosophers are only paid and rewarded based on their pedagogical skills. (3) We must go to the public and convince them to pay us to share our ideas with one another (i.e. subsidize journals, hire based on publications) even though we do not aim at creating any sort of consensus view that might be of use to other parts of society. To go on pretending that we are doing the same thing as the sciences when we are really just writing things for funsies and ignoring what other philosophers outside of our respective cliques have said is to accept the role of con-artists and social parasites. Anyone who complains about the “professionalization” of what is–by its very nature–a profession is doing exactly this.

I, for one, fully embrace the idea that academic philosophical research is a collective enterprise. Thus as far as I’m concerned the idea that one is not obligated to deal with ideas that could be relevant to one’s argument and the establishment of one’s conclusion is wholly unacceptable on its very face, as it conflicts with the intrinsic goal of academic research.

Whether one should also cite things that might relate to one’s general topic that do not bear on one’s specific argument is an additional question that is to be answered by broader social considerations. However, it should not be run together with the more basic obligation, which cannot be in question. This obligation means that one must cite anything that has a direct bearing on the establishment of one’s conclusion. If one is arguing for p then this must include all other arguments for p, as well as all other arguments for ~p, regardless of whether one considers them good or bad. If this seems too burdensome, it is because p itself is too broad and you should instead be arguing for q, where q is either a conjunction of p and other theses or something more specific that clearly supports p. This simply amounts to the claim that one’s paper must be reasonable in its scope. Anyone who rejects this does so from the mentality that philosophy is an isolated enterprise where lone geniuses weigh in on questions of incredible depth. We may call this the “college freshman mentality”. If narrowing the scope of your paper is too hard, then you are not cut out for an academic career. You can gesture at some broader implications in your conclusion, and can do so without citing massive numbers of people. What you cannot do is claim to have provided good evidence for some broad conclusion while ignoring what other people have said about the topic, as this violates the requirement of total evidence.

Finally, it may be the case that reading all of the relevant literature is hard. The solution is fairly straightforward: publish at your own risk. The blind reviewers can try to catch things as best they can, but they are ultimately not responsible if you miss something. A certain amount of faith on the part of the journal is involved. If you miss something and someone spots it, then they should contact the journal and the journal publish the correction in the editorial notes. If the mistake is particularly egregious (e.g., if your entire argument has been given before/you blatantly misrepresent your opponent/you blatantly ignore an objection to your position that is directly relevant to the success of your main argument) then the journal should publish a retraction. Hopefully your initial convenience was worth the eventual professional humiliation. What we really need to do is to start an organized effort to call people out: i.e. some established forum for submitting complaints about improper citation that can then be verified. If everyone emails journal editors in a piecemeal manner the editors will likely ignore it. After all, it obviously hurts the reputation of their journal if they have to publish editorial notes and retractions.

Charles Pigden
Charles Pigden
9 years ago

Turdstile grants that Academia in in general and philosophy in particular are winner-takes-all economies at least with respect to fame and attention. S/he then asks ‘what obligation[s] do the losers have to support the winners? Why should the losers support the winners and further compound their losses?’ The answer is that the losers are under no *obligation* to support the winners *but in fact they will do so* since the winner-takes-all structure of the philosophical economy is an ‘invisible hand effect’, in which the collective result of our choices is not what we individually intend. This is a constant cause of frustration to many (perhaps most) of its participants. There are a lot of professors out there who are conspicuously not getting their hearts’ desire, namely fame and recognition as a gifted philosopher. (They are like all those the drug dealers in Levitt’s Freakonomics who are living with their mothers. Drug-dealing too is a winner-takes-all economy: for every drug-lord there is an army of drug-serfs. The difference is that an obscure philosopher can still make a comfortable living. It’s esteem and recognition that a they are not getting enough of rather than money, though for the army of VAPs and untenured staff, money too can be in short supply. ) To put the point another way, the number of desirable locations on philosophy’s mattering map is strictly limited and though this may be a problem that can be mitigated it is not a problem that can be fixed. We can’t all grow up to be famous philosophers for much the same kind of reasons that we can’t all grow up to be presidents and prime ministers. This is bad enough for the relatively pure in heart, people who don’t care that much about money or the outward trappings of success (prestigious fellowships and distinguished visiting professorships, named chairs, Locke lectures and all the rest of it). It’s a lot worse for those that do. Besides this, there are two other factors that compound the problem.

1) Financial success tracks philosophical success, though the financial rewards are a lot more evenly distributed than the fame rewards. (I have more than twenty times the citations that I had twenty years ago. I do not have twenty times the income.) It is nonetheless galling for many people to see the famous Professor X getting several times their salary especially if they think that Professor X is not a much better philosopher (or even a worse philosopher) than they believe themselves to be.

2) One’s place in the pyramid of philosophical prestige is partly determined by talent and partly determined by luck, including antecedent privilege. Privilege can include such things as being white, being male or coming from a wealthy, famous or well-connected family or from a family well-endowed with cultural capital (not necessarily the same things). It can include going to a prestigious school near a big metropolitan center (which itself is more likely give the above factors) . You can get the opportunities that amplify your talent or you can simply get lucky breaks. (To take a very obvious example, Wittgenstein was the son of a billionaire, whose spotty academic record could therefore be ignored, and who managed to find a generous friend and mentor in Russell. Would he have made it as a philosopher without these advantages? The answer, I suggest, is No. Wittgenstein chose to be a poor man in later life, but had he been a poor boy to begin with he would not have had the chance to be what he subsequently became.) Since success is often partly due to luck (as most of the successful are willing to admit) this tends to cause envy and resentment in the less lucky and the less successful.

The winner-takes-all structure of the philosophical economy is, I think, inevitable. (The meta-data on citations suggests that all academic disciplines have a similar winner-takes-all structure.) If you object to it, there are two questions that you have to ask yourself.

A) Do you want be recognized (note the success word) as a brilliant, gifted or even an above-average philosopher?
B) Can you think of a socially realistic scenario in which you could achieve this ambition without a winner-takes-all economy of esteem?

B) If the answer to A) is ‘Yes’ and if the answer to B) is ‘No’ (as I think in all honesty it must be) then you can’t in consistency will the end A) – a situation in which you are widely recognized as a significant philosopher – without willing the means B) – a winner-takes-all economy of esteem. You must give up either your ambition or your professed distaste for the hierarchical structure of the philosophical economy. Indeed, if you really want to abolish the winner-takes-all economy of esteem you must not only give up your *own* ambition to be a widely (and justly) admired philosopher – you must will that everybody else should abandon their similar ambitions both for excellence in their work and eminence in the profession. For to excel is to do better than others and to be eminent is to be better recognized.

But although the winner-takes-all structure of the economy of esteem is, I think inevitable, obviously it can be more or less extreme. The concave curve in the pyramid of prestige can be more or less severe, the spire can be more or less lofty and narrow, the base more or less broad and shallow. Furthermore the distribution of rewards within the pyramid can be more or less meritocratic, more or less due to luck and antecedent privilege. Though we can’t *all* be famous philosophers, it could be either more or less true than it is now that irrespective of antecedent privilege *each* of us when starting out could grow up to *become* a famous philosopher.

But serious debate on these subjects must begin with the admission that so far as fame and attention are concerned, philosophy *is* (and is likely to remain) a winner-takes-all economy and that it is our own scholastic choices (specifically our choices about what to attend to and how to pursue our careers) that make it so. What I found a little irritating about Marcus Arvan’s original post was that it seemed to me (perhaps wrongly) that he was in denial as regards this obvious fact.

Charles Pigden
Charles Pigden
9 years ago

Grad Student writes:

Thus as far as I’m concerned the idea that one is not obligated to deal with ideas that could be relevant to one’s argument and the establishment of one’s conclusion is wholly unacceptable on its very face, as it conflicts with the intrinsic goal of academic research.

Isn’t there an obvious ought-implies-can problem here? Given Quine –Duhem the number of ideas that *could* be ‘relevant to ones argument and the establishment of one’s conclusion’ is indefinitely large. Hence it is not possible to ‘deal with’ all such ideas. And if it is not possible, then given ought-implies-can, we can’t be ‘obligated’ to do it.

Turdstile
Turdstile
9 years ago

More than the “game of academic esteem” is winner-take-all: the market tends to be, though you point out that it is possible to make a living (I almost said ‘decent’) as an obscure academic. Winner-take-all games are non-cooperative. When humans cooperate, they tend to punish transgressors at some cost to themselves. Now it would be one thing if everyone accepted the non-cooperative, winner-take-all games of academia in stride. But I believe that there is considerable scope for confusion on this score. While the losers may have no obligation to support the winners (but they will otherwise), one has to be careful about the word ‘support’. I would take this in the widest possible sense, to include political and financial support. If there are too many losers, some of whom may end up in, for example, academic administration, the winners may find themselves at a loss.

I myself was a member of a research group. For whatever reason, I was not included in the publications of the group–I was relegated to technical support. This is not so uncommon in the sciences. I decided that if this were my fault, I should leave; and if this was for other reasons, I should leave–which I did. I decided not to assist others in their careers at the expense of my own. Even after I left there were peremptory, last-minute demands to edit NSF grant proposals. I’ve turned down four offers to participate in NSF grants since then–these were important research projects. But given the history, I could see that they would be exploitative for me. In my case, my non-participation was a loss for them.

The losers have no obligation to help the winners. The losers would simply get lost, their labor replaced by an endless pool of academic laborers, and that would be the end of it–if academia were free of exploitation. The tendency to punish transgressors at some cost to oneself is the flip side of cooperation. It would be an error to assign this to ressentiment. The winner-take-all ethos of academia is harming it.

Cathy Legg
Cathy Legg
9 years ago

Hi Charles! First of all, if, as you say, you are a “a relatively generous citer with lists of references running to between twenty and fifty items…citations to the mighty dead or to histories, newspaper articles, novels, biographies, plays, autobiographies, court memoirs…[et al]”, then it seems to me that you are already taking on board some of Marcus’ criticisms, which were of the narrowness of only reading the so-called ‘top philosophy journals’.

Secondly, on the issue of a ‘winner-takes-all’ economy. It seems that you view 95% (or more) of your peers and colleagues as ‘losers’. This makes me sad. Particularly as you yourself acknowledge that whether one ends up a ‘winner’ or a ‘loser’ in our current system is merely [in your words] “more or less meritocratic, more or less due to luck and antecedent privilege”.

I can only say once again that I’ve learned a lot philosophically from certain people who don’t have a lot of citations. Taking some time to find such people amidst the other demands of a busy career has seemed to me to be worth it, because of what I’ve learned.

Grad Student
Grad Student
9 years ago

“Isn’t there an obvious ought-implies-can problem here? Given Quine –Duhem the number of ideas that *could* be ‘relevant to ones argument and the establishment of one’s conclusion’ is indefinitely large. Hence it is not possible to ‘deal with’ all such ideas. And if it is not possible, then given ought-implies-can, we can’t be ‘obligated’ to do it.”

I was obviously restricting my quantifiers to the ideas that have been presented in the literature thus far. Given that, no, there is no ought-implies-can problem. It is well within the relevant capacity of scholars to search for relevant articles, though it is no doubt a pain. At any rate, the potential retraction of an article that does not cite relevant work is not justified as a retributive act against the author for not performing her duties. Rather, it is obligatory for a journal to retract published work that violates scholarly standards because of the journal has a duty to uphold these standards. Caveat Auctor.

Charles Pigden
Charles Pigden
9 years ago

Hi Cathy,
The term ‘losers’ was Turdstile’s not mine. I was merely quoting. I thought of putting the word into scare quotes to make this plain but did not think it necessary as I thought that contextual clues would do the work. Evidently I was wrong.

Do I endorse the terminology that I borrowed? Only in a trivial sense. If people are playing a game Y where the object of the exercise is to achieve X, and if they are consistently failing to do so then they are losers wrt that game Y and that objective X. This does not mean that they are losers in any other sense. Thus if a team is playing soccer and they persistently fail to score more goals than their opponents then they are losers with respect to soccer. They may be the acme of success in other departments of their lives. Now many (though, of course, not all) professional philosophers are playing the game Research in which the object of the exercise is to produce good (that is better than average) philosophy papers *and to be widely recognized by their peers for having done so* (which means that their papers must be widely read and cited) . And there are structural reasons why the majority of philosophical players are losers with respect to *this* game and *that* objective. (Basically there just isn’t enough attention to go around given the number of people who want it and perhaps deserve it.) This does not mean that they are losers in any other respect. Indeed I would be inclined say that anybody lucky enough to have a job teaching philosophy at a decent university is in many respects a winner since teaching philosophy is both worthwhile in itself and intrinsically rewarding (apart from the marking which is a bit of a chore). I go to my mostly undergraduate classes (four hours of teaching today plus my office hours today with a further four hours and movie showing tomorrow) in the happy consciousness that I am both having fun and doing good. One day I remember meeting my former colleague David Ward (not a very well-cited man) in the corridor. He is a naturally cheery person and I was in a cheerful mood myself. ‘Hallo Lucky!’ I said. David caught on straight away. ‘You mean because I get to teach philosophy?’ ‘Exactly’ I replied. I meant it.

I would also like to take issue with a remark in Turdstile’s post. S/he says: ’Winner-take-all games are non-cooperative.’ This is, I think, a mistake. It is important to understand that the winner-takes-all structure of the of the philosophical economy has nothing much to do with the cooperative or uncooperative behavior of the successful players.

Firstly a successful player can be the very model of a cooperative academic without forfeiting her success. She can be notable for co-authoring papers, mentoring her students and citing her interlocutors. There is nothing to stop such a person being both highly cited and well-recognized; in fact she is likely to do a little better in this regard than her equally talented but more selfish counterparts, who are less likely to get gratitude-citations’ or citations in papers that they have helped bring into being via their stimulating conversations, helpful comments etc. Being successful is one thing and being selfish or uncooperative is another and there is no reason why the two should coincide.

Now let’s consider just such a highly-cited person who also very paragon of a generous and helpful academic. She excels as a mentor, responds generously to criticism (always looking for what best in an opponent’s piece) and at conferences does her best to single out the talented but shy young person who might have something intelligent to say. She has a huge appetite for philosophy and is ready to talk about it till late in the evening, especially to young philosophers just starting out in the world. She does her best to cite relevant material using not only her own resources but the gossip that she can glean from her many friends. The upshot is that her students, colleagues, friends and opponents are all buoyed up by her influence, producing better and much more cited work than would otherwise be the case. Those lucky enough to be in her intellectual vicinity are much more likely to be citation-successes than those who are not.

This raises he following question. Is the generous behavior of this philosophical paragon going to make any difference to the overall shape of the winner-takes-all pyramid? And the answer is obviously ‘No’. For the basic fact that explains the winner-takes-all structure of the philosophical profession is that our collective attention is a limited resource. If somebody is reading and responding to your paper, then there is somebody somewhere else whose paper is NOT being read or responded to. My philosophical paragon raises the profile of the people around her but she does it at the expense of other people elsewhere who are now less likely to be read or cited.

Suppose that top of the spire were *mostly* inhabited by super-mentors of the kind I have envisaged. Would that make things any better? No. It would actually make things worse. For their activities would tend to suck away even *more* of the attention from those at the periphery towards those at the center. However virtuous they may be, these super-mentors can’t attend to everybody, and they can’t raise everybody’s profile. It is only those within the charmed circle of their influence that can benefit from their activities. And the more attention they focus on the members of their respective charmed circles, the less there is available for everybody else.

The situation has interesting parallels with the work of David Lewis who, as a teacher, colleague, conference participant and mentor, was rather close to the ideal that I have been sketching. It is sometimes objected that his modal realism is morally objectionable since it presents moral action as ultimately futile. Whatever we do we can’t make the plurivierse as a whole any better. The summit of our moral ambition must consist in improving things *around here*, that is, in the actual world. Well that’s what it is like for people like him. They can have a dramatic effect on those they come into contact with, but since the attention that we have to pay is limited, this is only at the expence of those they never meet. David did a lot of good to those lucky enough to come into contact with him (which did not necessarily involved meeting him the flesh though that certainly helped). But that too was a case of improving things ‘around here’.

Let me stress yet again that the situation that people are complaining of is to a considerable degree an invisible hand effect which reduplicates itself in many, perhaps most, academic disciplines. It is mostly not due to anybody’s moral failings (though these can make things worse in various ways). The fact is that many of us want to be, and to be recognized as being, significant research philosophers. But the actions we individually take in pursuit of this ambition make it collectively likely that most of those who share that ambition will be frustrated. That’s the real problem and there is not a lot that can be done about it absent a collective change of heart and a radical reform of the institutional cultures. Are you willing to give up your ambitions comrades? Cos I’m not.

Turdstile
Turdstile
9 years ago

I more or less agree with Charles Pidgen about the philosophy Research game, which is structured as a tournament. Its outcome has a power-law distribution. Perhaps winner-take-all is misleading–most game theorists consider winner-take-all games to be non-cooperative. My experience is mixed. The merit-driven Research game was one game among many; the other “games” were more exploitative and one-sided. Perhaps it was my misfortune to have known too few generous and helpful academics, well-cited, or otherwise. I will leave it at that.

Cathy Legg
Cathy Legg
Reply to  Turdstile
9 years ago

Apologies for the tardy reply. To Charles: the term ‘losers’ was not introduced by Turdstile, but implicit in your extensive talk of ‘winners’, so scare-quotes would be disingenuous. You write: “Is the generous behavior of [a good colleague, previously described] going to make any difference to the overall shape of the winner-takes-all pyramid? And the answer is obviously ‘No’. For the basic fact that explains the winner-takes-all structure of the philosophical profession is that our collective attention is a limited resource.”
First of all, I just don’t see that this follows, any more than it follows from the fact that human beings have finite capacity to earn money, that 1% of the population should own, say, 90% of that money. There are other options in the economic sphere, and in the intellectual.
At this point your defence of the ‘Invisible Hand’ in Philosophy argues that in order to ‘interfere’ in the inevitable rise of the teetering pyramid, you must “give up your ambitions”. And of course nobody wants to do that. But what are your ambitions *for*? To get heaps of citations? Or to read and write as much excellent philosophy as possible? These are clearly not the same thing, but it seems that you wouldn’t know that, Charles, as you say that you only read papers that, like airport novels (!), are reassuringly popular with everyone and give you what you’re already expecting, and the papers that everyone else is talking about so that you won’t feel left out.
One last thought: A crazy experiment: what if we replaced the term ‘winner takes all’ system with ‘winner serves all’ system? How would we think about our profession then? I actually suspect such models don’t lie too far back in time.

Anonymous
Anonymous
8 years ago

Does the perceived quality of a journal really mean anything? Are women less likely to submit to the top journals? (Anecdotally, I am a woman, and I tend not to submit to the very top journals in my field because I’d rather have my work see print sooner. So I tend to write book chapters for editors in my network and articles for less well-known journals. I have had male mentors who questioned why I would do this. I suppose it could hurt me in not being cited as often. However, I don’t think the quality of work would change if I wrote for a different journal. I agree that it is foolish for scholars to judge work by a journal cover.)