Citation Problems in Philosophy—and Some Fixes
Philosophers widely violate the academic norm to “cite work that is clearly relevant to the topic at hand,” claim Meena Krishnamurthy (Michigan) and Jessica Wilson (Toronto), in a post at the What’s Wrong? blog.
They identify some varieties of citation failure, and argue that it’s a problem worth taking seriously. Failure to cite people’s relevant work deprives them of various professional benefits that often follow from being cited. It also, they argue, reinforces harmful biases in the profession. Additionally, philosophical progress is stymied when relevant arguments and examples are missed, leading possibly to vast literatures “founded on false presuppositions” as well as the wasteful reinvention of wheels.
What should be done about this?
“When doing literature surveys we need to ‘go deep’ down the list if we aim to identify all work that is clearly relevant to the topic at hand.” This may sound onerous, but it’s our job:
In many other disciplines (e.g., linguistics, history, physics and the other sciences), practitioners are expected to carry out, and do carry out, full literature reviews and to cite any clearly relevant literature accordingly. Philosophers do not appear to face any special burdens so far as literature that must be surveyed. If practitioners of other disciplines can carry out such surveys, then so can we…
[P]art of what we are pushing for is a reconception of how much time philosophers devote to scholarship. Yes, given current practices whereby many do not engage in anything like full literature surveys, implementing our suggestion is going to involve some rearrangement in one’s work habits, whereby sufficient time is scheduled for and devoted to engaging in finding out the actual dialectical state of play as regards the topic at hand. The resulting paper and associated dialectical context will be better for it, however—as will our profession and its practitioners.
We can also have an impact on the citation practices of others:
We suggest that the quickest way to improve citation practices in philosophy as a whole would be for large numbers of individuals to commit, in their capacity as journal or other referees, to rejecting for publication papers or other submissions that fail to cite work that is clearly relevant to the topic at hand, on grounds of failing to meet basic standards of scholarship.
Additionally, individuals should
be proactive on their own and others’ behalf, by contacting authors of papers, encyclopedia articles, etc., to let them know of citation failures. One convenient way to do this is to sign up to receive PhilPapers updates on one’s areas of interest in order to get a heads-up about recent work. If a piece of work shows up that is lacking appropriate citations, to oneself or to others with whom one is familiar, then one can take a few minutes to contact the author and nicely inform them of this.
The whole post is here. Thanks to David Boonin (Colorado) for the pointer.
I also refer readers to “Philosophers Don’t Read and Cite Enough,” a guest post by Marcus Arvan (Tampa) here at Daily Nous, which I note—with a touch of friendly irony—is not cited in the post by Krishnamurthy and Wilson.
It seems to me that it may be wise to exclude references from word counts.
(Sorry hit enter accidentally too soon). That seems like a good idea because a quick way to trim a paper is to eliminate all discussion that is not absolutely essential to the main argument. To the extent that some other author’s work is relevant but not in direct logical contact with one of one’s premises, that is likely to go.
But word counts are there because print journals have a page budget, and they want to ration it out carefully. Unless you can have a way of the references not appearing on the page, it seems reasonable to take them into account.
For electronic journals, letting the citations section be arbitrarily long seems perfectly fine. I don’t particularly find long lists of citations that helpful – journal articles don’t need to replicate PhilPapers categories – but there isn’t a harm there. But if current articles increase their citations, then fewer articles will be published in print journals.
I’m a little suspicious of the claim that we’re falling down on the job here. I admit I may not be as scrupulous as I could be, tracking down every instance of some particular argument or trope, but I don’t think that’s a reasonable standard for journal papers, and less so for blog posts or conference papers. I am certainly sympathetic to the claim that we ought to be doing a diligent “deep dive” when publishing a monograph, and our editors ought to be part of that practise; I think we are doing that in general, though. So this seems to me to be a little bit of a solution in search of a problem.
Extrapolating from my own experience and from that of many others I have spoken with, citation failures are common: pretty much any mid-career woman, even when comparatively prominent, will have examples to share, as will non-elite philosophers more generally. People are naturally often hesitant to go on the record about specific cases, though, especially when the perpetrators are well-placed, as they often are. Meena and I didn’t highlight specific cases because our goal was not to blame, but to motivate improved citation practices.
Still, if you want cases in point, here are two. One concerns the proper subset account of realization, which I discussed here: http://philosopherscocoon.typepad.com/the_campaign_for_better_c/2015/03/jessica-wilson-toronto-on-the-pedigree-of-the-subset-view-of-realization.html. A second is discussed at the start of ‘No Work for a Theory of Grounding’, where I observe that the revolutionary rhetoric of the original progenitors of this notion (short cut: since Quine, philosophers have ignored metaphysical dependence, focusing only on what exists or at best supervenience; but philosophers should attend to such dependence; so we bring you… primitive Grounding!) doesn’t bear any resemblance to actual dialectical reality.
In my experience, philosophers have a different reaction than other scholars to being cited. Philosophers are much more sensitive to the risk that their views are being misrepresented or not being captured at the level of specificity they deserve. And again in my experience, these complaints can be very frustrating to engage with. Each citation needs a paragraph to explain exactly what the view is and how it relates to your topic. And even then you might not satisfy the person you’re citing – and they can be pretty rude about telling you that!
As a result, I think this results in one of two citation methods: The ‘pile’ method, which just lists related work without any more detail. Then you can’t be accused of misrepresentation. But such a ‘pile’ is also basically useless! Every paper presents the same pile, and you don’t actually benefit from in-depth consideration of the literature. (This is discussed in a nice Classics paper by Alexander Nimis, “Fussnoten: Das Fundament der Wissenschaft”. ) The other method is to restrict citations to the works that you think are directly relevant and worth taking the time to explain in depth. (And maybe that are written by your friends, who will appreciate being cited and not be as likely to get annoyed at you if they think you’ve made a mistake in presenting the view.)
So I think that to change citation practices in philosophy we need to rethink how we respond to seeing other philosophers cite our own work.
Like Daniel, I worry about the “piling” phenomenon, which in my opinion, is the norm in a lot of other fields. I’ve sometimes gotten the impression that authors in these fields do a keyword search or pilfer bibliographies from previous but related articles in a field, but don’t really do a proper reading of the texts that they’re citing. (Of course, this also happens in philosophy, despite the fact that we don’t have large piles of citations.) One potential solution is to have dedicated lit reviews that ARE exhaustive and comprehensive (perhaps by modifying one or more existing online encyclopedias), and to require that these be cited for purposes of publication. Presumably, authors and referees would be expected to have read these comprehensive survey articles.
I like this idea. It’s not optimal, since it doesn’t assist in citation counts for individuals, but comprehensive regularly updated literature surveys would at least provide a concrete baseline for what authors writing on a given topic should be expected to know. At least some Stanford encyclopedia articles are playing this role.
Justin – Thank you for pointing us to Marcus Arvan’s post at DN. We are in the process of updating our post so that it cites his – which is clearly relevantly related!
I agree with everything Krishnamurthy and Wilson say, and I consider it to be a very careful and thorough treatment of the subject. Yet I had to wonder while reading it why all this needed to be spelled out, and also whether it really does. In my experience the problem (at least for my generation) does not stem from lack of information about what correct citation practice requires. It’s hard for me to imagine that anyone (at least of my generation) is unclear about that. The deeper issue concerns the *professional consequences* of whom one chooses to cite or not cite (this point has already been made in the Philosophy Citation Practices Revisited discussion).
An example: In my first philosophy job, I taught an outstanding graduate student whom I’ll call XY. I was denied tenure at that job, and my colleagues and I did not part on good terms. XY went on to do a dissertation in that department. XY recently published an article in a very highly regarded journal, which I’ll call TopDog. The editors of TopDog are colleagues of mine of long standing. XY’s article made central use of concepts I had defined in an article I had earlier published in TopDog, and on which one of its editors had commented at a conference. XY’s article contained no citations to mine. If XY had included such citations, he/she would have endangered his/her relationships with his/her dissertation advisors. If the editors of TopDog, who could not fail to be familiar with my article, had insisted that XY cite it, they would have both victimized XY, and also endangered their own professional relationships to his/her dissertation advisors.
These are the sorts of considerations that led Matt Drabek (here http://philosopherscocoon.typepad.com/blog/2015/11/long-journeys-part-7-matt-drabek.html ) to close his very compelling essay by describing philosophy as dominated by a “culture of fear.” When that fear becomes so pervasive that it requires this kind of substandard scholarship as a condition of professional wellbeing, regardless of one’s tenure status or stature, the conclusion seems inescapable that in philosophy, sound scholarship and professional wellbeing are mutually exclusive alternatives.
I was initially rather sceptical that “cite work that is clearly relevant to the topic at hand” is a sensible norm. In itself, that sounds (a) too broad, and (b) broader than the actual norm in other fields (or at least: in physics, the field I know best outside philosophy).
However, Krishnamurthy and Wilson give a concrete breakdown of what they mean by “relevant” which is much narrower and much more persuasive – but still, I think, slightly too broad in one respect. To quote:
“First are cases where an individual (or individuals) fail to be properly credited for a specific argument, approach, or account, as when, e.g., philosopher X has a claim equal to or better than that had by philosopher Y to be considered an originator of account A, but A is typically attributed only to Y.
Second are cases where the presentation of a given dialectic ignores a large existing literature, as when, e.g., philosopher Z claims that approach A has been ignored or rejected in past decades, even though dozens of philosophers have been working on and developing A in past decades.
Third are cases where a philosophical wheel is reinvented, as when, e.g., philosopher Z proposes ‘new’ account A as the best way to accommodate phenomenon P, even though most or all existing accounts of P take A as their starting point.”
The first seems clearly correct; the third likewise, but really as a special case of the first. (If account A is being presupposed by existing accounts of P, then account A is not original to philosopher Z and so its originator needs to be credited.) The second I might agree with, depending on the details. (I’m convinced in any case by the specific example, but on other grounds: if philosopher Z claims that approach A has been ignored or rejected but actually dozens of people have been working on it, philosopher Z’s claim is *false*, and culpably so, since Z would have known about the falsehood if they’d done a proper literature search.)
However, Krishnamurthy and Wilson’s piece *can* be read as suggesting that any individual “clearly relevant” piece of work needs to be cited, even if there is no idea in the cited paper that needs to be acknowledged. That’s too strong. To take a relatively-specific example: if I write a paper on the direction of time in thermal physics, I am obligated to (1) acknowledge the originator(s) of any argument, approach or account used in my paper; (2) do literature due diligence to make sure ideas I think are original to me haven’t originated elsewhere. I don’t think I have other obligations of scholarship (I might have obligations of pedagogy) that require me to cite every other way of accounting for the direction of time that might have been suggested over the years. A research paper is not a review article; the point of citations is specifically to acknowledge authorship.( I’m sympathetic to the argument that philosophers don’t even do this properly!)
I think the concern at the end here falls under the heading of ‘use your brain when determining what is clearly relevant to the topic at hand’. What exactly you have an obligation to cite will depend on exactly what the topic at hand is. If you are arguing for a specific way of accounting for the direction of time, then perhaps you do have an obligation to register somewhere other suggestions. If you are arguing that a specific way of accounting for the direction of time has the resources to respond to a particular objection, then perhaps you don’t.
Another failure of citation that I’ve seen: prominent philosopher X misinterprets philosopher Y, and then subsequent philosophers propagate X’s misinterpretation because they don’t bother to read Y.
I’d think that we should cite papers that contain ideas that we (do — or, by extension, should) refer to in our own work. If a paper is (somehow) published on the relevant topic but doesn’t actually contain any interesting or relevant ideas, should we cite it anyhow? What would be the point?
(Insofar as citations are professionally rewarded, that’s presumably at least in part because they are regarded as a proxy for quality, or at least for producing ideas that other academics consider worth engaging with. If citations lose that meaning, and instead become merely a sign that someone has subsequently written a paper on the same topic as one that you managed to get published, it’s hard to imagine that they would continue to have the professional significance that we currently attribute to them. So this argument for revising our citation practices seems practically self-defeating in any case.)
I’m also not convinced that the opportunity costs of increased time spent on scholarship is adequately addressed by saying that “if others do it, so can we.” The worry, after all, is not that we literally couldn’t do it, but just that the benefits are outweighed by the costs — most obviously: less time to spend developing new ideas. Now, IF it’s true that many philosophical mistakes (reinventing wheels, missing crucial objections, etc.) are currently being made due to inadequate scholarship, then plausibly the benefits would outweigh these costs: better to have fewer, better-targeted and higher-quality papers, sure. But is this criticism really accurate? It’s at least not obvious to me. (Insofar as I have any complaints about the broad state of the philosophical literature, it would be that there’s an over-emphasis on minor scholarly details, and not enough development of really interesting, ambitious new ideas.) For example, when I see “vast literatures founded on false presuppositions” I tend to conclude that those involved have a fundamentally different approach to philosophy (or different fundamental intuitions, etc.) than I do, not that they’re simply *unaware* of the objections raised by those who share my way of thinking. So I’m dubious that more time spent on scholarship would have such significant philosophical benefits. I rather fear that it would just lead to fewer, more bloated papers (and hence even fewer interesting new ideas). Maybe I’m wrong about that; but that seems to me the crucial question, anyhow.
“If a paper is (somehow) published on the relevant topic but doesn’t actually contain any interesting or relevant ideas, should we cite it anyhow? What would be the point?”
As far as I can see nobody is suggesting that, though. It’s really frustrating to see this straw-man trotted out at every opportunity.
That absolutely is being suggested. Krishnamurthy and Wilson explicitly say we should cite everything relevant, and not exclude things based on our judgment about quality. That seems to be obviously unworkable.
Notice that K&W think you should cite *everything relevant*. Chappell, on the other hand, said “[…] but doesn’t actually contain any interesting or relevant ideas […]”. K&W are talking about ideas that are relevant, Chappell about those that are not.
Ah, we should cite the relevant but uninteresting ideas from 3000 years of philosophical discussion. Okay, you’re right, that’s much more reasonable.
The ‘reply’ button isn’t showing up for anon prof’s last post, so:
K&W actually say “cite work that is clearly relevant to the topic at hand”. That’s pretty reasonable.
Anyway, you’re (deliberately?) missing my point: that they think relevant literature should be cited actually *confirms* what I said, it doesn’t *disconfirm* it. “Relevant literature” is the *opposite* of literature that “*doesn’t* contain […] relevant ideas”. See the difference? One set contains relevance, the other does not.
Grad Sockpuppet, I have tremendous respect for your persistence and skill in using the tools of analysis and exegesis to clarify the surprising misreadings of K&W that have surfaced in this discussion. Hang in there.
Thanks, Grad Sockpuppet. Your reading of our suggestion is correct.
Richard, re the ‘crucial question’, see my reply to Will Behun, above. The first case is illustrative of the disbenefits accruing to an individual (namely, me) from poor citation practices; the second case is illustrative of the disbenefits accruing to the profession as a whole—IMHO, anyway. So yes, I do think the dialectically better-pitched game would be worth the candle.
Re whether we should cite papers “(somehow) published on the relevant topic [that don’t] actually contain any interesting or relevant ideas”—that’s not our suggestion, and indeed (as Grad Sockpuppet notes) is incompatible with our suggestion. As per usual when operating with non-algorithmic principles, when figuring out what is “clearly relevant to the topic at hand” one has to use one’s brain. And as you agree, figuring this out and citing accordingly is within the realm of the possible.
Finally, re the concern that following our advice will lead to citations being degraded from their current ‘honorific’ status, such that our suggestion is in some sense self-defeating: I don’t see this. It will remain that cited authors will get credit for their published contribution, will be more likely to be read and discussed, and will be seen as contributing to a topic of ongoing discussion. These markers will serve as bases for the continuing honorific status of citations—indeed, markers that arguably track genuine value better than present elite-centric citation practices do.
(Put more briefly: I agree with David!)
I highly recommend Helen de Cruz’s “Bechdel Test for Philosophy Papers” in relation to this:
(1) At least two women cited,
(2) Actively engaged with at least one female author,
(3) At least one woman mentioned not only because she discusses a man
These criteria may be neither necessary nor sufficient, but then what criteria are?
In my subfield, I find it difficult to find *men* to cite. It may just be because women have done the better scholarly work.
With apologies, I want to go meta. (This comment is a remark in metametaphilosophy. I’m glad we weren’t already just talking about metaphysics or metaethics…) I think it’s very difficult to have this conversation in the abstract—it’s not obvious to all, but it seems to be to some, that philosophers don’t cite enough in general. And principles purporting to lay out the proper standard of citation are likely to involve significant judgment calls about e.g. which arguments are ‘relevant’. Compared to some suggestions I have seen, the criteria laid out by Krishnamurthy and Wilson seem relatively weak. (Grad Sockpuppet above suggests that no one suggests that people should cite papers on topic even if they lack interesting or relevant ideas, but I think this is wrong—Marcus Alvin has argued that one should cite everything relevant, whether or not it’s any good.) But statements of such generalities are a bit too close to tautologies to be very informative. There’s a tendency, which we can already see in this thread above, for the discussion to focus on uninteresting questions about what kinds of cases fall under what kinds of hypothetical rules, instead of interesting ones about what should have been cited.
I suspect that these conversations tend to happen at this unhelpfully general level because people are reluctant to call one another out by name. This reluctance is understandable, but in my opinion, it’s interfering quite a bit with clarity. Maybe we should try to relax the taboo on publicly criticizing one another on these grounds. (It’s not like it’s unprofessional or out of bounds to point out that someone has published a fallacious argument; why should it be out of bounds to point out that someone has published a piece that overlooks important literature?) There are various ethical dimensions, of course: we should take care to be criticizing the publication rather than the individual, and we should be gentler to those more junior in the profession than to those more senior. But these norms apply equally to criticizing arguments. I think if we could be having specific conversations about whether and why specific things should have been cited in specific publications, we could have a much more productive discussion about what the norms are and should be.
“(Grad Sockpuppet above suggests that no one suggests that people should cite papers on topic even if they lack interesting or relevant ideas, but I think this is wrong—Marcus Alvin has argued that one should cite everything relevant, whether or not it’s any good.)”
I mean no disrespect, but this is clearly wrong. Notice that I said that that nobody claims we should be citing *irrelevant* ideas (in your characterization, that follows from “lack[ing] interesting or relevant ideas”), and that Marcus *Arvan* argued merely that we should cite everything *relevant*. The latter claim isn’t an instance of the former, it’s the opposite!
We can disagree about the quantifier we want to use to modify “relevant”, but nobody thinks we should cite everything on the topic without considering its relevance. And yet that claim is attributed to proponents of citation reform in almost all of these discussions.
Marcus Arvan has said “My proposal was that there should be a rule to cite every paper in the past 3-5 years on the topic the paper is about.” Previously he praised the sciences, where “an author is literally expected to cite everything that has appeared on the issue in the recent literature,” and said that on PhilPapers “you can literally search for any relevant term and have everything that has been recently published on a topic at your fingertips literally in seconds.” So if there’s a distinction between “relevant” papers and papers on a topic, Arvan is pretty explicitly plumping for citing all papers on the topic, not just the relevant ones. That’s not to say that Krishnamurthy and Wilson are doing the same thing.
(I’m also concerned by Arvan’s claim that this is not too onerous because it only takes a few minutes to do the search on PhilPapers. Surely it takes longer than that to read the papers.)
I think this is a very bad rule. Why should I cite the bad papers from the last 3-5 years over all the really good ones from the millennia before that? Of course, you can say we should cite those as well – but the issue, of course, is that that is *hard* to do, and if we knew how to do it well we wouldn’t have the *need* for an arbitrary ‘cite everything in the last 5 years’ rule.
Sorry for the double post. I should add that I don’t actually disagree with anything else in your post (at least, I think not!), and I actually think that your suggestion is a helpful one that would be beneficial to these discussions (as in the “closing PhD programs” discussion).
It’s not reluctance, Jonathan. It’s *fear*. I have a huge personal database of cases like the slightly less abstract example I offered above. When I first realized that my colleagues were systematically and deliberately not citing my work, I felt as you do – that they should be named and criticized, in the interest not only of getting people to read my work, but also of clearing the air and improving the scholarship of the field. Then I became aware of the many small but moving gestures some were making in order to compensate for not *being able* to discuss my work publicly *without fear of professional retaliation*. And I thought about the very high price many of them had paid in order to achieve their professional status in the first place. Although I completely agree with your argument in the abstract, the reality is that it is just too costly to expect anyone, at any professional level, to put it into practice – at least if they have a family to support and no trust fund. As long as the field is dominated by second-rate bullies who produce mediocre work and protect their outsize stature by plagiarizing, suppressing and punishing others who threaten it, no one will be willing to “call one another out by name.” Would *you* be willing to endanger your own chances of tenure/ promotion/ publication/ review/ job offers/ etc. by doing this? Of course not. Don’t even go there.
Yes, I am sure that fear is often part of the explanation for the reluctance, and that in some cases, the fear is rationally justified. But I don’t think this is something we as a discipline just have to accept. A lot of the reason it can be dangerous to call these cases out is that it’s just not generally done; it’s seen as exceptional, and so threatening. I am suggesting attempting to move towards a culture where critical discussion of philosophers’ citation practices is normalized. As in so many other cases, most of the work needs to be done by more philosophers with more secure career positions.
I think it’s also worth pointing out that there are cases and there are cases; I am sure that you are right that there are some instances of poor citation that are performed in bad faith, but there are also many that are the result of more simply overlooking parts of the literature. I don’t see a lot of value in running around accusing people of acting in bad faith; but calling out someone for failure to discuss something important doesn’t require that. That’s what I meant about avoiding ad hominems above.
Finally, I’m not sure why you assume it obvious that I wouldn’t be willing to take professional risks by calling people out. I don’t want to turn this into a thread about me, but I think you’re making false assumptions about me, and consequently that you’re making false assumptions about at least some members of the field on the whole. Running a search on my name at this blog, for instance, does not in my opinion reveal a particular reluctance to call out inappropriate action from philosophers in positions of considerable power. I absolutely recognize that having the career stability to make such actions feasible is a very significant privilege that many philosophers do not enjoy. But I think it’s just false that there aren’t a lot of us who could do so.
“A lot of the reason it can be dangerous to call these cases out is that it’s just not generally done; it’s seen as exceptional, and so threatening.”
– True. I am offering a hypothesis about why it’s just not generally done.
“I am sure that you are right that there are some instances of poor citation that are performed in bad faith, but there are also many that are the result of more simply overlooking parts of the literature. I don’t see a lot of value in running around accusing people of acting in bad faith; but calling out someone for failure to discuss something important doesn’t require that.”
– True. You might find the Anti-Plagiarism Policy of the Berlin Journal of Philosophy of interest: http://adrianpiper.com/berlinjphil/anti_plagiarism.shtml
“Finally, I’m not sure why you assume it obvious that I wouldn’t be willing to take professional risks by calling people out. I don’t want to turn this into a thread about me, but I think you’re making false assumptions about me, and consequently that you’re making false assumptions about at least some members of the field on the whole. Running a search on my name at this blog, for instance, does not in my opinion reveal a particular reluctance to call out inappropriate action from philosophers in positions of considerable power.”
– I did not run a search on your name before posting my comment and did not direct it at you specifically or personally (the use of “you” as in “How would you like it if someone did that to you?” does not imply a specific personal referent). So I made no assumptions about the professional risks you specifically would be likely or unlikely to take. Nor did I make any assumptions about any specific other “members of the field on the whole,” aside from those with whom I have had the experience I described. I find no foundation in my comments for your claim that my assumptions about those specific individuals are false.
Jonathan, see my response to Will Behan, above, for a couple of cases-in-point.
We suggest that the quickest way to improve citation practices in philosophy as a whole would be for large numbers of individuals to commit, in their capacity as journal or other referees, to rejecting for publication papers or other submissions that fail to cite work that is clearly relevant to the topic at hand, on grounds of failing to meet basic standards of scholarship.
How will this work in practice, though? Suppose author X is unfamiliar with relevant work Z, and so fails to cite it, and X’s paper is sent out to referee Y, who also happens to be unfamiliar with Z; then that paper can get published even though it omits reference to a relevant work. So I fear this is going to largely improve citation practices to the same old stuff that everyone who’s anyone is familiar with, or in some cases to people who have an in with whoever was chosen to referee the paper. It could become a way of perpetuating citational injustice rather than rectifying it.
But, you might ask, shouldn’t the referees be familiar with all the relevant literature? Well, I don’t think that a proposal that further increases the burdens on referees is a good idea, or that makes it more difficult for editors to find a suitable referee. For instance, if a paper applies ideas from one field to another, will the editor have to find referees who are familiar with all the literature in both fields, or will the referees have to familiarize themselves with it? (I recently reviewed a paper that crossed a field I know very well with one I don’t; in order to figure out whether a point in the paper had already been addressed in a certain way, I had to read several papers in the field I was less familiar with. This was much more work than I can do for the average referee report, and I barely scratched the surface of what I’d need to know whether there was any other literature out there that the author should’ve cited.)
I also think that this will push people toward the kind of citation pile that Daniel and Kareem have mentioned. And although Grad Sockpuppet is surely correct to say that no one actually says that you should cite irrelevant work, the effect of an injunction to cite everything relevant would surely be that authors feel the need to cite everything, relevant or not. Operationally, it would be “cite everything that the editors and yet-to-be-determined referees will judge to be relevant,” and it would be too risky to fail to cite something just because you don’t think it’s relevant. If I’ve read paper W and don’t think it contains relevant arguments, but I think someone else might think it does, I’m not going to risk rejection by failing to cite W. Even more so if I haven’t read paper W and I don’t know whether it’s relevant. Putting W into the citation pile without having read it in depth, or without seriously engaging with its argument, would be much less risky.
The elephant in the room here is the bad faith failure to cite, because the author wishes to appear as if he or she came up with a new idea. This is especially prevalent among people awaiting tenure, but it is by no means limited to them. It is rampant in the field.
I don’t see how it is practically possible to cite everything that is relevant. I asked my partner, who works in 17th century English literature, if this norm held in her discipline and she just laughed and said ‘How could it? I have four centuries worth of work to deal with!’ Right – and we have 24 centuries worth!
I’d love to hear an editor’s perspective on this. If I were an editor, I’m not sure I’d be happy at the prospect of every bibliography getting so much longer.
The only discipline to evade the systematic nihilism and literary madness of hypercitation, and here it gets torn down with indignant enthusiasm. Publish or perish started in the 70s; the entirely predictable devastation of literacy, coherence, and merit started soon after, and was quickly followed by disciplines drowning in worthless papers conceived, drooled out, and rushed into print for absolutely no other reason than adding another line to the author’s vitae.
It has been true for several decades that the honourable way to respect a discipline’s literature and its integrity as a field is to refuse to cite intellectual dross and filler. A paper or book that says and contributes nothing should never be cited; period.
Agreed. I found what K&W said about the quality issue rather baffling:
“As with so much else, evaluations of quality are subject to implicit biases that, both on the path to publication and afterwards, unfairly operate against individuals in disadvantaged groups and unfairly operate in favor of individuals in advantaged groups. If an author doesn’t like a paper, then they can downplay it or, better yet, briefly say why they don’t like it.”
Isn’t this just as good an argument for not taking quality into account when refereeing? Our implicit biases are in play there, so we’d be disadvantaging certain groups. Conclusion: accept anything that’s on topic, and if you don’t like a paper you should just publish something yourself in which you say you don’t like it? What is the disanalogy between the cases?
I, for one, will stick to recommending rejection of bad work, and to not citing the stuff that gets through the process somehow.
Some context, if people are interested. Some years ago a dean at my former university did an elaborate publication and citation analysis of all disciplines, based on Web of Science data. The analysis considered only papers by authors at Canadian universities, and only publications and citations in journals covered by the Web of Science (which include the most-cited journals in most disciplines), in the period 1981-2000. One of the figures he calculated was the average number of citations per published article, for each discipline. Here are the figures for humanities disciplines:
Literature 0.33 citations per article
Art and Architecture 0.35
Classics 0.81
Religion and Theology 0.81
Philosophy 1.11
History 1.30
By comparison, the figure for Political Science is 2.57, for Economics 6.24, and for Chemistry 14.97.
So if Philosophy has a practice of low citation, it’s not unusual among the humanities; we’re actually toward the high end of the humanities. That’s not to say our practice is a good or a bad thing; it’s just to say it’s not unusual or unique to us.
Interesting. Are these figures normalised by size of discipline?
If the norms for citation were to move toward what Krishnamurthy and Wilson (and others) recommend, it seems like that ought to impact, at least to some degree, the expectations for scholarly output by junior faculty and, especially, graduate students entering the job market. Obviously, mastering an area of literature takes time–more time than just developing a good philosophical argument that engages with some recent and/or well-known articles. Graduate students–at least from schools outside of the PGR top 5 or so; but perhaps more and more in that group as well–are often told that they ought to have a couple of publications at top journals before hitting the job market. Perhaps I am wrong, but it seems like this sort of early productivity would at least sometimes be more difficult to achieve if students were to adhere to more thorough literature review standards. In other words, current expectations for early publishing may be in tension with more thorough literation review standards. I don’t think this is a reason to reject more thorough standards. But it might be a reason to reconsider what’s appropriate to expect from job market candidates (and perhaps junior faculty).
My own view is that if anyone should be in position to engage in a thorough literature survey of a given topic, it’s a graduate student working on the topic; so I don’t see any reason to reconsider what’s expected from job market candidates here. The same applies to junior faculty. If they don’t have time to do a thorough literature survey on a given topic, then they shouldn’t be writing on that topic. That seems compatible with their writing on topics for which, as per their graduate school preparation, they did have time to do a thorough literature survey on.
Another concern here is access to the literature that authors will be expected to cite. Earlier on this blog Rick Grush wrote:
To take one example, a while ago when I was railing against commercial publishers online, I got an email from an instructor at a community college on a native american tribal reservation, who informed me that not only could his college not afford any journals or books, that the only academic work his students had access to were a handful of books and one journal subscription that he had to pay for out of his own underpaid pocket. And to the point of this discussion, just getting the commercial presses to return the subscription prices to their rates from 10 or 20 years ago doesn’t solve this problem. Faculty at research institutions are blissfully out of touch with the realities of the 99%, and continue to support the current model and commercial presses.
If we reject every paper that does not cite all the most recent publications in the field in, say, Philosophical Studies, then the only people who will be able to publish are those whose institutions can afford subscriptions to Philosophical Studies. Is this desirable? It seems to me that (in line with what Michael Morse said) it’s a strength in philosophy that it can have a low barrier to entry; someone can read an article, formulate a response to it, and have an interesting argument. Someone like Edmund Gettier can revolutionize a field that isn’t even their primary field. In the sciences, history, and linguistics that won’t be as true; you can’t do the research without a lab, or access to archives, or fieldwork. I don’t think it’d be good for philosophy to raise the barrier for entry in this way, especially when it can be an economic barrier for entry.
(It’d be disingenuous if I suggested this was my main issue with the proposed citation reform. Even if everything became open-access tomorrow, I’d still agree with Kareem, David, and Richard that the proposal would tend to lead to pile-citation without much critical engagement with most of the works being cited, and I’d still have the concern I stated above that in practice this would not actually lead to more citation of neglected works, since they would also be neglected by the reviewers. But the question of access is another question to consider.)
In my view, cases of problematic access should be addressed directly, rather than used to downgrade citation standards.
I’ve only run across this reply now,* but for anyone who reads this thread: If the proposal makes it harder for people with problematic access to the sum total of published philosophical work, then saying that that’s a problem that should be addressed directly doesn’t actually do anything to mitigate the adverse impacts of the proposal. We have to deal with the access people have, not the access we wish they had.
*I was going back here in an effort to find that blog that was compiling cases where people had not been given credit for their contributions. In case anyone else comes here looking for the same thing, it’s to be found at The Philosophers’ Cocoon.
There is one aspect of this discussion that I haven’t seen alluded to (except briefly by Michael Morse above). If there is an obligation on people to read (and engage) with all material that is relevant, then any time you publish an article, you impose a burden on others. This means that citation practices shouldn’t be considered in isolation from publication practices. I think many people just publish too much stuff; it’s like monopolizing a conversation (and all the more so if there is a norm that others have to listen). And there is, as many have noted, a strong push on graduate students and junior faculty to publish just for the sake of having publications. If that increases the obligations on others, it is an even more pernicious trend than it might seem.
I also have the sense that K and W are implicitly treating philosophy much like a science and supposing that there is a clear notion of progress at work. I’m not sure that the result of their suggestions wouldn’t be to break the field down into smaller and smaller, manageable ‘chunks’, with people in one area investing so much in getting up to speed that they would be seriously deterred from ever venturing out of their comfort zones. There is something to be said for those of us who are fools enough to rush in where angels fear to tread. Although I do recognize that this can often be an excuse for the exercise of privilege and result in embarrassing and tone-deaf contributions. It’s all very complicated!
That said, I’m not unsympathetic to the tenor of much of K and W’s critique. I certainly agree that there are in-group biases that mean that some people routinely get cited and others do not, even where they clearly should be. I’m just less confident about a general standard for what should be cited (and engaged with).
I agree that changes in citation practices might (should?) impact publication practices. We should all be publishing less. Part of our message is that not just individuals but the discipline as a whole would benefit from people taking the time to do due diligence.
And though I do not see that we are treating (implicitly or otherwise) philosophy as a science, in supposing that there is a clear notion of progress at work, one thing is clear: there are a number of ways in which philosophical progress is, by any reasonable standard, impeded by failures for philosophers to properly survey the literature concerning topics on which they write.