Discipline Size and Progress, in General and in Philosophy
When a field of study becomes large enough, its size “may impede the rise of new ideas,” according to Johan S.G. Chu and James A. Evans, in a new paper, “Slowed canonical progress in large fields of science,” in Proceedings of the National Academy of Sciences.
Why is this? The authors believe that
too many papers published each year in a field can lead to stagnation rather than advance. The deluge of new papers may deprive reviewers and readers the cognitive slack required to fully recognize and understand novel ideas. Competition among many new ideas may prevent the gradual accumulation of focused attention on a promising new idea.
Chu and Evans collected data which they claim support this explanation, finding that
when the number of papers published per year in a scientific field grows large, citations flow disproportionately to already well-cited papers; the list of most-cited papers ossifies; new papers are unlikely to ever become highly cited, and when they do, it is not through a gradual, cumulative process of attention gathering; and newly published papers become unlikely to disrupt existing work. These findings suggest that the progress of large scientific fields may be slowed, trapped in existing canon…
Examining 1.8 billion citations among 90 million papers across 241 subjects, we find a deluge of papers does not lead to turnover of central ideas in a field, but rather to ossification of canon…
Canons crystallize as fields grow large. Churn in the identity and ordering of the most-cited papers decreases with larger field size. The pattern holds consistent when looking at data across all fields and at individual large fields across time… This crystallization of canon happens because the most-cited papers maintain their number of citations year over year when fields are large, while all other papers’ citation counts decay….
The authors believe their findings are relevant to what they take to be the “more-is-better, quantity metric-driven nature of today’s scientific enterprise,” writing that “the scientific enterprise’s focus on quantity may obstruct fundamental progress.”
What about in philosophy? While the authors did not single it out for attention in their article, I got in touch with Professor Chu, who kindly analyzed the data for philosophy available from the Web of Science dataset. Here’s what he had to say:
Philosophy is still a relatively small discipline in number of papers published per year at only(!) ~6,500 papers published in 2014 (listed in Web of Science). Note that the data don’t include books: to the extent the field still prizes (and cites) books over articles, our analysis may not fully reflect trends.
With these caveats, the dynamics we found in the paper still appear in the philosophy data.
I’ve attached a graph showing trends in the Spearman rank correlation of the top-50 most-cited papers from focal year to next (line) and Gini coefficients of citation concentration (circles). The x-axis is logged (base 10) number of papers published in the focal year.
You can clearly see the trends. When the field has more papers published, top-cited articles garner a larger share of the field’s citations and the papers in the top-cited list and their rankings in the list change less over time.
It would be interesting to run this analysis across subfields in philosophy to see whether the larger subfields have less citation “churn” or are slower to produce ideas that “disrupt existing work.”
I also think it is reasonable to wonder whether the relationship between citation churn and progress is consistent across disciplines. It would not surprise me to learn that, moreso in philosophy than most other disciplines, authors cite works with ideas they are criticizing and aiming to replace, and so citation consistency over time does not as strongly correlate with lack of innovation.
If Chu and Evans are correct, they realize that addressing the issue will be challenging:
Reducing quantity may be impossible. Proscribing the number of annual publications, shuttering journals, closing research institutions, and reducing the number of scientists are hard-to-swallow policy prescriptions. Even if a scientist wholeheartedly agreed with the implications of our study, curtailing their output would be impractical given the damage to their career prospects and those of their colleagues and students, for example. Limiting article quantity without altering other incentives risks deterring the publication of novel, important new ideas in favor of low-risk, canon-centric work.
Nonetheless, they do offer some ideas—mainly changes to incentivize the production of novel work that could spur progress:
Some changes in how scholarship is conducted, disseminated, consumed, and rewarded may help accelerate fundamental progress in large fields of science. A clearer hierarchy of journals with the most-prestigious, highly attended outlets devoting pages to less canonically rooted work could foster disruptive scholarship and focus attention on novel ideas. Reward and promotion systems, especially at the most prestigious institutions, that eschew quantity measures and value fewer, deeper, more novel contributions could reduce the deluge of papers competing for a field’s attention while inspiring less canon-centric, more innovative work. A widely adopted measure of novelty vis a vis the canon could provide a helpful guide for evaluations of papers, grant applications, and scholars. Revamped graduate training could push future researchers to better appreciate the uncomfortable novelty of ideas less rooted in established canon. These measures, while not easy to implement across large fields, may help push scholarship off the local attractor of existing canon and toward more novel frontiers.
Discussion welcome.
(via Molly Gardner via Eric Brown)
Related: Citation Patterns Across Journals, Citation Problems in Philosophy—and Some Fixes, Philosophy Citation Practices Revisited, Philosophers Don’t Read and Cite Enough
Relevant past discussions by Adrian Moore (Oxford) (Specialization, Technicality, and the Production of Philosophy | Daily Nous) and Jennifer Whiting (Pitt) (A Modest Proposal: Slow Philosophy (guest post by Jennifer Whiting) | Daily Nous).
Yes! Thank you for mentioning these.
I see that they seem to operationalize field size by “subject” categorized by Web of Science. It seems to me that in a world with a growing population, where poverty is decreasing and thus more people are able to devote their careers to research, we should expect academic fields to continue growing, as they have historically. However, historically, fields have also split. Should we expect fields to revitalize when they split in this way, because each part of the field then is able to focus more on new ideas within their smaller field, rather than ossifying the canon? Or should we expect this kind of fragmentation to constitute its own kind of ossification?
What immediately follows are a few paragraphs from Nicholas Rescher’s book, Nature and Understanding: The Metaphysics and Method of Science (Oxford University Press, 2000). It’s material I’ve quoted on occasion before (for various reasons) over the years in several online discussions and comment threads.
“The ongoing refinement in the division of cognitive labour that an information explosion necessitates issues in a literal disintegration of knowledge. The progress of knowledge is marked by an ever-continuing proliferation of ever more restructured specialties marked by the unavoidable circumstance that any given specialty cell cannot know exactly what is going on even next door—let alone at a significant remove. Understanding matters outside one’s immediate bailiwick is bound to become superficial. At one base one knows the details, nearby one has an understanding of generalities, but at a greater remove one can be no more than an informed amateur. [….]
The emergence of new disciplines, branches and specialties is manifest everywhere [think, for example, of the various neurosciences; of the emergence of ‘law and economics,’ ‘behavioural economics,’ ‘cultural economics,’ ‘institutional economics;’ specialized cognitive sciences; AI paired with this, that, and the other thing; ecological and environmental sciences; specialized fields in ethics: metaethics, applied ethics, bioethics, neuroethics, computer or machine ethics, animal ethics …, and so forth and so on]. And as though to negate this tendency and maintain unity, one finds an ongoing evolution of interdisciplinary syntheses: physical chemistry, astrophysics, biochemistry, [sociobiology], etc. The very attempt to counteract fragmentation produces new fragments. [….] An ever larger number of ever more refined specialties has made it more and more difficulty for experts in a given branch of science to achieve a thorough understanding about what is going on even in the specialty next door. [….]
Yet complexity is not an unqualified negative. It is an unavoidable concomitant of progress. We could not extend our cognitive or our practical grasp of the world without coming to terms with its complexification. Throughout the realm of human artifice—cognitive artifice included—further complexity is part and parcel of extending the frontiers of progress.”
For what it’s worth, and to piggyback off Patrick’s quoting Rescher’s book, the growing field of “ontology” in the information and data sciences (and to some extent, also philosophy) is an attempt to respond to, or even resolve problems concerning the inevitable increase of complexity within and across scientific disciplines (which some scholars casually refer to as The Tower of Babylon Problem). The book “Building Ontologies with Basic Formal Ontology” by Arp, Smith and Spear (2015, MIT Press) outlines methods on how to accomplish better integration of scientific knowledge/information.
I mean, isn’t it a clear demonstration of Polanyi’s analysis (but in the micro view rather than the historical macro one)?
Trends tend to stick for a while, until the “break” happens and the next “revolution” begins (to use Kuhn’s term). It’s merely a natural social flow of ideas.
Not to belittle the data and its analysis, but I feel like we know this for a very long time.
Should we do something about it? Or are we fighting against a natural fact of a society?
Perhaps what should be emphasized upon is the reflectivity of scholars, not by mere criticism of a certain theory but of criticism of its social dogmas. Focusing on the ability to criticize and reflect in academic circulum (and I obviously mean outside of the philosophy department).
This proposal struck me as interesting: “A clearer hierarchy of journals with the most-prestigious, highly attended outlets devoting pages to less canonically rooted work could foster disruptive scholarship and focus attention on novel ideas.”
One might have expected that a clear hierarchy of prestige among journals would be inimical to new entrants. That seems to be the tenor of much extant commentary. (I don’t have the time to cite examples, chapter and verse; I am supposed to be grading exams right now.) And, it is clearly possible for such a hierarchy to encourage ossification if the journal editors, wittingly or not, act in ways that discourage new entrants.
The authors suggest, however, that a clear prestige hierarchy among journals can actually be a tool for including more new ideas, and that the lack of such a prestige hierarchy, however journal editors behave, tends to exclude them in large fields.
WoS is an awful source for philosophy, and not only because it excludes books. It doesn’t index the main journals and it misses very many articles in the journals it does index.
But there’s a more interesting issue with this analysis. I suspect “field” is the wrong unit of analysis. Social psychology is huge and a lot of it is bad. That doesn’t prevent subfields making progress: for the most part, researchers simply ignore papers outside their area. What may look like ossification at the level of abstraction chosen misses lively, illuminating, debate at a finer grain.
The same dynamics probably characterise philosophy. I have no idea what’s happening even in subfields close to mine. People sometimes lament the way in which we fragment into different conversations. Perhaps the lesson here is that fragmentation is needed to prevent ossification.