In some domains, “overall quality depends on how good the worst stuff is,” while in others, “overall quality depends on how good the best stuff is, and the bad stuff barely matters.”
An example of a weak-link problem is food safety, he says:
You don’t want to eat anything that will kill you. That’s why it makes sense for the Food and Drug Administration to inspect processing plants, to set standards, and to ban dangerous foods. The upside is that, for example, any frozen asparagus you buy can only have “10% by count of spears or pieces infested with 6 or more attached asparagus beetle eggs and/or sacs.” The downside is that you don’t get to eat the supposedly delicious casu marzu, a Sardinian cheese with live maggots inside it.
It would be a big mistake for the FDA to instead focus on making the safest foods safer, or to throw the gates wide open so that we have a marketplace filled with a mix of extremely dangerous and extremely safe foods. In a weak-link problem like this, the right move is to minimize the number of asparagus beetle egg sacs.
One of his examples of a strong-link problem is music:
You listen to the stuff you like the most and ignore the rest. When your favorite band releases a new album, you go “yippee!” When a band you’ve never heard of and wouldn’t like anyway releases a new album, you go…nothing at all, you don’t even know it’s happened…
Because music is a strong-link problem, it would be a big mistake to have an FDA for music.
Mastroianni thinks it’s important to know what type of problem one’s facing, because they’re solved differently:
Science, he says is a strong-link problem:
In the long run, the best stuff is basically all that matters, and the bad stuff doesn’t matter at all. The history of science is littered with the skulls of dead theories. No more phlogiston nor phlegm, no more luminiferous ether, no more geocentrism, no more measuring someone’s character by the bumps on their head, no more barnacles magically turning into geese, no more invisible rays shooting out of people’s eyes…
Our current scientific beliefs are not a random mix of the dumbest and smartest ideas from all of human history, and that’s because the smarter ideas stuck around while the dumber ones kind of went nowhere, on average—the hallmark of a strong-link problem.
Yet we tend to treat it like a weak-link problem:
Peer reviewing publications and grant proposals, for example, is a massive weak-link intervention. We spend ~15,000 collective years of effort every year trying to prevent bad research from being published. We force scientists to spend huge chunks of time filling out grant applications—most of which will be unsuccessful—because we want to make sure we aren’t wasting our money.
These policies, like all forms of gatekeeping, are potentially terrific solutions for weak-link problems because they can stamp out the worst research. But they’re terrible solutions for strong-link problems because they can stamp out the best research, too. Reviewers are less likely to greenlight papers and grants if they’re novel, risky, or interdisciplinary. When you’re trying to solve a strong-link problem, this is like swallowing a big lump of kryptonite.
In short, our options for approaching problems often confront us with what we can call the tradeoff between preventing the worst and allowing the best.
Mastroianni doesn’t bring up philosophy in his piece, but we can ask about how this distinction applies to it. We can ask, for instance:
(1) Is philosophy is a weak- or strong-link problem? (Related: maybe philosophy has various tasks, some of which pose weak-link problems and some of which pose strong-link ones?)
There’s some reason to think we treat philosophy as a weak-link problem. After all, philosophy uses peer-review and other forms of gatekeeping. So we should ask:
(2) If we treat philosophy as a weak-link problem, why? What’s so bad about bad philosophy?
(Note: this question is not asking what makes a bad piece of philosophy a bad piece of philosophy; rather, it’s asking: what is or would be bad about having an abundance of bad philosophy around?)
We should also ask about the extent to which philosophical practices actually manifest the tradeoff. After all, the tradeoff is an empirical possibility, not a conceptual necessity. So we can also ask:
(3) To what extent do institutions and practices meant to filter out bad philosophy tend to also filter out or discourage good philosophy?
(4) To what extent do institutions and practices meant to filter out bad philosophy tend to promote or draw attention to good philosophy?
(5) Are there alternative institutions and practices for philosophy that could minimize the tradeoff?
Traditional peer review at academic journals serves a gatekeeping role, determining whether a piece is publishable or not; this decision comes after the piece is nearly complete. This type of peer review practice often proves hostile to new ideas, unproven authors, and unfamiliar audiences…
CCR nurtures new ideas by supporting pieces through their development, creating supportive experiences for authors and audiences. The goal of this review process is to both prepare pieces for publication and improve them in those preparations. CCR is structured to encourage peer engagement rooted in trust and a shared commitment to improving the work through candid and collegial feedback.
You can learn more about the CCR process here. It is just one of many kinds of possible approaches to minimizing the tradeoff, and it would be good to hear about others.
It would be good because there’s some value, at least in principle, to sorting.
Consider an art museum that will accept any artwork, displaying them in a layout that reflects the order in which they were received, and that keeps expanding, building new galleries as its existing ones fill up. There might be a lot of good art here at MegaMuseum, including some good art that might not have ever found a way to be in the public’s eye but for this museum. Still, MegaMuseum is not my model of an ideal museum. After all, life is short, and so is patience. How much time and effort do I want to put into sorting through the works there to find good pieces? I’d rather go to MiniMuseum, which is a much smaller, more carefully-curated museum. It’s more valuable, even if it doesn’t have as much good stuff, and even if not all of the stuff it does have is to my liking.
Of course, if we had to choose between MegaMuseum and MiniMuseum, then it seems that MegaMuseum is the better option, despite the downsides. We might see art, as Mastroianni does, as a “strong-link” problem.
But even better than MegaMuseum would be an enormous array of MiniMuseums, each different in some of the myriad ways museums might differ (eras, media, themes, purposes, standards, intended audiences, etc.). We’d have a better sense of where we ought to go, given our aims, and how to make sense of what we’re presented with—basically how to get more value from our museum-going experiences.
We might think the qualitative and thematic sorting work journals do is valuable in a way that’s analogous to having many and varied art museums. We might also think that such a system has downsides. For example, attention is limited, and so sorting risks directing attention in unequal or elitist ways. Resources, including human labor (referees) are limited, and sorting risks directing them in unequal or elitist ways, too. And so on. Is there a way to capture the good here, and minimize the bad?
(via Marginal Revolution)