The replication crisis in psychology and other fields, in which researchers have found it difficult or impossible to replicate the results of many earlier experiments (see the Reproducibility Project) is now being addressed by those working in experimental philosophy (x-phi), a subfield of philosophy that borrows surveying and experimental methods from psychology, whose work may suffer from similar problems.
The X-Phi Replicability Project enlisted 20 teams across 8 countries—over 40 researchers—to conduct replications of a “representative sample” of 40 x-phi studies, and has recently released its results. They found that x-phi studies “successfully replicated about 70% of the time.”
By way of comparison, the Reproducibility Project was able to replicate findings in only around 35% of a representative sample of psychology studies.
What explains the relatively high replication rate? The authors consider a number of explanations:
- the effect sizes in x-phi, especially early on, were large, and it has been found that effect sizes are a good predictor of a study’s replicability
- because x-phi studies are less costly to run and re-run, there is less of a downside to getting results that are not interesting enough to publish, and so there is less motivation to engage in “questionable research practices”
- the effects studied in x-phi are generally “less subtle” than those studied in psychology and more likely to be affected by factors under the control of the researchers
- the academic culture of philosophy encourages researchers to be “more sensitive to certain methodological questions, such as what counts as strong evidence for a given claim,” or have “a greater tolerance for negative or null results.” More generally, for a few reasons, philosophers may be less susceptible than psychologists to the pressure of “publish or perish” when it comes to empirical studies.
You can read more about the results here.
(via Florian Cova)