I claimed in my first post that uncovering hidden biases (i.e. “values”) was worthwhile. So I agree it is better to know what values people have when they begin theorizing. But it does not follow that building an ontology on top of values is a sound move.

Sure. But what if it is the only move available to us?

Because if values are real then the values of those they disagree with are also real. Unless the social ontologists with to take only their own values as real–in that case the method will generate a secular theology that is indefensible from anyone’s view but their own.

I am really not one to go to bat for moral realism, but this seems fallacious. A typical moral realist would say that someone can be mistaken about value in the same way that they can be mistaken about empirical matters.

It seems that you think that in order to get to the truth by value-laden methods, we must first settle (or stipulate) what the right values are. But this does not follow. (So moral realism is not required here.)

Compare: knowledge is “concept-laden” in that all our knowledge is found and expressed using concepts (duh!). But it does not follow that we first must settle what the right concepts are. We work with the concepts we got, we revise as we go along. I don’t see why we can’t do the same for value.

The only position available to social ontologists based on the above is conceding that they may be mistaken about what the right values are at any given time, and therefore conceding they might be mistaken about the ontology at any given time. This does not make for the kind of footing needed to convince others–whether philosophers or the public.

Again, sure. But if value-ladenness cannot be avoided, then this is everybody’s problem.

I suppose to convince others, one could insist that one’s methods are value-free or that one’s values are the right ones. But that would really be “secular theology”.

And there are known problems with reflective equilibrium approaches, namely that each goal post (either the ‘values’ goal post or the ontology goal post) can always be moved to guarantee the success of the resulting theory. This creates a situation where there is nothing at stake in the theory.

Again: everybody’s problem. The way out is that some of the values fix what is at stake.

We must be talking past each other. What I gave you is a bog-standard fallibilism.

It’s just that if we are fallibilists about theory and value, we can acknowledge that values determine what “falsification” means. If a value/theory combination gets falsified, you can reject part of the values or you can reject a part of the theory.

And it seems to me that you are denying this because you deny that the categories of empirical social science (“gender pay gap”, for example) are value-laden. And many feminist economists have argued that they are; for example, Marilyn Waring with respect to the construction of the UN’s National Accounts.
To take the Goldin example, the paper on childbirth as a driver of the gender pay gap is this one: https://pubs.aeaweb.org/doi/pdfplus/10.1257/app.2.3.228. 

The paper never explicitly says this but the background default ‘partnership unit’ is the ‘nuclear family’ (for example, the summary statistics on ‘career and family’ in the appendix; or the ‘married’ status, which is a binary dummy variable in Table 7). So the story is, as the title explicitly says, about differences between people with different self-identified gender among a fraction of the white-collar US class with a background expectation of a nuclear family, where both the parent(s) and the child are in the same household/country. (Close to 80% of the respondents are US citizens, per the Appendix.) Under a different definition of ‘partnership unit’, the picture could be very different. For example, remittances and extended family (not covered in the paper) are huge factors especially among migrant workers, with parents and children often residing in different countries. 

Now given the narrow scope of the paper, here’s how you presented the results:

“To pick one of the examples from Asta’s paper, suppose we’re interested–for value-laden reasons, perhaps–in explaining gender pay gaps. So you might ask which factors play the biggest roles in explaining gender pay gaps. Claudia Goldin won the economics nobel prize for this sort of work. Here’s a very rough summary of some of her conclusions. The gap is largely driven by childbirth; pre-child, gaps are small; post-child, they widen dramatically. Why is that? Many high-earning sectors (finance, law, consulting, corporate management, some medical specialties) heavily reward long, continuous, inflexible hours. Because families need someone to provide temporal flexibility, couples tend to “specialize,” and women disproportionately provide that flexibility. The result is a persistent pay gap among otherwise similar workers.”

(This basically mirrors the presentation in the Nobel scientific background paper: “To dig deeper into the role of having children, Bertrand et al. (2010) examined within-individual changes in employment and earnings in the years after the birth of a first child. They estimated this event-study-like design separately for men and women to assess the differential gender responses to the birth of a child – the parenthood effect. Female employment, earnings, and hours immediately decrease after childbirth, and continue to do so in the following years. But male labor market outcomes do not deteriorate – if anything, earnings increase.”)

To draw such conclusions on the basis of the narrow study is to already assume what counts as explanatory relevant childbirth, in which kinds of partnerships, and so on. Not to mention that the paper is not really about ‘childbirth’, but about ‘caring obligations due to the *presence* of a child within the same household unit’ (whether through biological birth or adoption). Otherwise, the paper should have included surrogacy, where childbirth also happens, and how the latter affects wages within such ‘partnership units’. 

I take you to believe that empirical social science works with measures of operationalisable categories. These however, presuppose a prior construction of what this category entails: what or whom it includes, what or whom it excludes. These are all value choices. If you say, well, we want to measure some effect of a category premised on a conception of what a ‘standard family unit’ is, and these other types of ‘family units’ aren’t operationalisable, yes, but the prioritisation of ‘operationalisability’ and ‘measurement’ in what *empirical* social science means is also a value choice. (As any practitioner of qualitative research would, I imagine, argue.)

This helps in understanding what you are claiming, thanks. So the claim is, specifically with respect to statistical estimation in economics, to stick to the example, that the choice of categories, the operationalisation, the hypothesis to be investigated, the choice of statistical versus another kind of empirical method: these are all value-laden. But the choice of a specific type of statistical method itself doesn’t rely on values? 

I don’t see how it doesn’t. The applicability of any statistical method relies on probabilistic assumptions that rule out conditions under which the method doesn’t perform well: for instance, OLS is BLUE under specific conditions, which, if violated, would render the results of OLS meaningless. So the economist must choose which conditions to assume away as irrelevant to a specific context. The choices during categorisation and operationalisation stage regiment the ‘unruliness’ of social experience into what is standard and what not; the choices during statistical analysis regiment the remaining ‘unruliness’ in even such standardised phenomena into outliers and how much they misbehave.

By the way, the surrogacy point was not about finding the effect among households that are ‘surrogacy importers’, but among households that are ‘surrogacy exporters’, to use economicy jargon, since that’s where the childbirth happens. But this was a quibble with the framing of the paper. 

For this specific case, let’s take an example of such a paper: https://www.aeaweb.org/articles?id=10.1257/aer.20141467. For the outcome of a (first) IVF treatment to be a good instrument for the (causal) effect of fertility on earnings, it has to be uncorrelated with earnings. The paper does a few checks by testing for correlation between IVF outcomes and ‘health’, where ‘health’ is accounted for by “additional controls for the number of eggs collected and transferred, diagnoses, causes of infertility, type of IVF treatment, and clinic indicators, which could all reflect potential health risks at the time of treatment”. This is a very medicalised understanding of ‘health’ and there is one, what seems to me, obvious missing factor not just here but in the entire paper, including the Appendix: ethnicity or race, which is, in fact, correlated with IVF success rates. (So the sensitivity checks would have perhaps turned out differently had the authors controlled for that. And IVF outcomes would not have been a good instrument then.) There are at least two value judgements here: an underlying medical versus social conception of ‘health’, which relies on taking biological features as natural rather than social phenomena, something that is only possible if the social determinants of health remain invisible because they work in favour of the group doing the studying. Second, treating IVF outcomes as a ‘good instrument’, which might be compatible with “colour-blind” types of feminism but not with more critical variants of it. 

This doesn’t mean that the studies that do examine the relation between ethnicity/race and IVF outcomes need not have problems of their own: https://www.sciencedirect.com/science/article/pii/S0277953621003403.

“That is, a “colour-blind” feminist, in this telling, isn’t particularly concerned with differences between women of different races, and is happy to just ask questions about women as a group. This seems almost directly analogous to the person who is happy to just look at GDP, rather than distributional national accounts.”

The point was a bit more subtle than this, in a way that’s different from the GDP/distribution point. The paper cannot be asking questions about “women as a group” because, among women *as a group*, IVF outcomes are *not* a good instrument for the effect of fertility on earnings. They might be within racial groups. So there is a difference between what the authors think and say they are doing (effect among all women) versus what they might actually be doing, depending on what some missing relevant summary statistics of their sample are. The latter is (on a charitable interpretation) unbeknownst to the authors. Why is it unbeknownst to them? As I said, lack of awareness is the most charitable interpretation: a lack of awareness about the relevance of race to any social phenomenon that is baked into “colour-blind feminism”. This is different from a person who is presented with a choice between analysing growth versus distribution and going for the former, at least as you’ve been describing this choice. 

“I had a harder time understanding how the medical vs social conception of health distinction was meant to drive different decisions about study design, and how to understand that as one or the other kind of value-ladenness.”

The distinction literally makes or breaks IVF outcomes as a good instrument here. IVF outcomes cannot be correlated with earnings and so with labour history. If IVF outcomes are correlated with the ‘health’ of a subject, then that’s bad news because ‘health’ is correlated with labour history and earnings. So how exactly ‘health’ is understood and operationalised, what factors are measured in testing for this correlation, would either entail that IVF is a good instrument or not. The paper uses a medicalised category of ‘health’ where ‘health’ means the factors I enumerated above. If ‘health’ is not just a matter of counting eggs, diagnoses, and so on, but a product of one’s life and generational history, as the social conception has it, then it would have to be operationalised differently: and importantly, in that case IVF outcomes and ‘health’ would be correlated, making the former a poor instrument. In fact, a social conception of ‘health’ would invalidate many seemingly medical phenomena as good instruments for something like earnings, making the whole IV approach unsuitable. To take ‘health’ as a ‘purely’ medical category is to describe medical outcomes as they appear; to take ‘health’ as a social category is to take medical outcomes as a product of a social history. This, I think, relates to Ásta’s point about what a theory is for: the former project illuminates, but not fully, patterns in the ideology; the latter project interrogates the ideology itself. I say ‘not fully’ because, in this example, by not even entertaining the idea of ‘health’ as a social category, the paper doesn’t seem to be aware that it’s just describing ideological patterns. 

As for the Humean account, yes, we disagree there. But as this account, and the whole fact/value distinction that underlies expected utility models, strikes me as very simple, let me ask you: How has the whole Putnam debate on the fact/value distinction affected your thinking with respect to this distinction, if at all? My impression was that these debates had largely dispensed with this simplified view of the distinction as untenable; and that people had moved on to pragmatist or what have you conceptions. 

“I suspect you’ll probably reject some or all of that picture. Is that right?”

Yes, I would reject that this is *not* value-laden:

“questions about how confident to be in some given proposition, given some body of evidence–or concretely, what conclusions to draw after the above questions have been decided and the analysis has been performed–will not be [value-laden].”

How confident you are in some proposition and what conclusions you draw are not innocent moves. I mean, isn’t that a fundamental lesson of Humean causation? That the latter is a relation of the mind, so the inference itself is not something you find in the evidence, it’s something you add to the evidence. 

More generally, I agree with the fact/value distinction debates that the distinction is conceptually untenable but I also think it’s very often pragmatically unfruitful, to use your words. Similarly for the positive/normative distinction in economics. So that, consistent with some other comments here, I don’t think there’re any decisions in social science that are not value-laden; one might conduct some research assuming certain values, for pragmatic reasons, but it’s important to at least be aware of the values that one is holding fixed and that are driving what one makes of the research.

Thanks for the chapter, Daniel, that’s helpful. Though still, I find, quite simplistic. Including the use of ‘fact’ and ‘value’ (or ‘criteria’ and ‘values’ if you like) in the economics example you give. In fact, I’d say that re-framing the distinction in this way is what amounts to pulling off a “verbal trick”.

Let’s take your example and, again, let’s look at the actual details, including those around how competitiveness is measured. Take two economists, a socialist S and a libertarian L, who disagree on whether one should distribute away from the rich to help the poor: this, you say, is disagreement over values. Take two socialists, M and N, who agree on the distributive question, etc., and only disagree on one thing: how competitive the relevant industry is, which matters because it’s a parameter in the estimation of the minimum-wage effect. This, you say, is disagreement over facts.

What facts? The degree of competitiveness of the relevant industry. What does this mean and how is it measured? Because, if it’s ‘just facts’ that can be captured with ‘epistemic criteria’, surely M and N can settle their disagreement by stepping out into the (data) world. Except it isn’t so easy.

There is, famously, no single way of conceptualising or measuring an industry’s degree of competitiveness; not only that, but different concepts and measures rely on different underlying understandings of what ‘markets’ are, etc. Take two popular measures, the Herfindahl–Hirschman Index (HHI) and the Lerner Index (LI). HHI is the sum of the squared market shares of all firms in an industry, LI is the ratio between a firm’s markup and its price. Suppose M and N disagree over whether the industry’s degree of competitiveness should be measured by HHI or LI. This is not mere disagreement over ‘facts’. HHI is a measure of concentration, while LI is a measure of (profit-based) market power. An industry could have a maximum HHI and a minimum LI (one firm selling at marginal cost, making no profits), or the other way around (many firms with zero market share charging exorbitant prices). So what does lack of competitiveness mean? Dependency (the result of concentration) or exploitation (the result of profit-based market power)? I don’t think calling these questions ‘descriptive’ is helpful.

Suppose M and N agree on using HHI and so that the real threat to competitiveness is dependency, rather than exploitation. M and N just disagree about how to measure things like ‘market share’, etc. Again, we already went over this, such measures are not innocent: M and N need to agree on what ‘ownership’ means, what constitutes a ‘firm’ (versus other forms of enterprises), etc.

I concede that at that last stage, M and N have agreed on much more values than at the start, or in comparison with S and L. But what appears descriptive is, once you get into the details, just that: an appearance. And it is hiding this appearance that I find pernicious in the claim to the fact/value, or criteria/value, distinction: by calling one set of things real, substantive values and another set of things criteria, etc. one (who?) is not just introducing a hierarchy of values but also hiding this. ‘Simplicity’ is such an obvious example: what about those people whose experiences aren’t captured by ‘simplicity’? The point is that all these things require justification and by re-labelling certain ‘values’ as ‘criteria’, etc., the researcher implicitly decides what things require justification and what not. The alternative is not relativism; jumping to relativism is also a way of evading epistemic responsibility here. The alternative is to acknowledge that justification is owed; depending on the context, of course, the requirement for justification might be stronger or weaker, but I don’t see how it’s helpful to pretend it’s not there.