A New Model for Conducting Job Searches in Philosophy?
“We typically get around 300 applicants. In our first pass through those applications, we read one and only one thing by every single candidate: the Abstract of their job market paper.”
That’s from a tweet by Ethan Bueno de Mesquita, professor of public policy at the University of Chicago. It was brought to my attention by philosopher Nils-Hennes Stear (Southampton) in a post at his blog about the academic job market in philosophy.
As Brian Weatherson (Michigan) remarked in a follow-up tweet, “Philosophy searches could do this.”
Hiring departments could ask candidates to initially just send in an anonymized abstract of their writing sample, say, up to 400 words (or perhaps the whole writing sample, anonymized, along with the anonymized abstract in a separate file), their c.v.’s, and a cover letter, with the first round of cuts being based solely on the abstract. The c.v.’s of those who made the cuts could be consulted to see if various other criteria are met. The departments then ask for the rest of the information from those who’ve made the cut.
This may not work for all types of academic philosophy jobs—perhaps just assistant professorships or post-docs for which research is among the most important considerations. But it seems like it could help counter the influence of some biases and focus attention on the candidates’ ideas, at least at the initial stage. It could also cut down on candidates’ application expenses, as they would not need to pay to have transcripts or confidential letters of recommendation sent out for all of the jobs they apply to.
Your thoughts—including your estimates as to the percentage of applicants who would make it through an abstract-first screening process, on the assumption that its use is well-publicized—are welcome.
(photo of untitled sculpture by Tara Donovan)
I’ve applied for 50+ jobs this cycle and haven’t yet paid for official transcripts. (Most places accept unofficial transcripts for the first round.) However, I do pay $1-3 per application to submit rec letters via Interfolio’s forwarding service, with the exception of Interfolio-hosted position which are entirely free.
In departments like mine, very few of us are experts in the AOS we are hiring in. We’re not fit to judge, not really, whether someone does good scholarship or great scholarship in their area. I’m mystified why reading an abstract in lieu of a C.V. is an improvement. At least with a C.V. I can see whether or not someone has teaching experience in the courses we’ll need them to teach and enough of a track record of publishing to earn tenure at my institution. C.V.s have the further advantage of letting me know whether someone really does have an AOS in the category they claim to have an AOS in (does their scholarship clearly match the AOS area?). A single abstract wouldn’t tell me that at all.
Furthermore, the job market being what it is, it would be in everyone’s interest to submit an abstract on a topic even if it wasn’t their AOS (I’m not an Early Modern person but could – and have – whipped up a Hume paper under the right conditions). This sounds like it would end up taking more time overal for no net benefit in the search.
I could actually see something analogous for teaching positions: somebody writes up a few course descriptions, each of which is the length of an abstract. This would basically be equivalent to answering the standard interview question, “How would you teach…?”
Whether the proposal has merit, I would urge those adopting it (or relevantly similar approaches) to ask applicants for only as much material as would be initially reviewed. Materials reviewed after the applicant field has been (significantly) cut could be requested later, saving tens or hundreds of people as many hours of pointless labour.
(For more on this, see my post linked to above).
To amplify what I take to be the point in what Caligula’s Goat says above: Please, no. No, no, no, no, no.
I can see a lot of problems with the suggestion, but I think we have to measure this and any other suggestions to improve the philosophy hiring market fairly. The question isn’t are they a good way to do it in the abstract but are they an improvement over the current system and better than any serious rivals you might put forward? This suggestion seems a huge improvement over the current system in three ways: 1. It’s transparent about how applicants will be evaluated. 2. It does something to combat the rampant prestige bias that goes into hiring decisions now. and 3. It will save applicants a bunch of money they’d have to otherwise waste on paying Interfolio’s exploitative fees for forwarding letters of recommendation. I for one think that the ball’s in the court of those who criticize it to say how they’d improve hiring. Yes it’s a pretty crude and arbitrary way to cut candidates but any way I can imagine of getting the pool down to manageable numbers in the current climate will have a lot of arbitrariness to it.
Now I’ll say that this wouldn’t work at all for teaching focused institutions like mine. But I’m wondering if teaching focused institutions shouldn’t make their first cut in much the same way but substituting teaching statements for abstracts. You can’t tell if someone is a good teacher from a teaching statement, but you can at least weed out the bad ones I think. It’s pretty apparent if someone has never actually thought about what they’re trying to do with teaching or how to do it from a teaching statement.
I could see doing this with full writing samples. But just the abstract doesn’t seem like enough. Perhaps in a field that relies heavily on empirical data, the abstract tells you enough about the methodologies being used, and then the fact that they claim to have gotten a significant result using this methodology tells you that the full thing is going to be good. But it seems to me that in philosophy it’s often quite difficult to judge quality of a paper from the abstract.
I do think that we need to change our systems so that we can cut back on the huge number of person-hours involved in many of these processes. In particular, things like grant review, and perhaps graduate admissions, should involve randomization to get from the short list to the set that are accepted, rather than hashing it out in hourslong committee meetings with multiple people and encouraging applicants to tailor their work to the effectively random whims of that committee in selecting from among the very best. But I have less of an idea of how to cut down the hours involved in hiring just one person.
I second Kenny’s opening point here. Philosophy abstracts routinely advertise goods the paper fails to deliver. The hard part is usually making the argument or constructing the theory, which you can’t assess from the abstract.
You could still cut some papers as Brian noted on twitter—those that wouldn’t meet the bar even if they delivered what they advertise. I reckon the majority of applications (at least) would still make first cut, at least in the kind of searches we do in my department. And I suspect the survivors would include pretty much all the people who would have made the long-short list anyway, by the old-fashioned method. But I’m not confident enough to say it’s not worth a try!
“Philosophy abstracts routinely advertise goods the paper fails to deliver.” — One way to fix this is via the process the Pacific APA has taken: ask for 750 word abstracts. In the few I’ve seen, one doesn’t learn much from the paper that wasn’t already in the abstract. (Of course it’s true that APA papers are a lot shorter than published papers.)
My paper demonstrates that the collection of underpants is a significant and underutilized source of profits.
???
I’m pretty sure this refers to the classic underpants gnomes episode on South Park, where the gnomes had a business model in 3 phases:
phase 1, steal underpants
phase 2, ….silence…
phase 3, profit!
I take it as an obscure (to many) way to make fun of papers that don’t deliver what they promise, because they lack a complete argument.
It’s a great idea to minimize the material that a candidate has to submit in the first round of applications. It’s at this stage that most of the unfairness and arbitrariness occurs, and much of it occurs because of the glut of information. If you get between 200 and 300 applicants for a job, and each application is about 50 pages long, most candidates will be rejected for biased reasons. If applications can be reduced to between 8-10 pages, there’s a much better chance of fairness. How to do this? Abstracts probably won’t work in philosophy, for reasons given by Kenny Easwaran, and letters are worse than useless at this stage of the game. Maybe cv plus 3 page summary of a writing sample plus one page summaries of research and teaching?
I am in awe of the ability of philosophers to reflect on the awfulness of the job market and then come up with ways to make it even worse.
An abstract is an advert and summary for what a paper is going to do. It usually doesn’t contain very much information about how successfully it does it.
Optimal search strategies depend on underlying parameters. Harris School of Public Policy is resource rich. I’m guessing they make a few hires a year, and it wouldn’t be the end of the world if an assistant professor didn’t get tenure (in terms of them getting a replacement hire). This allows them to take more risks in the search, which allows them to make decisions based on promise (e.g., a compelling research question) rather than results. If (i) I got one hire every few years, and if (ii) in the event that this hire got denied tenure I was unsure whether the line would be renewed, then I would be more conservative and rely more on a proven record (e.g., publications). I think (i) and (ii) characterizes many philosophy departments, but do not characterize public policy/econ departments like Harris.
One of the problems with streamlining the process is that different documents are important to different members of search committees. This is something that I would imagine that almost anyone who has served on a committee knows. Some of my colleagues find letters of recommendation incredibly valuable – I on the other don’t (way too much grade inflation and variation). Some of my colleagues find the cover letter incredibly valuable – I on the other hand don’t (if I’m interested in your teaching I’ll check out your teaching dossier, if I’m interested in your research I’ll look over your CV or your actual writing sample). Some of my colleagues find the writing sample incredibly valuable – I do too but don’t usually read it for the first round of cuts (simply because there are so many applications).
So the long story here is that I think streamlining is really difficult. When you combine the documents that HR requires all searches to include (graduate transcripts – even if unofficial to begin with) with the documents that various members of the department find significant (the other traditional documents) I’m not sure there’s much that can be removed. I also am not comfortable telling my colleagues that they’re just wrong about the value of cover letters or teaching philosophies or sample syllabi. Epistemic humility won’t allow me to do that. Instead, what I’ve come to learn is to trust the process: that a group of well-meaning people, each of whom has overlapping but distinct areas of concern with an application, can come together to make choices that are good for the department.
That isn’t to say that burdens on applicants shouldn’t be addressed or cared about. To the degree possible, it shouldn’t cost applicants *anything* to apply for a job. The only documents that currently seem to cost applicants anything are letters of recommendation and this is only because letter writers prefer using a service like Interfolio than sending out several dozen versions of the same letter (multiplied by several candidates they’re writing letters for). I’m open to suggestion here but don’t think that streamlined application materials are a good starting off point.
All applicants should be assigned the same essay question to be answered in, say, 2500 words. Presumably hiring committees will pose questions where they’re competent to judge answers.
This proposal and process would violate my university’s equality opportunity and access policy. I am sure it likely violates the one at the University of Chicago as well.
Someone who was rejected from one of these searches should formally complain: [email protected]
https://voices.uchicago.edu/equity/affirmative-action/
Could you please explain how this violates your (or the University of Chicago’s) university equality opportunity and access policy?
Looking at U Chicago’s statement, it has an interesting tension that I think is quite common in these sorts of statements, and that is brought out by this case.
The statement says this: “the University of Chicago considers students, employees, applicants for admission or employment, and those seeking access to University programs on the basis of individual merit. The University does not discriminate on the basis of race, color, religion, sex, sexual orientation, gender identity, national or ethnic origin, age, status as an individual with a disability, protected veteran status, genetic information, or other protected classes under the law”. A genuinely anonymized abstract-assessment process looks obviously compliant with this statement; indeed, any objection to anonymizing on access grounds would prima facie seem to be blocked by this statement, since its intended purpose would have to be to allow for that discrimination.
But U Chicago also says: “The Office for Affirmative Action is a proactive compliance unit within Equal Opportunity Programs that supports the University’s commitment that its community of faculty and staff is composed of a diverse and rich mix of individuals who contribute to the University.”
And that does seem to require information about candidates’ background to implement.
You can square this, and see how anonymized statements could violate equal opportunity, if you thought that
(a) some candidates’ backgrounds have disadvantaged them so that their anonymized writing is less good than others’ even though their ‘individual merit’ (whatever that means) is equal, so that you have to compensate;
(b) that sort of compensation is appropriate in equal-opportunity policy, either because you think that a candidate’s longer-term academic ability won’t be affected by background, or because you think it will but that’s a price worth paying for other goals.
(c) The ‘diverse and rich mix of individuals’ sought by the Office for Affirmative Action is also the mix (or at least, is compatible with the mix) that you’d obtain by an admissions process that compensates for unfair background advantages but is otherwise entirely merit-based.
I’m not convinced of the plausibility of these, but they’re at least defensible, and I think they’re the (tacit or explicit) basis for a lot of institutions’ policy here.
Sure Josh: I’ll make it simple without telling you where I work. Our job ads have required qualifications. You must evaluate the candidates based on those required qualifications as a first screening. So imagine that a required qualification is Ph.D. at the time of application (for ease of example). Submitted writing sample abstracts is irrelevant to whether the applicant satisfy the required qualification of Ph.D. by time of application. Our EOA office would come down on us like a ton of bricks if we tried to do it as suggested above. The same would be true if you required say, 3 years of full-time teaching experience, or competency in a particularly area of philosophy. Those are the areas that a candidate must be judged upon, not some writing sample abstract alone.
I won’t go into any more detail than that, but in my view, their office of equal opportunity should investigate and likely train them on how to conduce searches in a proper manner.
That’s interesting. What’s the rationale for requiring that screening to happen at the first step, rather than just making sure it happens at some point in the process?
Reducing the costs (in the broadest sense) to apply this much will make the number of applications skyrocket, as Caligula’s Goat suggested. That seems relevant!
Some things seem uncontroversial, like deferring letters till later, and (official) transcripts till wayyyyy later, like maybe only to be returned on your first day of work (i.e., for 1 person, not 300).
But judging on an abstract seems weird, not just for the reasons given above, but also insofar as it would suppress someone’s publication record. Would I rather have one good abstract or an ok one that comes with 10 publications? Not sure how that’d be prestige bias. Obviously that abstract thing would get hacked/hoaxed pretty quickly, too. Pretty sure I could write a strong abstract on say, Hume, without having any chance of writing the full paper.
But the other thing that never gets talked about here, is why 300? Specifically, why do so many people apply to jobs there’s 0% chance they’ll get? Like if we have a broadly analytic, non-historical department, but we get 100 applications from continental or historical folks. (Yes, this happens every search–probably, candidly, because those programs have such a hard time placing candidates.)
Of course you could say the job ad should clear that up. But it really doesn’t, or can’t. If you say “broadly analytic”–which people do say–then you’ve still got those 100 continental applications figuring they’re the exception. If you say your interests are “contemporary”, then 100 historians tell you how their work has contemporary upshots. So it’s just not clear what ad–that’d get approved by deans, HR, etc.–could be the right filter.
My point is simply that figuring out how to humanely treat 300 applications has already lost the dialectic: there shouldn’t *be* 300 applications in the first place. Not because there should be more jobs in philosophy, fewer unemployed philosophers, and all that–maybe true, just irrelevant for present purposes. Ads should be narrowed to the extent possible, and, beyond that, applicants shouldn’t be spamming every possible application just because they can. I get why they’d want to, but it’s in nobody’s interest (including theirs) to put forth non-competitive applications.
As someone who’s been on both sides of the hiring / applying process over the last few years, I think the problem is not so much departments asking for a full writing sample rather than an abstract (since, assuming everyone is going to have a writing sample, it’s no extra effort to send the whole thing) but departments having a whole range of subtly different criteria for the same basic materials. For example, an applicant can find themselves having to put together four different research statements or teaching portfolios depending on precisely what people say they want in them and how long they’re supposed to be. One simple change that would make a big difference would simply be to have a shared set of expectations for what a research statement, teaching portfolio etc. is supposed to look like, and require only materials in that standard format for the purpose of making the first cut. That’s still plenty of material to work with, but providing it wouldn’t place undue burdens on applicants. Once the first cut is made, departments could then ask for more specific, tailored materials without wasting the time of applicants who weren’t going to make it to the final stages.
Yes. Seconded.
But since standardizing the process across departments is a big task that is unlikely to ever happen, I’ll propose something smaller:
Please stop requiring us to fill out electronic forms.
Even with auto complete, it’s an enormous burden to create an account just so we can enter our names, institutions, physical addresses, email addresses, phone numbers, websites, letter-writer names, institutions, addresses, and so on, over and over and over again. All of that information is on our CV.
I think “People Admin” is the name of the most commonly used service that requires the completion of such forms. Please, for crying out loud, ask HR to stop requiring your department to use that awful service.
Mark and Ed get it.
Isn’t it funny how many people fear that papers might not live up to the promises of abstracts, while they hardly ever doubt that candidates will live up to the promises gathered from their CVs?
I think part of the worry is that abstract screening can be gamed in a way CV screening can’t. There’s a lot more room to make big promises in abstracts. I don’t think it’s a big problem, though. At worst you’ll screen those false positives at the next stage.
A “job search” is what the candidate conducts, not the department.. Perhaps “faculty search” is a better term for the post’s title? Please revise, Justin, or we’ll have to refer the matter to the EEOC (Excellence in Editing Online Council).
My impression is that candidates in the current market are primarily evaluated by the prestige of the university and department from which they received their doctorates. This is a valid and efficient method of selection because acceptance by such institutions requires very impres-sive test scores and outstanding performance from equally demanding undergraduate programs, often drawn from same pool of highly selected institutions. It provides at a glance a mass of highly relevant, valid and reliable information.
Much the same is true of merely noting the number of publications in prestige journals. There is little to be gained by actually reading, even that were possible, a great many papers, which must inevitably be very much alike.
Evaluation of additional evidence serves to narrow choices within this class of candidates by assessing the same qualifications in much less valid and reliable ways.
What would provide different and very valuable information would be to ask each candidate to summarize any work he or she has done or plans to do which he or she considers original, important, and likely to be right, and to explain briefly why he or she thinks that is so. Similarly, candidates might be presented with a difficult problem not normally met with in standard graduate curricula and asked to sketch very briefly how he or she would go about solving it.
Also, this approach might limit the possibility, latent in any select[on process in which applicants greatly out number jobs, of hiring committees finding a candidate whose views, interest, values, goals, and temperament are identical with their own.