Hiring Departments Ask Candidates To Anonymize Materials
At least a couple of philosophy departments that are hiring this year have instituted measures to shield the identity of applicants from those reviewing some of their application materials.
Yale University’s advertisement for an open rank, open area position says:
Applicants should submit the following items through Interfolio: Cover Letter, Current Curriculum Vitae, Writing Sample (writing sample should be prepared for blind review), Evidence regarding teaching qualifications, Three Reference Letters (to be submitted by the reference writers).
UCLA’s advertisement for a similarly open position goes even further:
Please submit the following: 1) a letter of application; 2) a CV, a dissertation abstract, and a statement of research interests; 3) a sample of writing in the candidate’s field of specialization; 4) three or more confidential letters of reference; 5) evidence of teaching effectiveness. To facilitate blind review of writing samples et alia, applicants are asked to also submit their dissertation abstracts, research statements, and writing samples without references that would reveal their identities, professional titles, or PhD granting institution.
This seems like a good idea. If you know of other departments doing this, please mention them in the comments. If your department isn’t doing this, why not?
(Thanks to Tom Anderson for bringing these ads to my attention.)
J. Henry Fair, “New Cars Queued for Loading onto Transport Ship” (detail)
This has the potential to backfire in several ways:
1) Overwhelming whoever has to go thru a stack of 400 resumes that now all look alike.
2) Feeling compelled to fly people out who end up being duds because they fooled the initial resume reviewers.
Where you went to school, what you studied, and with whom, all serve as useful signals to prospective employers about what kind of employee they will be getting. Why not just strip out class grades while you are at it ?(some schools have grade inflation, after all, and it isn’t fair to compare a tough grading institution to an easier grading institution)
^ Maybe you should just list the acceptable schools and criteria that must be met to get hired at your school. That way not only our we not wasting your time, but our time as well. I mean, let’s just be transparent about what really going on.
“Overwhelming whoever has to go thru a stack of 400 resumes that now all look alike.”
(a) Hiring departments already do this, and
(b) Does removing schools really make all the cv’s look alike? I suppose so if all you want is someone from Big Name U, but otherwise there are these neat other things on CVs like number and quality of pubs and classes taught and awards earned that make them all interestingly different. But maybe you’ve never made it that far down a CV to realize they contain such info.
“Feeling compelled to fly people out who end up being duds because they fooled the initial resume reviewers.”
(a) This already happens.
(b) To try to cut down on it, we’ve adopted this really neat strategy of doing skype interviews. You should check it out. It’s free and lets you sift out `duds’. (Of course, if by `dud’ you just mean `not from Big Name U’, then yes, this is a problem.
“Why not just strip out class grades while you are at it ?”
We already do that. It’s intensely uncommon to be asked for a transcript for a job application.
Any further objections?
In any case, neither of the quoted advertisements calls for anonymizing the CV, only the writing sample and (for UCLA) dissertation abstract and research statement. These are samples of your work product that do not ordinarily contain your name or institution name in the body. Presumably the idea is to read the papers anonymously once they get to the stage of reading the papers.
(This would be an interesting experiment:
Do whatever it is you do with CVs, recommendations, etc. to winnow the pool to the point where you’re reading the samples.
Supplement the samples you get with a random drawing of samples from the rest of the pool.
Rank them anonymously.
See whether the candidates you initially picked to be the best ones wrote the best samples.)
“Any further objections?”
No. And I would call them concerns more than objections. I am not from a top ten program and likely would have benefitted from such anonymizing of materials. But that doesn’t mean such a policy can’t have negative consequences. Whether it does is an open question.
The spirit of these efforts is admirable. But I have a hard time imagining that anonymizing application materials will prevent members of the search committees at Yale, UCLA, etc. from recognizing a significant number of applicants, unless those search committees are comprised of members who aren’t actively engaged in the broader philosophical community. And if that’s the case that also seems to raise another set of issues.
Sally Haslanger came to give a talk at Yale about minorities in the profession and there was a fruitful conversation after the talk about the benefits and risks of anonymized materials for job candidates (as well as for graduate student applicants). I don’t know if this had any causal impact on Yale’s decision, but I’m glad to see them trying it out.
We (Elon University) started using an annonymized process for our last hire (for a post-doc) two years ago and will use one for our upcoming (TT) hire. We are a teaching school, and a department looking first and foremost for exceptional and innovative pedagogues, so we asked for annonymized teaching statements and c.v.’s. and make our first cuts from those. We then read the other non-annonymized materials to get down to our Skype call group. And we let applicants know that this was our process.
We found that applicants did not always anonymize there materials, and one member of the committee (what I came to call our CRO–Chief Redacting Officer) made sure they we anonymous before they went to the rest of the committee; our CRO then was not part of the first cut (and thus was only in conversation once the materials were no longer anonymous).
My hope is that as more departments ask for anonymized materials, more applicants will be in the habit of anonymizing their packets (or parts, thereof) and we will need to do less work.
It’s occurred to me that this could be done more easily by crowdsourcing. Most people apply to a bunch of jobs, so if everyone anonymizes, the same person’s application will be anonymized many times. On the other hand, if we shared anonymized versions of things, we could all do a lot less anonymizing while still anonymizing everything.
Here’s the idea: suppose there were a central place where job-seekers could put all their materials. And suppose this were linked to PhilJobs, so that applicants could select jobs to apply for and materials to send to them. Committees, in turn, could be `charged’ a small amount of anonymizing as a price to pay for accessing said material. But it would, in principle, be drastically less anonymizing than if the committee anonymized all the material sent directly to them, because some of that material would be anonymized by other people also using the system.
Such a setup could probably even be configured in such a way that people were never anonymizing material for applicants to any job whose search committee they were on.
But I don’t know how to do these things.
Why is it a good idea to have it not be known what your PhD granting institution is (and as a result, I assume, who you worked with)? Are we to assume now that it’s not the case that some people and some institutions train people better (at least in certain subjects)? That would be a bizarre coincidence, unless the training really didn’t matter, which would be pretty depressing.
Those who are better trained will presumably exhibit that fact at some point in the process, regardless of the institution they attended–provided they’re given the chance, of course. The idea is not to *assume* that someone from a particular institution is better than someone who isn’t, thus giving more people a chance to shine.
And again, in the ads under discussion the part that’s being anonymized is the writing sample (and, for one of them, the research statement and dissertation abstract). When you’re reading the writing sample you should be able to tell how well trained the applicant is, without knowing where their PhD comes from.
Ah, it wasn’t clear to me from how it was worded that the PhD institution would remain on some of the material. (I do kind of wonder what the point is, then, but that’s another issue of course.) And of course you’d expect the training to shine through in the writing sample *to some extent*, but surely we wouldn’t expect that to give us the entirety of the information – otherwise, why not ask only for that? Why ask for letters, e.g. – after all, if they really are good, as so-and-so says, surely it will be made apparent in their writing sample.
Of course you don’t want to make an *indefeasible* assumption about who is better than whom based on where they got their PhD. But it doesn’t strike me as a bad extra bit of evidence to throw into the mix. Not every university is on a par. It’s ridiculous to think otherwise.
If you want to balance the quality of the writing sample as one among many factors in hiring, I’d think the way to go is to evaluate the sample independently, and then combine that evaluation with the other factors, rather than allowing other factors such as the degree-granting institution to influence my reading of the sample.
In any case, I don’t see why it’s relevant if not every university is on a par. If university A is better than university B, and candidate X from university A produces work of equal quality to candidate Y from university B, how does that tell in favor of X? Surely the way the superiority of A will manifest itself is that candidates from that university will overall write better philosophy–in fact, that seems constitutive of superiority in the relevant sense.
As for why not ask for the writing sample only, a lot of it is that in many cases it’s prohibitive to read every writing sample. Four hundred 25-page papers = 10,000 pages.
“Surely the way the superiority of A will manifest itself is that candidates from that university will overall write better philosophy”
Right. And if we had time to look at the entirety of work people produced, spend time talking to them about each paper, etc., we could just look at the quality of work, and ignore institution, letters, etc. But of course, we don’t have time to do that. So we look to other factors. One of those is, I think understandably, where people were trained. And other things being equal, if two candidates have produced *one paper* that is equally good but one is from a much better institution, I think that’s a pro tanto reason to think that the one from the better institution has been better trained and is going to produce better work on the whole. That is defeasible, and I’d want to see what their letter writers had to say, etc. But it seems like valuable data to me.
It’s also the case – and it’s silly to pretend this isn’t true – that there are advisors who are known for being . . . let’s put it charitably . . . extremely hands-on helpful when it comes to their students’ writing samples. Shouldn’t that affect our judgment of the writing sample? Isn’t that a useful thing to know?
Matt Weiner:
If, as you admit, we don’t have time to read all the writing samples, then other non anonymized factors will be used to eliminate candidates before the writing sample stage. At that point enough will be known about the remaining candidates’ work to make educated guesses about identity likely. Furthermore candidate Y may already have been eliminated at that point.
If the writing sample of candidate Y was really good though, and if it were available to the readers at the first cut stage, it might actually get a quick look. That might keep Y in the running.
In this way anonymizing only the writing samples might hurt, not help, candidate Y.
One drawback I see in this is that it adds one more time consuming and in some cases fairly tedious task to the application process. Applying for jobs is very time consuming and stressful. The more that different schools tweak their standards or demands to their own preferences – justified or not – the more this increases. Even if each school has a very good reason for its own idiosyncratic requirement, and even if each one is not a huge burden on its own, it adds up over time, especially as many applicants are applying to many jobs. (Perhaps the worse aspect of this is the demand for specific, individualized, but nonetheless easily fudged cover-letters.) Perhaps, all things considered, this demand on job seekers is justified, but it shouldn’t be assumed that it’s not without some real burden to job seekers as well. I wish I felt sure that this consideration got significant thought among those putting in the requirement, but seeing the sorts of individualized but really meaningless demands that are very common for applications, I’m not so sure I believe it was thought through carefully.
Anonymizing the writing sample, dissertation abstract, and research statement isn’t very hard or time-consuming. Frankly, it’s hard for me to see why they wouldn’t already be anonymized. What need do I have to refer to myself and my PhD-granting institution all the time?
You’re right that requiring idiosyncratic documents (like teaching statements dealing with particular texts, or “what you can do for x/what x can do for you” documents) is a fairly significant burden, but I just can’t see this falling under that.
“Anonymizing the writing sample, dissertation abstract, and research statement isn’t very hard or time-consuming.
That depends – the writing samples I’d use are developments of some of my previously published work, and discuss that work to a fair degree. This is so with my research statement, too – it doesn’t really make sense except insofar as it talks about how I am extending and going beyond my previously published work. Now, if you don’t have a good amount of previously published work, it’s a lot easier to anonymize, but that’s not so of at least a significant number of people. So, even if it’s not difficult or time consuming in your case, it can be for others. Again, maybe the benefits of this sort of thing outweigh the costs, but I’m skeptical that they have been clearly thought through, especially when considered with all of the other time consuming aspects of applying, and how idiosyncratic demands from different schools add up extremely quickly.
At University of Oklahoma, we’ve anonymized writing samples in the last three searches we’ve run, although we have heroic staff who do the anonymizing for us and for the candidates. Details about this and other strategies we’ve been using to avoid a host of bias issues were published in an APA Newsletter a few years ago, here: http://c.ymcdn.com/sites/www.apaonline.org/resource/collection/D03EBDAB-82D7-4B28-B897-C050FDC1ACB4/FeminismV13n2.pdf We’ve been very pleased by using anonymized samples and, more generally, by how our searches using bias-reducing strategies have worked.
The letters I write tend to say a lot of specific things about the writings sample and dissertation project of the applicant. Like, super specific and detailed things that would easily allow any reader to link up my letter with the writing sample. (I take this to be part of writing a good letter, actually. I assume that’s true of a bunch of other people writing letters.) So I’m not sure how anonymizing the writing samples gets at the intended goal.
Hiring in academia, and certainly in philosophy, presents an acute problem. The body of evidence that is available (the dossier) has a dubious signal/noise ratio. Properly assessing that evidence requires (a) distinguishing signal from noise, (b) paying close attention to the signal, and (c) trying to avoid being influenced by the noise. Most people in the profession agree, I think, that properties of the applicant like gender, race, age, etc. are pure noise. Alas, the implicit bias literature tells us that we have a hard time not being affected by it, so best to get rid of it if possible–hence the drive to anonymize what can be anonymized fairly easily.
Reasonable people can disagree, of course, about what constitutes signal and what constitutes noise. For instance, some people on this thread think that pedigree (where one got one’s PhD, for example) is noise, whereas others think it is signal. It’s helpful to keep in mind that elements of the dossier need not be pure noise or pure signal. You might think that pedigree is, by itself, a pretty noisy signal. The point here is that the disagreement need not be so stark–pedigree tells us nothing vs. pedigree is crucial.
My very unpopular view about the hiring process is that there is an awful lot of false precision usually involved. The fact is that what we are looking for is very hard to measure. *Nothing* in the dossier is a very clear signal. How to run a search in light of this fact (if it is one) is not altogether clear. But it is not as though there were some highly effective, precise method that we would be disrupting if we made some serious changes in hiring procedures.
I was on a search last year that used anonymous writing samples (demanded by the administration). It’s pointless, and arguably counterproductive, window dressing in the name of diversity. The identity of the author was always obvious from the rest of the materials. Moreover, the best route to diversity is of course to scour the applicant pool for diverse candidates, those who might otherwise fall through the very big cracks left by successful anonymization. It struck the committee and the some thoughtful people in the otherwise clueles administration, as lip service paid to diversity promoted by administrators who actually had little idea how hiring works in the details and therefore as an annoyance to work around in the effort actually to promote diversity among our pool of finalists.