University of Pennsylvania Philosophy Stops Requiring the GRE
The Department of Philosophy at the University of Pennsylvania has decided to, at least temporarily, stop requiring applicants to its PhD program to submit GRE scores, and not take them into account even if included in applications.
Associate Professor of Philosophy Quayshawn Spencer sent along the following announcement:
The University of Pennsylvania’s Department of Philosophy will, on an experimental basis, not require PhD program applicants to submit GRE scores. For clarification, this means that we will not look at them even if an applicant submits them. The faculty reached this decision unanimously after discussing the matter.
Key factors in this decision were, first, that the GRE can be financially burdensome for low-income applicants ($205 for the general test in the USA, only 50% of which is waivable by the ETS, plus the non-waivable $27 per school to send your scores to after 4 schools) and offer unfair advantages to wealthy applicants (e.g. ETS offers a score review service for an extra fee and Kaplan offers test prep services for a fee that isn’t entirely waivable). Second, GRE scores do not, in general, accurately predict academic performance in graduate school (e.g. Q,V, & AGRE scores explain only 4.4-7.8% of graduate GPA variance according to replicated studies). Third, significant gaps in GRE performances by women and underrepresented racial and ethnic minorities made it especially difficult for them to be accepted, even though their scores sometimes dramatically underpredicted their academic performances in our program. Fourth, in our judgment, nothing of significant epistemic value was gained by our use of the GRE that we couldn’t figure out from looking at transcripts, writing samples, etc. So, women, minorities, and low-income applicants, apply to Penn philosophy! We will not discriminate against you based on an outdated, expensive, biased, and predictively invalid test. Our deadline for applications is December 15th.
If your program is taking similar steps, please let us know in the comments.
This is remarkable, and hopefully the first step to a long wave of universities going this way. There is a whole slew of inequities and blatant unfairness that is hidden within the GRE.
We have never required, nor do we look at, GRE scores for applicants at The University of British Columbia.
Michigan hasn’t been asking for GRE scores for a few years now.
I’m preparing to apply to programs now to start next year, and the GRE has proved a real distraction and an annoying cost for what’s clearly little reward (particularly since the GRE counts for little even at schools that require it). I’d much rather have more time to focus on my writing sample, personal statement, and honors thesis. I’m admittedly very fortunate to be able to afford the cost of the GRE, but as noted above, others are not…
Kudos to Penn, and hopefully we’ll see more schools follow soon!
UW-Madison also does not require or consider GRE scores.
Glad to see a department acknowledge (and put into practice) what we all know to be true. The GRE is a scourge and the sooner it is ignored and done away with the better. All it measures is how well you do on it, and not much else.
Oops, I meant this to be a general comment — somehow I mistakenly typed a reply to you and did not mean to. Sorry about that!
Leiter posted something similar yesterday. Interestingly, there are two sentences in the quoted text here that do not appear in the quoted text at Leiter’s blog: “So, women, minorities, and low-income applicants, apply to Penn philosophy! We will not discriminate against you based on an outdated, expensive, biased, and predictively invalid test.”
This is no comment on the broader issue, but I don’t find the “GREs don’t explain much of graduate success” point to be very persuasive. It’s looking at variation among people who have already been admitted to grad school. But GRE, insofar as it’s useful at all, is going to be useful in filtering out comparatively weak candidates who won’t go to grad school at all, not in making fine distinctions among people who will get in somewhere.
“But GRE, insofar as it’s useful at all, is going to be useful in filtering out comparatively weak candidates who won’t go to grad school at all, not in making fine distinctions among people who will get in somewhere.”
first, in light of the evidence that the GRE is biased against women, minoritized people, persons with low-income, i’m worried—how should i read ‘…filtering out comparatively weak candidates…’?
second, written: ‘…who won’t go to grad school at all…’; in the context, i think it means: ‘…who won’t get into grad school at all…’. that together with the filtering-out phrase suggests that the GRE, insofar as it’s useful at all, is going to be useful in filtering out people who won’t get into grad school at all. but, that’s false. my GRE scores were horrible. i eventually was admitted to a program and now have my PhD.
”first, in light of the evidence that the GRE is biased against women, minoritized people, persons with low-income, i’m worried—how should i read ‘…filtering out comparatively weak candidates…’?”
You could read it as ”women, minoritized people, persons with low income…are comparatively weak candidates”. Because it’s very much possible that a better socieconomical standing quite objectively leads to academic success compared to people with lower socioeconomical standing, and that GRE is biased against people who would, quite objectively, perform relatively weak in the academia.
Life isn’t fair.
The “GREs don’t explain much of graduate success” point was supposed to be this: committees use GREs in admissions because committees believe them to be useful predictors of whether students will succeed in a program, but since GREs don’t do well enough at predicting this, we do not wish to use them in this way. You’re not taking issue with this, right?
You simply think that there are other reasons for using GREs. The reason you mention is that GREs will still “be useful in filtering out comparatively weak candidates who won’t go to grad school at all.” This reason is not very weighty. Some of those students who otherwise were filtered out by GREs might succeed in grad school–and perhaps be more successful than those who are not filtered out. There have been people like this–who, despite being filtered out, did succeed, and did better than others who were not filtered out.
“This reason is not very weighty. Some of those students who otherwise were filtered out by GREs might succeed in grad school–and perhaps be more successful than those who are not filtered out. There have been people like this–who, despite being filtered out, did succeed, and did better than others who were not filtered out.”
Isn’t this true about literally anything we could use to determine who gets into graduate school? Some student with a good writing sample, good GPA, etc., could end up doing worse than a student with a bad writing sample, bad GPA, etc. Admission committees aren’t omniscient; they have to use the best tools at their disposal. If the GRE, all things considered, is good at determining who is *likely* to succeed in graduate school (and I don’t know whether it is or isn’t), then it’s a good tool.
The point was that the fact that the GRE filters out some weak candidates is a weak reason for Penn to keep it in light of other facts: among others, (1) it does not do *good enough a job*, (2) it provides an unfair advantage to wealthy students, and (3) it provides an obstacle to students from groups historically underrepresented. If something similar is true about other tools to filter out students, that is irrelevant to my point. Penn found weighty reasons against using GREs, and without showing that the deliberative process they used was faulty, there is no force in merely pointing out that using the GREs does *some* thing valuable.
I’m curious about the claim that “GRE scores do not, in general, accurately predict academic performance in graduate school.” That may well be the case, but is graduate GPA really the most appropriate metric? Graduate GPAs are (notoriously) inflated, and if arbitrariness is an issue in undergraduate grading, it’s surely worse at the graduate level. More useful metrics might include program completion rates, publications, placement, etc. I recognize these metrics have problems of their own, but surely they’re better measures of *actual* grad student success than the GPA. (What matters more to hiring committees: publications or graduate GPAs?)
Much is made of the burden that the GRE places on low-income students. I’m sympathetic to the concern; paying for the GRE would have been significantly burdensome for me were it not for an unexpected, earmarked gift from my grandmother. (Seriously.) At the same time, coming from an undistinguished college where I lacked access to famous letter-writers, I hoped the GRE would be an equalizer for me. (That hope was probably misguided, but that’s besides the point.) All that’s to say: it’s not at all clear to me that, on balance, the GRE confers significant advantages to high-income students, given that (i) high-income students are more likely than low-income students to attend SLACs or well-regarded research schools, which already serve as proxies for excellence; (ii) because high-income students are more likely to attend selective schools, they’re more likely to have famous letter-writers; and (iii) if you’re applying to 10 or more schools, the cost of the GRE (including the cost of additional test scores) pales in comparison to the cost of applications. (If someone has empirical evidence that disconfirms (i) or (ii), please correct the record.) Regarding (iii), I realize that application fees might be out of the department’s hands and that waivers are sometimes a possibility. Still, it’s worth keeping the cost of the GRE in perspective.
By my lights, the real action is in points (3) and (4) of Penn’s statement. I can’t speak to the empirical evidence for (3), though I wouldn’t be surprised if the evidence is significant. It’s also possible that the writing sample says a lot more about the skills of the applicant than a timed exam—though a non-technical writing sample might not fully convey the applicant’s logical acumen, whereas a Q-score *might*.
You ask: “is graduate GPA really the most appropriate metric?” No one claimed that it was, and it is compatible with the announcement that graduate GPA is not the most appropriate metric. The announcement suggests that grad GPA is one among many metrics for grad performance and success. Additionally, what matters to hiring committees is performance and success, not GPA and not publications by themselves…
You say: “All that’s to say: it is not at all clear to me that, on balance, the GRE confers significant advantages to high income students…” For the purposes of deciding not to use GREs in grad admissions, it doesn’t matter whether the GRE confers *significant* advantages to high-income students. All that matters is that the GRE is financially burdensome to low-income students, and one unfair advantage for wealthy students is removed by not using GREs. This is the first key factor for Penn’s decision, as the announcement basically states. Nothing you say undermines using this as a reason for not using GREs.
Nothing you have said has dealt a significant blow to either the first or the second key factors for Penn’s decision.
To your first point: True, the statement doesn’t say: “GPA is the most appropriate metric.” But as a matter of fact, that’s the metric they’re using empirically defend (1). Perhaps they considered other metrics when they made their decision, but the GPA metric is all we’ve been given. If they *don’t* actually think GPA should weigh heavily in assessments of student success–and, quite plausibly, GPA doesn’t and shouldn’t weigh heavily–then we shouldn’t accept (1) on the basis of the evidence they’ve provided. That was my point. (Also, I wasn’t ruling out the importance of other metrics. I’m saying that GPA *alone* is too crude a measure of student performance and that publication record–despite not capturing the *entirety* of student success–is likely a better metric.)
To your second point: I shouldn’t have used the term “significantly.” I’ll stand by the unqualified version of the statement (until I’m proven wrong!) If low-income students are disadvantaged by their lack of academic pedigree–which, plausibly, many of them are–then performing well on the GRE might help them compete with more privileged peers. To be sure, taking the GRE is costly–low-income students might need to work overtime to afford it, and they may have less free time to study than their peers–but it might at least give particularly scrappy students a *chance* to compete with wealthy Ivy Leaguers. So the benefits for low-income students (e.g., increased competitiveness for strong graduate programs) might outweigh the serious costs (e.g., work hours, money). Of course, if the GRE is systematically biased against women and minority students (in which case my appeal to “scrappiness” is unwarranted), or if graduate programs in fact put very little stock in the GRE, then perhaps the costs really do outweigh the benefits. That’s why I suggested the real action is in (3) and (4).
Your first reply includes a false claim, that grad GPA is *the* metric that Penn is using to defend their second key factor in their decision. What is true is that is the only one they list. But I’m sure you’ll agree with this. You’re trying to make a subtler point. That’s fine.
Your second reply does nothing to address the point of my response: the fact that one barrier is removed by not using GREs is not a bad reason to stop using GREs, unless there is evidence for thinking that using GREs is not a genuine barrier. You’ve provided no such evidence.
First paragraph: I’m not sure I understand the response. I explicitly acknowledged the possibility that other metrics were considered in making the decision. I said that the GPA metric *is all we’ve been given*, i.e., their only defense of (2) invokes the GPA metric. I’m guess I’m not seeing the problem.
Second paragraph: I’m not sure what evidence is being requested; I’m just raising concerns about the defenses proffered in the statement. (Though I understand that it’s a statement, not a treatise.) I haven’t offered evidence directly in favor of keeping GREs because I’m not sure whether I’m in favor of keeping GREs. I don’t possess enough evidence to rule confidently one way or the other. I’ve raised concerns about the GPA metric and highlighted *potential* equalizing opportunities afforded by the GRE.
Thanks for the discussion. I’m bowing out for now.
I have a question for people in departments that *do* use the GRE.
Am I right in thinking that you only use the GRE scores to weed out very bad candidates, so that, while it’s important for applicants not to have very low scores, otherwise the GRE scores make little difference?
A poll to gauge how often GREs are used this way might be instructive, if someone has the platform to run one.
(I haven’t seen GREs used this way myself, and I don’t think I’ve heard others discuss it until now. But my admissions experience is very narrow.)
Just one small data point: I had somewhat poor grades (in non-philosophy classes) as an undergraduate for a variety of good and bad reasons. I think I had somewhat strong letters from somewhat known people. I also had notably strong GRE scores. My impression is that the scores helped significantly when applying to grad school, since without them the strong letters and the poor grades would have sort of cancelled each other out. I think I was a relatively strong grad student and, in any case, I now have tenure. In any case, I totally agree that it would be bizarre to base one’s admission decisions on the GRE along, or even to weigh GRE scores very heavily. But I’m not yet convinced that there’s anything wrong with using it as an additional bit of evidence: after all, the OP claims that higher GRE scores are, in fact, weak evidence for higher grades in graduate school. I think a lot hangs on whether minority GRE scores merely sometimes under predict grad success, or consistently under predict grad success.
re: Tom. Bet there’s some variability. I’ve heard at least one faculty member at a Leiterific program say that applicants’ GRE scores are the only things that should be looked at. The thought being, I think, that letters and GPA aren’t standardized across schools and that writing samples might have benefitted from third party assistance.
Another reason why departments might use the GRE is because, if there are college-wide graduate fellowships that departments compete for, the easiest way for philosophy nominees against nominees from other departments to stand out is to have really high GREs. This has, for better or worse, been one of the reasons why CU-Boulder uses GREs. But it is obviously only one of many factors in our admissions process.
oops, something was garbled — meant to say: “the easiest way for philosophy nominees to stand out against nominees from other departments”
“Another reason why departments might use the GRE is because, if there are college-wide graduate fellowships that departments compete for, the easiest way for philosophy nominees against nominees from other departments to stand out is to have really high GREs”
In my opinion, this is the only reason to include GRE scores as part of the admissions process. However, the underlying problems with the GRE remain. The question I have is what exactly does the GRE measure besides one’s ability to take the GRE? For if the GRE does not measure one’s aptitude for graduate work in their field, it is reasonable to ignore it. But, if the GRE does not measure one’s aptitude, why use it to determine who wins fellowships? If one accepts the GRE is not a reliable measure of aptitude, what justified use (in admissions, or fellowship competitions) does it have at all?
This letter does not demonstrate the claim that Penn will be less biased against women, minorities, and low-income applicants, as it claims. To demonstrate that claim, Penn would need to show that their decision process is less biased based off the evidence it uses than other schools will be based off evidence using the GRE.
Penn cites evidence that the GRE is biased against women, minorities, and low-income students. Let’s assume that evidence is correct. Now Penn needs to show that the combined evidence it does use (e.g., writing sample, transcripts, recommendations) is not *more biased* than the GRE. In particular, I would be worried that other sources of evidence are closely related to “soft skills” that have been shown to be areas where high-income college students vastly outperform their peers.
Notice that the proper goal here is to have a fair final procedure, not to eliminate biased evidence. Doing otherwise would treat absence of evidence (of bias in other evidence) as evidence of absence (of bias). Grades and recommendations are likely to be very biased, even if it is more difficult to gather evidence on that subject. If I had to guess, I might say that writing samples will be even more biased because of the difficulty in learning to write in a “philosophical style” at that stage in life. I do not, however, know how these biases will fall, other than likely disadvantaging low-income students.
To your first point: the announcement does not purport to demonstrate the claim that Penn will not be biased. It does claim that Penn will not discriminate based on GREs, and it has offered some reasons for thinking that the claim is true.
To your second point: If the announcement were trying to demonstrate the claim, perhaps it would need to show that what it does use in admissions is not more biased than GREs. But the announcement does not try to demonstrate the claim.
To your third point: the proper goal is to have a fair procedure for admissions. But eliminating biased factors can achieve the goal, but it is not to treat absence of some bias as evidence of absence of all bias. Rather, it is to treat absence of some bias as evidence of absence of some bias. And this is completely fine.
To your first response: I agree, Penn did not claim that it will not be biased. That would be a strawman, and I would not waste my time arguing against it. I said that they did not show that they will be *less biased.* I took that claim from their fervent invitation at the end of the announcement.
To your second response: I did not ask them to show that they were not biased, I asked them to show that they were comparatively less biased.
To the third response: This is the meat of the objection. Eliminating biased factors *does not necessarily* make a procedure as a whole less biased. This is shown mathematically by the Theory of the Second Best (https://en.wikipedia.org/wiki/Theory_of_the_second_best), but we can break it down intuitively. Suppose I am trying to shoot an arrow at a target. I face two “biases”: the wind is blowing 6 inches to the left and the bow is naturally biased to miss 2 inches right. Now suppose that someone shows me that the bow is biased to the right, and I adjust without also accounting for the wind. I will now miss by 6 inches left, whereas before I missed 4 inches left.
I apologize, I typed that example too quickly, I will now miss 8 inches left, whereas before I missed 4 inches left, since I will aim 2 inches left to account for the bias. (2 inches of aim left plus 6 inches of wind to the left).
To your first reply: my point stands. It does not purport to *demonstrate* any claim. It makes the claim that Penn won’t discriminate using GREs, and offers reasons in support.
To your second reply: No, you did not ask, you did something much stronger. You claimed “Penn needs to show” what it uses in admissions is less biased than the GRE. But it is not at all clear why Penn needs to show this unless they are attempting to show that their admissions are less biased now than before. But Penn is not attempting to show this. My second point stands.
To your third reply: your reply makes a different point from the one it defends. My response to the point is compatible with your defense of the point. So your reply is irrelevant.
1. Yes, I am questioning the validity and appropriateness of the reasons.
2. Not sure the disagreement here. I don’t disagree that I am asking them to show something they don’t show. Penn is advocating departments disregard admittedly probative information. If evidence of bias is to serve as a reason in that argument, I think it must make the comparative claim.
3. I would need more detail to respond.
Jen is right that the announcement doesn’t claim that Penn will be less biased than other institutions. But IGS is right that our goal should surely be to have as little bias in our admissions requirements as possible, and the fact (?) that the GRE is biased doesn’t entail that removing it will make the overall procedure less biased. I also agree with IGS that there’s at least a plausible story according to which the removal fo GRE scores will make the overall process more biased. But of course that’s an empirical question.
Penn’s decision (inclusive of the decision of other universities listed and not listed who also do not use the GRE in admissions decisions) as well as the recent decision by Ryerson University to not use student evaluations in tenure track promotion or hiring, gives me some hope for the future of our field.
The Penn letter doesn’t do enough to acknowledge that evidence-based admissions is hard and there are no easy answers. Nathan Kuncel et al’s large meta-analyses in 2001 and 2007 say the GRE is about as predictively valid as GPA – both exhibit small to moderate correlations with metrics of grad school performance after statistically correcting for range restriction (the issue David Wallace notes above). College GPA has weak predictive validity too, and disparities in GPA between whites and URM’s are substantial — plus getting a college GPA obviously costs a fortune, especially from a brand name. The predictive validity of letters is known to be terrible, and they tend to reinforce status cliques, not break them down.
I definitely respect Penn’s decision to ditch the GRE but I am surprised at the unanimity of their department as the trade-offs here are complex — reasonable people can (and really should) disagree.
Don’t know if they actually act this way, but GRE scores could be a decent equalizer when it comes to applicants from lesser-known schools with lesser-known professors writing recommendations for them. GRE scores are just one data point, but why consider it completely valueless as a metric (compared with, say, GPA, or even undergraduate institution)?
Anecdotally I have heard that Harvard has also gotten rid of it.
At my university (Marquette) , we are also considering dropping it.
The point about performance in grad school and GRE is misleading because it’s conditioning on a collider (Berkson’s paradox):
https://thehardestscience.com/2014/08/04/the-selection-distortion-effect-how-selection-changes-correlations-in-surprising-ways/
GRE doesn’t explain much of the variance *among admitted students*, but that doesn’t say much about applicants. In fact, even though there is an inverse correlation between undergrad GPA and GRE among admitted students, the correlation among applicants is positive. How can that be? If you get into grad school with a low GRE, you probably have a high GPA (and vice versa). Something similar probably holds for grad GPA and GRE.
Plus, it’s not obvious that focusing on other selection criteria (“transcripts, writing samples, etc.”) would be better for underrepresented groups. What if it’s even worse? Does Penn have any data on that?
If it turns out that other selection criteria would not be better for underrepresented groups, it does not make the GRE any better. The GRE is, for all of the reasons already mentioned, not a good selection criterion. So, Penn is getting rid of it as an admissions criterion. If focusing on other criteria ends up hurting underrepresented groups more, then it just means both those criterion and the GRE are bad for underrepresented groups. That Thing X might be worse than bad thing Y does not change the fact that thing Y is bad.
Upenn is taking a step in the right direction.
To note: I am not a faculty member who is screening potential graduate students. However, I have heard from faculty members that some depts use GRE as a first -step elimination procedure. Which means that only the very top scores make it into the actual selection process. Of course, I have no idea whether this is a widespread practice or whether it is still a practice at all.
From a personal point of view: I came to the US as an international (first generation) graduate student eight years ago. My financial situation was not so good compared to my colleagues in my program. I took the GRE nine years ago. I remember it well how much of a financial toll and what a stress and anxiety ridden test it was. On the day of the exam the system was down. So, although we were scheduled to take the exam at 9am, we ended up taking it at 4pm in the afternoon. I was wiped out by then and whatever results I got, they did not speak in favor of my intelligence or aptitudes. Not to mention that it was an extremely unfamiliar test compared to what I was used to. As far as I’m concerned, I could have been spared the trauma and I am grateful that the selection committee decided to look further than that. Generally, it is unclear to me what exactly the GRE conveys about one’s intellect (it certainly supports the testing industry well enough). It is also unclear how exactly success in graduate school (grad school performance) is to be understood given that philosophy PHD programs (in the US) seem to be quite varied in terms of their hoops and loops. In many cases, the particular and personal challenges that someone overcomes in order to successfully complete a program do not transpire in their grades or exams, just as they do not show on the GRE tests. This might be particularly the case for minorities and women.
I think it’s a pity programs are moving away from the GRE. As several commenters have noted earlier, the argument that the GRE is not predictively accurate for graduate program success is extremely weak given the serious problems of (1) selection bias, (2) severe range restriction, and (3) difficulties quantifying “graduate program success” (first-year GPA is a very bad, but standard, measure). Furthermore, to only look at studies that investigate the relationship between the GRE and graduate school success is myopic and ignores vast masses of relevant data. The GRE test is essentially an IQ test, and there are tons of studies that show the predictive accuracy of IQ tests in cognitively demanding professions.
What exactly do you think the GRE measures? Considering that test makers themselves admit the only skill the GRE measures is one’s ability to take the GRE, and that one must pay a fees to both take the test and send scores to institutions which can disqualify talented but poor students, what justification is there to take it seriously as a measure of anything relevant to one’s ability and potential as a graduate student?
While it may be true that there are tons of studies which show the predictive accuracy of IQ tests and success in demanding professions, it does not follow from this that one’s IQ score is a reflection of their skills as a professional.
If I were a student applying to a Philosophy graduate program, I’d wish to be considered on my ability to do Philosophy, not on my ability to take a standardized test, but I guess that’s only because I’m pitiful.
I’m happy many people are explaining why some of the statistics and reasons offered perhaps aren’t as convincing at first glance, but it does worry me how many philosophers – the people supposedly good at critical thinking – here are misusing statistics or jumping to the conclusion that this is obviously a good idea. So many people didn’t consider that a measure’s ability to predict variance within a group is not the same as ability to predict achievement in a population overall, under-representation is not the same as bias or discrimination, whether bias on one measure might be more preferable because all the alternative measures introduce even more bias, or recognise that with limited spots and resources that some method of filtering is still going to needed. I’m not saying Penn’s move is bad, they can do what they like and I don’t have enough evidence to form an informed opinion. But the speed with which people fall prey to confirmation bias or motivated reasoning astounds me. Caring lots about injustice does not thereby make one good at assessing empirical evidence or identifying which measures are likely to be effective.
This is great news. The GRE is such a ridiculous test or at least it was 15 year ago. I presume it’s probably still ridiculous. To be a successful philosopher you need a strong writing ability and a philosophically creative mind. Most of the GRE is just entirely irrelevant to doing philosophy. Even the reading comprehension components are only abstractly related, as you’re required to read boring excerpts in a timed, unfamiliar environment. The idea that performance in such a situation translates to reading comprehension of philosophy while relaxing on your couch with a coffee is just strange. For one test anxiety absolutely locks many people’s minds up so that they can’t think clearly. For another comprehension depends on familiarity. If you’re passionate about philosophy and you’re familiar with the language of the subject, that’s going to affect your comprehension for philosophy. Back in the day they had all this vocab you had to memorize, most of which you don’t need to do philosophy. The math is even less related. The test may be more appropriate for core subjects, which it seems more directly comparable to, but for philosophy it seems mostly irrelevant. However for many reason, e.g. the timed high anxiety environment for one, I think the test is pretty ridiculous as an indicator of anything.
The GRE also discriminates against some students in another way. Some students don’t know that for most, if not all, departments GRE scores don’t count for much — definitely not nearly as much as the writing sample. And such students might invest a ridiculous amount of time studying for the GRE — 4 months in my case — while somewhat neglecting other parts of their application file. I think this tends to happen with students with little to no access to faculty that are familiar with the admission process, which is probably more common outside North America and the UK.
Meant to say: “the GRE requirement discriminates”.
I, too, probably invested *far* too much time studying for the GRE, thinking it would make me competitive for Leiterific programs. I hit the 90th percentile across the board but wasn’t so much as wait listed for ranked programs. There were other problems with my application—unprestigious academic background, unfocused statement of interest, unpolished writing sample—whose significance I wouldn’t fully appreciate until after I started grad school. It’s quite possible that the GRE played a significant role in getting me where I am now, but I confess that—at the time—I was seriously mistaken about my competitiveness. (Perhaps it’s also worth noting that I come from a small-town, non-academic family, and that the philosophy department at my modestly selective alma mater was small.)
Not to be crass, but do these students just not take the time to research the process? I got a massive amount of information when applying from doing a web search for “how to get into philosophy phd programs”, reading the gradcafe forums, and some posts on the most visible philosophy blogs. I was at least able to figure out that the writing sample would be the best use of my time.
You make a fair point, but i do think information about the GRE’s importance on online forums and such isn’t always obvious, and sometimes is conflicting. I’ve even heard varying advice from faculty with plenty of admissions experience. One can’t afford to entirely discount or neglect it in any case, if one is trying to make one’s application as strong as possible.
I agree that what should be clear from online forums and even this discussion is that the writing sample is far and away more important than the GRE, and that if you have to choose between spending time on one or the other, spending time on the sample is the obvious choice for an applicant.
FWIW, I had relatively low GRE scores (I actually ran out of time writing the final paragraph of my essay, which I assume influences whatever metric Phil depts look at when determining an applicant’s aptitude for graduate study) and the combined costs of application fees, transcript fees, and GRE score fees made it financially impossible to apply to more than 4 schools my first time on the application circuit (I opted for a funded terminal MA) and 6 schools when applying to PhD programs. Realizing the huge risks of such a low application rate, I corrected for my financial shortcomings by focusing on writing the best writing sample I was able (I still think it might be my best piece of writing, which may or may not say something of my progress in grad schoo sincel) and by working closely with advisors who a) had a large network of professional contacts, and b) would be able to write the strongest and most effective letters (this probably comes across sounding pretty mercenary, but I really loved every one of my letter-writers). In the end I applied to 6 PhD programs which I thought would be a good fit, with 2 of them being reach schools and none of them what I would consider safety schools. I was accepted to both of my top choices (neither of which was a reach), even with my poor GRE scores. All this anecdotal evidence to say, the GRE does not necessarily serve as a barrier to graduate study in all cases. Of course, my experience may be worth little to those who have found the GRE to be prohibitive, but I thought I might share anyway, just to contribute to the discourse.
I recall the halcyon days when you could just memorize a few key words to ace the GRE and get into grad school. I hope this decision does not precipitate an end to all that.
Hah! Good one Tim. After 42 years I can still recall learning “chillblains” prepping for the GRE before taking it at UMKC in Missouri. Thanks for that!
And then mis-spelled it for added humor. 🙂
“We knowers are unknown to ourselves.”
I’m surprised at everyone’s willingness to accept the debate is in good faith and departments truly want to be unbiased. I think it’s more likely a debate about different forms of bias that are attractive for different reasons.
It’s telling that some of the anti-GRE comments defend UPenn by pointing out it hasn’t claimed to be reduce overall bias. That tells us the anti-GRE crowd may not be motivated by the noble aims they claim. So, paying attention to possible concrete outcomes that they’d like us to not consider while we cheer their righteous symbolic action, let’s ask: who benefits and how?
On the question of more or less bias (and whether low income and minority students will benefit), we don’t know. But we do know what *kinds* of bias are favored. The biases of the GRE aren’t very manipulable. They’re imbedded in the test, so admissions committees can only take them or leave them. The biases of letters, grades, and pedigree, and “soft skills,” in contrast, are much more flexible, much more under their control.
So, departments have an interest in getting rid of the GRE so they have more control over how material is weighted or overweighted, what counterweights can, if needed, be downplayed or ignored, and how to interpret the value of materials. It allows for each department to more effectively impose its own distinct biases to create and sustain the academic culture it prefers.
It’s not a debate about how to get rid of bias, but how to make bias more useful to the interests of departments in selecting the kind of people they want in respects more related to culture and fit than academic and philosophical ability.
You’re reading too much into the comments. Making claims about reducing overall bias is different from wanting, aiming, and hoping to reduce overall bias. I pointed out that there was no claim that Penn reduced overall bias because a defender of the GRE criticized Penn for making a claim that it did not demonstrate to be true. This criticism is inaccurate, and one way to show that the criticism fails is to point out that no such claim was made. That’s the only reason for doing it.
I’m not sure whether you had my comment in mind when referring to the “anti-GRE comments.” But my comments are not anti-GRE. They’re anti-weak-criticism comments. I’m not sure whether Penn made the correct decision (and none of what I said above suggests otherwise), but I’m sure that many of the criticisms of Penn’s decision are weak.
Jen,
I specifically had in mind the commenters who enthusiastically praised the decision without qualification or question. But I did suspect you were opposed to the GRE, since your replies to the critics seemed aimed at complete refutation of objections rather than, as I’d expect from an undecided, revision of them or raising your own concerns.
I think you’re focusing on the letter of statements at the expense of their spirit. The spirit of Penn’s statement is that they want, among other things, to reduce discrimination and bias. If they’re disregarding the GRE for that reason in substantial part, then they are implying that doing so will reduce bias more than it will increase it.
Likewise, I think you miss the spirit of non-Leiterific and IGS’s comments. I don’t think they’re necessarily “defender(s) of the GRE”, but skeptics of the implied Penn position that getting rid of it will reduce bias. Now, you can reasonably argue there is no such implied claim, but the spirit here isn’t about Penn, but about the practice generally.
We all want (I think) to reduce discrimination and bias. Some worry that removing the GRE could decrease it in one way, but increase it in others, making us worse off. That’s the real, bigger issue here–not the particular case or statement by Penn.
I don’t know who would deny that failing to use the GRE might reduce bias in one way and increase it in another. It is obviously true. It is also true for (perhaps most if not all) other measures of potential. Further, any measure brings bias. Given this, why argue over it here? It seems to me that there is no good reason. The question we should be focused on is: which biases are acceptable? Ones that continue to prevent people who are historically underrepresented in philosophy and academia? Or ones that don’t? A combination?
I would also add the note to Yan’s statement “Now, you can reasonably argue there is no such implied claim,” which is certainly true, with the added statement, “so long as you replace it with a different implied claim that justifies the position.” Denying that the implied claim was made does not defend the argument on its own.
Jen seems to be arguing that she can defend the position by arguing for (1) the truth of the reasons given and (2) identification of the claims made in the announcement. This is simply not true. A majority of rhetorical arguments feature implied but necessary assumptions to justify the conclusion. I take it to be an important task of philosophy to identify and debate these implied assumptions.
Consider an example. Suppose I put together a team and I reject someone “because the applicant is from New Mexico.” This is presented as a reason, and it may be true, but it does not justify my rejection on its own. To do so, we would need to identify an implied assumption that *justifies* the use of the fact that the applicant is from New Mexico to reject the applicant. I can imagine good implied assumptions (e.g. we take representatives from each state and already have a NM rep.) and bad implied assumptions (e.g. I dislike the New Mexico football team). Yan calls this the “spirit” of the argument, where I would call it the justification, but it is hinting at the same requirement of holding institutions accountable for justifying their decisions.
Showing implied assumptions are false is of course a good way to undermine a claim or argument. To do this again Penn, there has to be such an assumption implied by some claim of the announcement. There simply is no such assumption. The closest thing is the second to last sentence, roughly the claim that that Penn won’t discriminate by using the GRE. This does not imply or assume that Penn is less biased *overall*. It assumes that the GRE is a source of bias, and that eliminating it is to eliminate this bias. Apparently, Yan and IGS, you two think otherwise. But you are mistaken.
Jen: “The question we should be focused on is: which biases are acceptable? Ones that continue to prevent people who are historically underrepresented in philosophy and academia?”
Perhaps there’s a source of misunderstanding here. The quoted question implies skeptics are worried about tradeoffs between *different kinds of* bias (“which…are acceptable?”). But my own worry, and I suspect others share it, is that dropping the GRE might increase bias *precisely against* “people historically underrepresented,” specifically against low income students, minorities, and women.
The worry is that while the GRE might wrongly eliminate some of those underrepresented students, it might also help out more of those underrepresented students than it eliminates, since they may be even more disadvantaged in the materials that are now given greater weight, while a wider number and variety of materials considered reduces the significance of any one item.
If your goal is to have more historically underrepresented students who have been denied in the past due to bias accepted into programs, then we’re worried about the same thing.
They need not be worried about the same. This is because Penn is worried about one specific source of bias, the GRE. Eliminating it is to eliminate the bias the GRE brings. Of course, they should worry that doing so brings further bias to those who they aim to protect against bias. But insofar as it is not yet clear whether there is such a bias, what it is, and what options there are to deal with it, Penn need not be worried so much about the bias brought by not using GREs. Penn can instead focus on what is clear: that to stop using GREs is to eliminate a source of bias.
“insofar as it is not yet clear whether there is such a bias, what it is, and what options there are to deal with it, Penn need not be worried so much about the bias brought by not using GREs. ”
This is true only if there’s no good reason to suspect such biases exist. Otherwise, of course they should be worried. (A good reason to suspect so: a major source of the GRE’s bias is systematic disadvantages in education as a whole, which also affect the other application materials.)
“to stop using GREs is to eliminate a source of bias.”
My first worry is that this may not be true. Bias isn’t an intrinsic property of the GRE, but a relational, social property. For example, one form of that bias is that one group is given an unfair advantage by being given better preparation. I can replace it with a different material, but if the relational advantage remains, the bias remains. If the problem comes from extensive, systematic disadvantages in the educational system, removing the GRE may not in fact eliminate the real source of the bias–it may treat a symptom not the cause.
Second, eliminating a source of bias is not always the best thing to do; sometimes we have to tolerate one bias to protect against a greater one. For bias isn’t a solitary property of the GRE. The GRE can put some underrepresented group applicants at a disadvantage while *also* protecting other underrepresented group applicants from disadvantage. So eliminating it might indeed “eliminate a source of bias” but *also* eliminate a protection against bias.
Take, for example, the cost. For some, a severe disadvantage. For others, such as women and minorities who are not from low income backgrounds, it will be a surmountable disadvantage and will not lessen their chances at all. (In my own case as a low income minority student, I studied outdated test prep books at the library and applied to a very small number of schools, putting way too much time into carefully picking them.)
But other disadvantages are not easily surmountable. You can’t make up for a lifetime of educational disadvantages, for lack of the cultural experience and knowledge necessary to impress, befriend, and network among academics, for unconscious, subtle indicators of class or identity in one’s way of speaking and writing, etc. You also can’t make up for going to the wrong school and having letters from the wrong people.
So, when almost every other piece of information in your application is packed with biases that you cannot control, having one small thing–the GRE–that you can, with effort, use to minimize or recast and contextualize other biased information may be an important protection against bias.
To my mind, the most important question is which materials are most vulnerable to misuse. An less biased system can’t be designed on the assumption of virtuous actors. We should assume that admissions committees will try to misuse materials to promote their own biases.
With that in mind, I think it’s good to have some materials, like standardized tests, that are the same for every student and that produce information that committees can’t really interpret in dramatically different ways. A good GRE score can’t really be read as a serious mark against you, no matter who you are or where you come from. A really bad score can’t be read as a positive.
Contrast that to the other materials. A good letter from the wrong person could be interpreted as a mark against you. Great grades from the wrong school won’t help you, while bad grades from the right one won’t hurt you. A good writing sample with the wrong topic or style (often a reflection of disadvantages in experience of the academic world and culture) can hurt you, a mediocre one with the right cultural or topical fit may not.
Which kinds of materials have biases that resist such manipulation by bad actors, and which lend themselves to manipulation? Which kinds of materials have biases that applicants can overcome or minimize, and which have biases that applicants can do nothing to remedy?
It is not at all clear that what you’re calling “a good reason to suspect such biases exist” is a good reason at all.
Not using GREs eliminates a source of bias *in philosophy admissions decisions*. I did not mean that it eliminates a source of bias outside of admissions.
The decision to stop using GREs needn’t have been made in attempt to do what’s best overall. It may have had a more humble goal: to eliminate one obstacle preventing certain people from gaining admission to Penn.
IGS,
I generally agree, but called it the “spirit” because Penn does have a possible way around the argument. They’ve given two reasons that aren’t specifically about bias (no predictive value and no specific epistemic gain), so they could conceivably claim that even if dropping the GRE fails to reduce bias, it’s justified by the other reasons. Not that I think they’d take that technical win, though!
People used to believe, with or without warrant, I don’t know, that the analytic part of the GRE (which was basically logic puzzles) was an indicator of promise in Philosophy. Since they got rid of it, and replaced it with the essay, which literally no-one in grad admissions knows how to interpret, I suspect the pace of abandonment of GRE will increase. And its a good thing too. I agree with the people who think that very bad GRE scores, on average, indicate low promise, but only on average, and if that is genuinely the only negative in a dossier it would be stupid to reject on that basis alone (this is Penn’s fourth point).
We, also, just abandoned requiring GRE scores, and will not look at them even if they are sent.
Is there research evidence that GRE is biased against racial minorities and women? I don’t know the evidence on GRE at all, but I do know about SATs, ACTs, and APs (I’ve a research interest in undergrad admissions, but not in grad admissions) , and the evidence does not indicate that they are discriminatory.
I am surprised at all of the concern over the dropping of the GRE and can’t help but think that part of the explanation for it is status quo bias.
I mean, suppose the GRE had never been invented. Is there anyone who would be saying, “Hey, fellow philosophers, you know what we need? A test that we can give all applicants to grad programs to see which will be the best philosophers. What’s that? No, the test wouldn’t be just for those who want to study philosophy, but for all disciplines. What’s that? No, it won’t have a philosophy section. What’s that? No, there probably won’t even be any philosophy questions on it. I’m telling, you, this is a great idea.”
Yes, I would definitely have been in favor of such a test. I think standardized IQ tests are, on the whole, a very good idea in many contexts.
Not a particularly charitable or substantive contribution.
“All the concern over” probably overstates it. More like a solid “eh…” Just skepticism toward the degree of certainty supporters seem to have that it will make a significant, positive difference. If I had to guess, I’d predict it won’t make a difference at all, positive or negative.
But yeah. I suppose it would be weird for people trying to promote fairness for applicants from underrepresented groups to worry or wonder about whether an action will really accomplish that.
Just like it would be weird for philosophers on a philosophy blog to respond to a confidently asserted claim with questions, criticisms, and doubts. So weird.
Justin, it really comes down to the question of what tests like the GRE aim to measure. One of the most extensively studied questions in psychology over the last 100 years is the structure of cognitive abilities: Are there as many abilities as there are contexts of application or are there just a few abilities that have substantial cross-domain generalizability. There is a continuum of positions here among the relevant experts but my take is that most cluster on the “just a few generalizable abilities” side of things. Standardized tests like the GRE are supposed to capture such generalizable abilities that manifest across multiple academic domains. So that is why it might be rational to administer a test that has no actual philosophy content and yet expect it to correlate (modestly) with performance in philosophy grad school.
Your point about what’s rational is irrelevant to Justin’s comment because the comment is about preferences not sensitive to rational considerations. Justin is suggesting that people criticizing or questioning Penn’s decision likely have such a preference for the status quo.
I’ve seen some decent suggestions to the effect that eliminating the GRE doesn’t necessarily accomplish what Penn (and others) say they want to accomplish by eliminating it. Fair enough. But I’ve seen very little little that should convince me to think that it constitutes a positive contribution to the admissions process. It seems to me that (as an admissions criterion) it’s neutral at best. Given that it’s pretty pricey, and at best neutral, I say, “Good riddance.”
A little late in on this, and I read most of the comments but not all, so I apologize if this has been mentioned. I did want to mention one way that I’ve thought about the value of the GRE. It seems to me that, in some ways at least, it can be indicative of some sort of intelligence/work ethic ratio. What I mean is this:
The GRE is a tough test, so if you get an insanely good score on it, it is likely that you are a) extremely intelligent, or b) whatever you lack in intelligence you made up with hard work (i.e. you put in the effort of LOTS of unpleasant study time). Whether a or b, you are likely desirable for a department. If you did not get a good score, it is likely that c) you’re just not that intelligent, or d) you’re intelligent enough to do well but just weren’t willing to put in the time/energy. Whether c or d, you are likely less desirable for a department.
Obviously luck can play a role in the GRE (e.g. you got the “perfect” batch of questions), but 1) true luck will be statistically negligible, assuming decent test design (and I say “true luck” to remain neutral on whether there are question biases that favor some groups over others and how we should evaluate such biases), and 2) it’s not clear that luck should count that much against the GRE any more than it does against other parts of the application (e.g. GPA – got unlucky and had a bad professor in a small department and had to take a majority of classes with him or her, which resulted in a not great letter and/or less than perfect GPA). Furthermore, I acknowledge worries about class bias (affluent students can afford to put in the time because they don’t need to work to support themselves, etc.), but if I recall from my applications, most of them had a section for you to write in whether you thought that part of your application wasn’t reflective of your potential and why. That seems like a perfect place for an applicant to say that they think their GRE scores do not reflect their potential and that if they hadn’t been working or whatever that they would have had the time to put more study time in.
So, while this doesn’t suggest that the GRE should be the sole deciding factor, or even valued very highly in an application packet, bad scores should raise red flags because it is likely that the person either isn’t smart enough to hack it or not willing to work hard enough to hack it. Grad school is tough and as great as studying philosophy is, there are a lot of existentially, emotionally, and physically draining years to go into getting a PhD (not to mention financially draining…not being able to invest very seriously in your 20s is a HUGE opportunity cost that not enough people think about). So if you can’t put in the time and effort to study for the GRE then why should a department think you’re willing to commit 5+ years of your life to studying philosophy? For these reasons, I’m still inclined to think that it has some value, at least minimally as a first round filter.
“So if you can’t put in the time and effort to study for the GRE then why should a department think you’re willing to commit 5+ years of your life studying philosophy?”
Because you have otherwise demonstrated a love for philosophy and are therefore likely to see the value of grinding in pursuit of a Ph.D., whereas studying for the GRE is boring as hell and of dubious value.
Why should Ph.D. programs value hard work for hard work’s sake, or, for that matter, for the sake of becoming more competitive for admission to grad school? I don’t want grad students (or new hires) who have spent their time becoming more competitive per se; I want them to have spent their time becoming better thinkers (and teachers).
As an applicant this year in a related field (psych) I think the GRE is absolutely absurd. I’m spending time (& way too much) learning words and formulas that have no bearing on my research interests. It has been destructive to my mental and physical health, from a combination of various factors such as stress, lack of sleep, and the starvation required to afford the test.
“If you can’t handle the GRE, you won’t be able to handle grad school” well actually, I find the GRE particularly dehumanizing and insulting in a different way than the kinds of stress you encounter in grad school. In general I love research and can handle paper rejections or whatever. This is NOT the same thing. Also, if grad school is so stressful, why not let me relax beforehand?
“What about the LSAT, MCAT, etc., ?” Maybe they are also bad. I wouldn’t really know because I haven’t studied for the MCAT. I have studied for the LSAT though and it totally makes sense that they would require it: law school really is about being “smart,” that’s all they care about, there are like three schools that everyone wants to go to and probably millions of people from around the world want a spot. I think the GRE is much less fair than professional school tests. I think first of all, because med school and law school are high-paid professions, the continued rat race makes sense (what is a philosophy PhD really gonna do?).
Grad school has the word “school” in it, but to me, really doesn’t feel like school in the same way as professional school like law school and med school. It’s not really about the classes, it’s about spending five or six years doing research with someone; it’s about starting the mentorship process that is followed by a postdoc and (hopefully) a faculty job (Do faculty jobs require aptitude scores? Do postdoc jobs require aptitude scores?)
Grad school should necessarily be less of a rat race than law school or med school. Grad school should be about finding those one or two people whose research interests overlap with your own. There’s probably only like five people (at most) interested in my specific topic applying to grad school right now. Grad school isn’t about the whole world trying to get into three schools. It’s more of a “matching market” (Roth). It’s about faculty mentors and students “finding” each other. And the GRE has nothing to do with that.
Ok, I need to make a psychology-specific point for those psych people who happen to be reading daily nous blog comments. Like many psych grad applicants, I am applying to the NSF grad student award, a very competitive award for hundreds of thousands of dollars that literally funds all of grad school and more. It would be great for a department/university if I got it – they wouldn’t have to pay for me and would get money. But unfortunately, I find myself spending a large portion of my time learning outlandish words and formulas about shapes. Why? Let me spend every possible waking hour working on the grant and win the damn thing. It is literally in the department and university’s best interest to do so.
Now, a broader point. I know that a department having high GRE scores helps prestige or whatever, but, come on, is that actually what matters? We all know what actually matters: faculty position placement. And you think that treating someone like a lab rat from the get-go and taking away large chunks of time before starting grad school is going to help them publish as much as they need to?
End this now.
I think a prior question here is how much time and effort should be put into preparing for the GRE. You apparently take the answer to be “a lot, too much even.” But on what basis do you make that answer?
I assumed the intention behind the test was to be another “un-study-able” evaluation. I.e. if it’s possible to prepare in a few months prior, then the test needs to be fixed. As such, I spent no time beyond reading the information on the website about the format, time, what to bring, etc. preparing for the GRE. As such, I never saw reason for any of these complaints. (I still find the cost of the test ridiculous, but the time commitment only seems to be a problem for people who choose to pour a lot of time into it. Otherwise it’s just an annoying use of six hours.)
Other famous universities in UK do not require GRE, so why the US? This decision is wise in encouraging more potential students to apply and pursue a terminate degree. It’s also a good move for international students particularly speaking. One less worry on their admission checklist. Many have hesitated pondering on taking or not taking GRE, US or other countries. In case their visa is denied then other countries do not require GRE to go there for study. In light of stricter laws that are threatening international students to the US, this decision from Penn will give them more encouragement to apply and try for U.S. as a study destination.
At the University of Illinois at Chicago we have been working on dropping the GRE requirement, for the last couple of years, for similar reasons to Penn. For what it is worth, other departments at UIC are taking or discussing similar steps, and most of the DGSs seem in agreement that they are not helpful overall.
We have faced a couple of institutional barriers, but we have managed to have them taken down, and so we too will not be collecting or considering GREs. It may take a couple of weeks for this to be fully reflected in all our online materials. Please apply to UIC!