Refereeing & Freedom of Information Acts (updated)
As noted in an update to a previous post, philosopher David Wallace (Pittsburgh) has made a request under the UK’s Freedom of Information Act for Oxford University Press (OUP) to provide him with correspondence related to certain publication decisions on recent submissions by Alex Byrne (MIT), Holly Lawford-Smith (Melbourne), and Richard Marshall.
Here’s the text of his request, which he shared in a comment here yesterday:
I have made the following request to Oxford University under the UK Freedom of Information Act (OUP is legally a department of Oxford University, which is a public body within the meaning of the Act). Oxford is required by law to respond within 20 working days; I will keep people informed.
1) All correspondence held by Oxford University Press and pertaining to the commissioning, reviewing, and editorial decisions for Alex Byrne’s book Troubles with Gender, excluding the names and other identifying information of any external reviewer used by OUP and of any employee of OUP without editorial authority, and also excluding any correspondence of a purely logistical nature, e.g. concerning payments or publication schedules.
2) All correspondence held by Oxford University Press and pertaining to the commissioning, reviewing, and editorial decisions for Alex Byrne’s contributed chapter on pronouns to the Oxford Handbook of Applied Philosophy of Language, excluding the names and other identifying information of any external reviewer used by OUP and of any employee of OUP without editorial authority, and also excluding any correspondence of a purely logistical nature, e.g. concerning payments or publication schedules.
3) All correspondence held by Oxford University Press and pertaining to the commissioning for publication, reviewing, and editorial decisions for Holly Lawford-Smith’s book Gender-Critical Feminism, excluding the names and other identifying information of any external reviewer used by OUP and of any employee of OUP without editorial authority, and also excluding any correspondence of a purely logistical nature, e.g. concerning payments or publication schedules, but specifically including any correspondence pertaining to the book’s reclassification as ‘academic’ rather than ‘trade’ and to making it available, or not, on Oxford Scholarship Online.
4) All correspondence held by Oxford University Press and pertaining to the commissioning, reviewing, and editorial decisions for Holly Lawford-Smith’s edited collection Sex Matters, excluding the names and other identifying information of any external reviewer used by OUP and of any employee of OUP without editorial authority, and also excluding any correspondence of a purely logistical nature, e.g. concerning payments or publication schedules, but specifically including any correspondence pertaining to OUP’s reversal of its original decision not to publish that volume.
5) All correspondence held by Oxford University Press and pertaining to the commissioning, reviewing, and editorial decisions for a proposed volume on women philosophers to be edited by Richard Marshall and commissioned in 2019 or thereabouts, excluding the names and other identifying information of any external reviewer used by OUP and of any employee of OUP without editorial authority, and also excluding any correspondence of a purely logistical nature, e.g. concerning payments or publication schedules.
In a comment at Brian Leiter’s blog, Byrne described the materials he received from OUP regarding his book manuscript:
Just to clear up any confusion about the reports (and then I will fall completely silent on this issue).
1. An annotated version of the MS. Helpful comments, all minor. Nothing in the comments to suggest that the book should not be published. I was very grateful to this reviewer, who put in the most work by far.
2. Notes taken by Peter of a conversation with a reviewer. Short, negative. This contained some clear mistakes, so I wasn’t sure what to make of it.
3. Very positive.
4. Very negative. Clearly a review from a philosopher who works on sex and gender; they tend not to be in my fan club. At least this reviewer attempted to address explicitly some of the arguments in the book, although seemed to misunderstand the audience. Easy enough to show that the reviewer’s complaints were all wrong (according to me!). I was not given the opportunity to try to show this, unfortunately.
5. The MS with Peter’s annotations—just about word choices and some places where Peter thought the “tone” was off. I had been very responsive to this sort of concern earlier.
Again, what was delivered was very close to what was promised (and, it seemed, eagerly anticipated). No one said, “Whoa. We thought it would have *a lot* more philosophy, or be less of a trade book.”
Even without the reviewers’ reports, which OUP has not yet released, we have a very different picture than what people might have been imagining in light of Byrne’s initial description of it. This now looks much more similar to a very common story: author submits manuscript, the reviewers’ comments are mixed and include remarks that the author takes to indicate mistakes, lack of charity, a failure to ‘get’ what the author is doing, the editor thinks it would be more work than it’s worth to get the kinds of changes made that would make it the kind of book they want to publish, and so the manuscript gets rejected.
While it is possible that political bias (or related commercial considerations) entered into the decision, the fuller set of facts we have before us now do not appear to provide much (any?) evidence of that. Of course, if one assumes rampant bias in our publishing institutions against views on transgender issues like Byrne’s (what he might call a “gender critical” position), then one will no doubt see such bias as part of the explanation for the rejection of Byrne’s book. But Byrne offered the rejection of his book as evidence that there is such bias. So assuming such bias was in play here is just question-begging.
I think that if we had had the fuller picture from the start, the story would have had less momentum, and probably not enough to roll all the way to a Freedom of Information Act request, but here we are.
At this point, some have raised concerns about the effects of Wallace’s request, should it be granted. One worry is that any university press affiliated with a public institution could be subject to FOIA requests (not just in the UK but in the US and elsewhere), and that this would deter people from agreeing to referee for them. For a system that seems already quite stretched to find enough reviewers, that might be a problem. In a poll on Twitter, Eric Schliesser (Amsterdam) asks people if they would still review for OUP if they knew their reports could become part of the public record:
David Wallace may change editorial practices forever if he succeeds at getting OUP to publish all the material pertaining to the Byrne manuscript. Interesting question: would you still review book ms for OUP if you knew it could become part of public record
— Digressionsnimpress. (@nescio13) April 20, 2023
At the moment, the results suggest publicity would deter many people from reviewing. I was surprised at this result (I answered Eric’s poll with “sure”). I’m curious what you all think about this.
Discussion of this freedom of information request, the general idea of bringing such requests to bear upon university press decisions, and their effects, are welcome.
(Not welcome: discussion of first-order matters on sex, gender, transgender identity, etc.)
UPDATE 1 (4/21/23): While no single example one way or another will itself establish a pattern, the following may be of interest. In a related discussion on social media, Sophie-Grace Chappell (Open University), a philosopher who is a trans woman, writes:
I have no intention of reading this article right now (no spare bandwidth) but is there a beef, really, beyond the complaint that OUP unexpectedly refused his typescript at a late stage in the process, and so he was (shock horror) “forced to” publish with Polity?
Because *exactly that* has also happened to me over the winter.
It all caused me a good deal of stress and distress, but I am inclined to say that it simply comes with the territory of trying to get stuff published. And that it would be obviously preposterous to regard that sequence of events as being silenced or cancelled.
Overall, I’d say, it’s almost like OUP Philosophy aren’t biased against gender-critters at all, nor for that matter against trans-inclusive work. And it’s almost like the philosophy publishing industry as a whole is not so much censoring/ silencing work on this topic, as lapping it up (“Sex sells!”) wherever that proves to be practicable in a complex and unpredictable world.
UPDATE 2 (4/24/23): Alex Byrne has shared the contents of the June 23, 2022 rejection letter for his book manuscript from Peter Momtchiloff. It doesn’t settle anything. This of course means that people will tend to read it as providing evidence for their preconceptions about what happened. For an example of that, see the post in which it was initially shared. Here’s the text of the email:
Dear Alex
Thank you for sending me the manuscript for Trouble with Gender. I have now read it, and consulted with advisers and colleagues. I regret to say that OUP will not be able to publish your book. I am sorry to have to give you a negative response. I’m afraid our judgement is that the book does not treat the subject in a sufficiently serious and respectful way. The same kinds of problems that I found with the introduction we now find continue throughout the book.
I am forwarding a copy of the manuscript which I have annotated. I hope that these comments might be helpful to you.
I shall also forward comments from four academic readers. Whatever you decide about the book, I hope that these will be useful to you. It may be that another publisher would not agree with OUP in finding this treatment of the subject inappropriate, and would be happy to be able to publish the book. Perhaps the comments I send may then prove to be useful to that end.
I am sorry to have to write to you with this decision, since you are a valued OUP author.
With best wishes
Peter
UPDATE 3 (4/24/23): I’ve closed comments on this post, as I do not have time to moderate them today.
I’m mystified why anyone would not review anonymously for OUP just because they realise what UK law allows (apparently). People know tenure review letters can become public if there is a lawsuit but does *that* fact deter a significant number of people from writing non-anonymous tenure letters?
But in any case, I’m confident that many people’s answers to Schliesser’s poll are powered by their views on Byrne’s book and the line Wallace was taking with respect to it. I would take the results with a very large grain of salt (quite apart from it emerging from the Stygian caves of philosophy twitter).
I think it matters not just what is technically allowed by law, but also whether you think it is likely to happen. While tenure letters *can* become public in a lawsuit, I know that this is fairly uncommon. If I thought that the tenure letters would *most likely* become public, then I would consider more carefully my decisions about whether to do them, and be more careful in some ways in how I wrote them.
Strongly agreed.
I also think it is worth noting, in this particular case, that the people who have least to lose from refereeing under conditions where reports might be made public are those whose names are already publicly identified with a “team” on these issues. If one worries that this area is already superheated in a way that inhibits the normal operation of knowledge production, as many in the comments on this and the other thread have seemed to, then one should also be especially worried by the prospect of forced publicity driving out everyone but the biggest keyboard warriors.
I think it is a very good thing that Wallace’s request excluded reviewer identities, and hope that that is a line no one else crosses either.
Sure — I think the equivalent point here is having anonymous book reviews become public is still going to be so massively unlikely to happen, that there is little reason to think very many people will be dissuaded from writing them (quite apart from the anonymity issue).
But tenure letters are fundamentally different. The task of judging at least the correctness (and arguably the importance) of papers is *the* product the discipline needs to produce. If you can’t get people to review a work because their views might be disclosed how can you have any hope that the discipline can come to any kind of judgement about the work at all?
Heck, if we are too scared to publicly review works in a certain area maybe it should just not be done because we can hardly count on people being brave enough to publish arguments if they won’t publish an evaluation so the process seems unconducive to truth discovery.
Indeed, arguably the prospect of having referee reports FOIA’ed *improves* the system, in ways which should make one *more* likely to write reports. The prospect of sunlight, let’s suppose, would (i) make editors and publishers react to reports more defensibly; (ii) make other referees write more defensible reports; (iii) induce one to be one’s most responsible self in writing the report. Recognizing all of these, one would have more confidence in the system, and feel more strongly that one’s best efforts are deserved — so, presumably, make one more likely to contribute them.
In the case of tenure letters, there is a complicating factor. Tenure is much more a matter of livelihood than is any individual publication: with a paper, one can “drive until one qualifies” much more comfortably than with one’s career. Moreover, tenure involves a judgement of *the person* and not merely of *one among many projects*, with vastly greater emotional resonance. Accordingly, a negative tenure letter is much more likely to provoke feelings of hatred toward the referee (and stronger ones) than is any negative referee report.
I would not feel threatened if my referee reports for various Presses were made public. Indeed, I write referee reports that can be shown in whole to the authors, both for book manuscripts and for journal submissions. Approaching matters this way, I consciously make the report respectful in tone, even when I think there is little merit in the manuscript or paper. I would be more inclined to think that the authors receiving the reports would generally want them shared publicly.
When I’m refereeing a paper, I try to get into the following mindset first:
You are seated in a comfortable chair at a table with all of the other referees and the author, the editors, perhaps anyone that the author chooses to share the report with, and indeed, the general public if for some reason people are interested enough to make a FOIA request. You have gathered to discuss an issue of mutual concern, and you are aiming to learn something from the conversation. Take off your shoes if you’d like. Wriggle your toes. Appreciate the wonders of everyday life in the twenty-first century. On the table in front of you is your favorite beverage. Through the window is your favorite view. And seated next to you is a child, who you brought with you for a lesson on how to discuss controversial issues with strangers.
(In all seriousness, we should all write reports such that we’d be happy if their content (though not our identity) were made public, and if learning that the law allows for this bothers people, perhaps they should rethink how they’re writing their reports.)
I’m going to comment once here, and then try very hard to stay away, because I’ve lost way too much time on this over the last few days.
Firstly and most importantly, I want to underscore that this was a request that explicitly excluded names or identifying features of reviewers. I say this partly because there seems to have been considerable confusion about this on twitter and in the earlier thread, and it’s not at all obvious from (e.g.) Eric Schliesser’s poll, which talks about releasing “all the material”, that I wasn’t intending to compromise anonymity.
I think referees have at most a weak, defeasible presupposition of *confidentiality* for their reports, that can easily be overridden. I, and Justin, and Brian Leiter all called for Alex Byrne to release the reports, and I was somewhat surprised that his own interpretation is that he isn’t entitled to. At any rate there is clearly at least not an agreed norm that authors can’t share reports; many other people in the earlier thread, including some who otherwise strongly disagreed with me, shared my (and Justin’s, and Brian’s) interpretation. (Perhaps authors have a stronger expectation of confidentiality, but at least in the case of Alex Byrne’s own reports, he (a) wrote a magazine article about his experiences, and (b) said explicitly that OUP was welcome to release anything they wanted.) So I am really not concerned about this issue. (I would not support FOIAing reports to satisfy trivial curiosity, but the stakes here are not trivial, as witness the impassioned discussions here and at Brian Leiter’s blog.)
*Anonymity* is quite another matter. The peer review process absolutely depends on reviewer anonymity, and I think undermining that would be very bad. I’m not at all convinced that a FOIA request in the UK for referee names would succeed (they are plausibly personal information under UK data protection law) but in any case I was very clear that I did not want to be given them, as I hope is apparent from the request.
Secondly, my request asks for relevant correspondence, which potentially also includes correspondence between author and editor, any internal communications about the projects, and any external interventions. I think that’s highly salient, both because it’s a key part of Alex Byrne’s argument that the ultimate editorial decisions were summary and offered him no chance to revise, and because of the allegations being made of political pressure to, and within, OUP. But again I explicitly excluded the names of, e.g., OUP production staff from the request. I think it is salient to know, say, whether there was an organized campaign within OUP to put pressure on editorial staff, but I absolutely don’t want named individuals to be targetable.
Thirdly, reasonable people can differ about Freedom of Information legislation, and there is a defensible view that it is all illegitimate and that people who work for publicly-supported institutions should have an absolute right of privacy as to their work-related activities. I disagree: I think part of the purpose of FOI is to encourage people to conduct their correspondence and work in a professional manner on the understanding that in some circumstances it may be visible to the public, and I think that’s a proper purpose. (Certainly I conduct my own work-related correspondence that way: in a small irony, I think Pennsylvania is one of the only states that doesn’t let you FOIA a state university, but if I’m wrong and somebody FOIAs my (fairly minimal) correspondence pertaining to all this, they will discover that I am sometimes more acerbic and exasperated in private correspondence than public, but they won’t find anything I’m not happy to defend as professionally appropriate.)
I appreciate that FOI requests sometimes just throw up harmless chit-chat that can embarrass people without serving any public purpose. But I wasn’t planning to dump the whole thing on Dropbox. I can filter out irrelevant content.
Fourthly, there is also a defensible view that this happened too quickly. I did it swiftly just because the lead time on FOIA requests is quite long. If the situation develops in the next few weeks in a way that makes this request moot (say, if the delegates of the University Press set up an inquiry that seems likely to satisfy any transparency expectations) then I can always pause or cancel it. (I thought about doing it privately, and would probably have had a less stressful time if I had, but it seemed hypocritical to make a transparency request but to do it secretly.)
Fifthly, as was discussed briefly on the last thread, I think it’s quite unclear just what, if anything, OUP will actually release. I think there are perfectly reasonable questions here of how to balance a public organisation’s legitimate interests with the transparency principles of FOIA, and to a large extent I’m happy to outsource those questions to UK law. The University will probably consult its lawyers; if their view is that they don’t have to release anything, that’s an end of it.
Finally, I didn’t invent the Freedom of Information Act. I’m slightly surprised that some people are acting as if some kind of Rubicon has crossed. FOIA has been law in the UK for over twenty years, and every time I, e.g., fill in an online reference form for a candidate, I’m reminded that in certain circumstances it might be viewable by the candidate. (And similarly mutatis mutandis for most state universities in the US.) I thought we all knew this.
I don’t have a horse in this race, and I typically find you one of the most reasonable people participating in these online discussions.
But I will just point out that there seems to be some tension between you (a) being strongly exercised by a perceived procedural violation of academic publishing *norms*, while (b) attempting to rebut the charge that your FOIA request is itself a procedural violation of academic publishing norms by replying that what you are doing is, and has for over twenty years, been perfectly *legal*.
FWIW, as I understand it, he isn’t attempting to rebut the charge that his request is a violation of publishing norms. He’s attempting to rebut the charge that his action is going to dramatically change peer-review (as the crossing of the Rubicon dramatically changed…). Hence the “Rubicon” analogy.
Secondly, my request asks for relevant correspondence, which potentially also includes correspondence between author and editor, any internal communications about the projects, and any external interventions. I think that’s highly salient, both because it’s a key part of Alex Byrne’s argument that the ultimate editorial decisions were summary and offered him no chance to revise, and because of the allegations being made of political pressure to, and within, OUP. But again I explicitly excluded the names of, e.g., OUP production staff from the request. I think it is salient to know, say, whether there was an organized campaign within OUP to put pressure on editorial staff, but I absolutely don’t want named individuals to be targetable.
It seems that if there were any “external interventions,” you do want named individuals to be targetable. Hmm.
Can you elaborate on this, please? Is the suggestion that a FOIA report couldn’t disclose correspondence with external parties without also targeting named individuals? If so, why think that? Couldn’t those names also be removed to protect anonymity? And isn’t this more likely what Wallace intends, rather than something nefarious? If that’s not the suggestion, then what do you mean?
A FOIA report presumably could redact the names of external or internal parties in correspondence. Wallace wrote his FOIA request so that two sets of names would be redacted: external reviewers and internal staff without editorial authority. If Wallace’s request is fulfilled as written, names of external correspondents who are not reviewers would be disclosed.
Here’s the language, again:
1) All correspondence held by Oxford University Press and pertaining to the commissioning, reviewing, and editorial decisions for Alex Byrne’s book Troubles with Gender, excluding the names and other identifying information of any external reviewer used by OUP and of any employee of OUP without editorial authority, and also excluding any correspondence of a purely logistical nature, e.g. concerning payments or publication schedules.
Wallace’s explanation of his FOIA request, above, explicitly states that he hopes to find out if there were “external interventions.” If there were “external interventions,” it looks as if Wallace wants names.
I see, thanks for clarifying. Surely, however, he means to extend his anonymity requests to any external interveners. That’s the charitable reading—why would he want names when (a) he wants anonymity everywhere else and (b) names aren’t even necessary for determining if there was external pressure? The other reading just seems to imply something pretty… dark about his intentions/goals, which feels unfair. That said, it might be worth explicitly revising the request so that this is clarified.
I don’t want to interpret Wallace’s intentions uncharitably. He wrote and submitted his FOIA request in haste.
Nevertheless, if the FOIA request is carried out as written, it will have the consequences I spelled out. I think Wallace should withdraw or formally revise the request.
Isn’t the problem that in order to know whether any ‘external influences’ were relevant or not, you need to know who the people are (even if you do not disclose those names publicly)? E.g. if an email were included from Joe Public to Peter M. saying “You can’t publish this.” that’s very different from an email from an OUP delegate saying “you can’t publish this.” So in order for David Wallace to know whether the external intervention were relevant or not, he needs to know who they were.
I see your point, but I guess I’m not sure why that would be necessary. Presumably whichever entity responds on behalf of FOIA will censor out the relevant names, even though (e.g.) clubs, organizations, parties, etc. could still be revealed. So, for instance, we could learn (dumb example incoming) that the anti-Alex Byrne league sent emails to get the monograph axed, and OUP replied back with “no prob, will do,” even though all individual names/identifiers other than that of the league itself are omitted. Actually, come to think of it, it seems like even omitting the name of the league could work, as long as it was clearly not OUP. That would be informative enough. Again, I’m not literally suggesting any such thing. I just don’t see why something like it isn’t an option.
Would it be nefarious to want external interveners to be brought to light? Maybe I’m missing something, but (i) if someone external to the process tried lobbying or pressuring my publisher to drop my work, I would want this to be exposed, and (ii) if I ever engaged in such lobbying myself against another academic (which I can’t imagine doing), I don’t think I’d have any right to have it be kept a secret.
It’s surely the external intervention, if it occurs, that is nefarious here, after all. The threat of exposure seems like a good thing to discourage any such actions in future. (Am I missing something?)
No, perhaps it wouldn’t be. I just meant that imputing nefarious intent to David Wallace is unfair (even if said intent would be acceptable after all).
That depends on the nature of the external intervention. Suppose a major funding body wrote to OUP to say, “We saw Byrne’s article on gender. Please don’t publish a book by him on the subject. Remember where your funding comes from.” Then both Byrne and the public would have an interest in finding out this influence was being exerted.
By contrast, suppose a bunch of trans junior philosophers wrote letters (or Tweets) to OUP saying, “Please don’t publish transphobic ‘gender critical’ books, by any author–we really don’t need that right now.” The only possible reason to want their names released would be to punish them for sticking up for themselves. That would be despicable.
I think the second scenario is more likely than the first (though neither scenario is likely).
I guess I’m curious to hear more about when external interventions of the sort being imagined are best described as “sticking up for oneself” versus, say, “attempting to undermine the academic review system by pressuring academic institutions to silence one’s ideological opponents”.
Perhaps it’s analogous to the question of when jury tampering is permissible. (I’d think, “usually not, surely!”, even when — as now — I have no strong view on the first-order issues. But I guess one can imagine extreme cases where the usual rules would not apply.) Insofar as one remains neutral on the first order issues, such extremism is apt to look concerning, and defending the integrity of the jury system (incl. by punishing jury tampering, even when undertaken by people who are “sticking up for themselves”) doesn’t seem obviously “despicable”. There’s an obviously decent motive for procedural liberals to do this (viz., protecting the integrity of their institutions). But in the extreme cases where the tampering was truly warranted, then I guess it would look different. So I guess there’s no neutral way to adjudicate this, i.e. without taking a stand on whether we’re in such an extreme case that it warrants trying to undermine neutral institutions?
I have a very hard time imagining what an “extreme” case would look like in the real world of academic philosophy, one where external pressure of the sort we’re speculating about would be justified.
We can come up with toy cases (“if the book is published, a nuclear bomb will be set off in New York City”), but in the actual world nothing like this is happening. We can read (or re-read) the open letter sent to OUP when Lawford-Smith’s book came out to get a sense for what external pressure in this case (if there was such pressure) might have looked like. This included vague claims of harm, violence, bigotry, not-so-implicit allegations of aiming at genocide (“explicit objective of denying trans people the right to live freely or to exist at all”), and so on.
Clarification: this letter was written before Lawford-Smith’s book was actually even available to read. The book had been published but was not yet available.
In MCE’s hypothetical, it seems like you don’t need to *share* the names of the junior scholars, even if you think being influenced or swayed by them in the editorial process is a poor reflection on the editorial integrity of the procedure (sharing the existence of the influence-seeking communications is the relevant goal?), though, just as DW’s current FOIA protects the various staff from identity disclosure, regardless of what the contents of their communications are? So I don’t quite follow your point.
Suppose junior trans scholars wrote OUP (or another press) saying, “Some journals and presses have recently published ‘gender critical’ arguments that are intellectually shoddy. We believe the decisions to publish were motivated either by animus against trans people or by a misguided attempt to appear balanced. We ask that you apply normal academic standards to your review of proposals and manuscripts about gender identity. Get reviewers who work in the area, and if those reviewers say that a submission is not merely offensive, but also poorly argued, listen.”
Would that be undue influence? Should people who write such a message to a press be publicly named, so that bigots and ideologues on hiring and tenure committees can blackball them and so that alt-right keyboard warriors can doxx them?
I see you asking three-ish versions of your first question:
- Is it undue influence to write to an editor to complain that they are publishing shoddy work
- Is it undue influence to write to an editor to complain that they are publishing shoddy work that you believe was motivated by an attempt to appear balanced
- Is it undue influence to write to an editor to complain that they are publishing shoddy work that you believe was guided by literal bias against one’s own preferred view
Because #1 happens every day in all sorts of publishing, I think people shouldn’t write letters like this at all but, if they do, it doesn’t amount to undue influence. Any customer-facing job, from retail to journal editor, requires you to basically be on the complaint end for all sorts of things.
#2 also seems like something that we shouldn’t be writing letters about. It’s impossible, barring intimate knowledge, to know why an editor does something but wanting to create a balanced representation of an actual debate seems like exactly what an editor should be doing. That I, or our proverbial letter writer, thinks the work is shoddy is par for the course for academic philosophy. Not undue influence but also a really silly reason to write a letter.
#3 is more interesting and, I take it, is part of what’s at stake with the Byrne affair also. I think this sort of letter can and should be written if there is evidence (even just prima facie evidence) that malfeasance is happening. Is this undue influence? No, I think it’s responsible to write this (so long as the email strikes a professional non-accusatory tone).
You then connect this with two other questions:
- Should people who write such a message to a press be publicly named?
- Should people who write such a message to a press be publicly named, so that bigots and ideologues on hiring and tenure committees can blackball them and so that alt-right keyboard warriors can doxx them?
And here the answer is, to me at least, extremely easy:
- YES if circumstances (like a Freedom of Information request) demand it. Our professional activity can (and sometimes should) be public. The more things happen in invisible backrooms or dms the worse off we are. Transparency is critical.
- NO of course not.
I was thinking of a question more like the following:
Is it undue influence to write to an editor to complain that they are publishing shoddy work that you believe was guided by literal bias against a trait that is the core of your identity?
Regarding your “extremely easy” answers to the last two questions: Items (1) and (2) are inseparable. You can’t have transparency about speakers’ identities in this context without harassment.
We’re not supposed to discuss first-order questions about the concept of gender in this thread. I think the discussion is showing that views on the second-order procedural questions are shaped by views on the first-order substantive questions. People who think there’s a “real debate” about whether gender exists tend to have different views about publication ethics, in this space, from people who think that the “gender-critical” position is off the wall.
Quick note: nobody denies that gender exists. I agree that people’s views on the sex/gender issues are likely playing a role in their views on the publication issues, but the former disagreement isn’t about whether gender exists. I suppose if one mistakenly thought that’s what the sex/gender debate was about, then one might consequently adopt a distorted view of what the publication issue is about.
There most definitely are people who deny that gender exists.
Maybe most “gender critical” academics do not deny that gender exists, in so many words, but to my eye, the conception(s) of gender they affirm differ(s) radically from the conception(s) others are working with.
>the conception(s) of gender they affirm differ(s) radically from the conception(s) others are working with.
This sounds like philosophy to me! In all seriousness, debates about race range from essentialism, constructivism, to nihilism and race is at least as important an aspect of people’s lives as their gender identity.
There’s been fantastic scholarship in Philosophy of Race without the snares we see in Philosophy of Sex/Gender.
There is a rich debate in philosophy of race among a range of reasonable views. Nineteenth century “scientific” views of race are generally considered beyond the pale, because they are both harmful and intellectually indefensible.
If an established philosopher with almost no track record publishing in philosophy of race decided to write a book defending 19th century “race science,” and the finished manuscript got two negative peer reviews, we would not expect the press to give the author an opportunity to revise it. We would not be posting on blogs about “censorship” and “academic freedom.” We would not be speculating about political pressure campaigns and filing FOIA requests. We would be astonished that the manuscript went out for review. It would be obvious that the irregularity, if there was an irregularity, was in letting such a project get past the proposal stage.
The first-order substantive question that is driving a lot of the second-order procedural disagreement is whether “gender critical” positions are reasonable and worth discussing, or whether they are both harmful and obviously indefensible, like 19th century “race science.”
The way to show that gender critical (or any other views of sex/gender) are not reasonable or worth discussing would be to do exactly what was done in response to 19th century race “science” (to continue with your example.) Can you please point to the body of literature that establishes that gender-critical positions are both “harmful and obviously indefensible,” a body of literature analogous to what scholars from many different disciplines have been able to establish in response to 19th century race “science”?
I haven’t been able to find such a body of literature, but perhaps the usual methods I employ when I’m interested in learning about any other topic in the world just aren’t for whatever reason effective when it comes to this one issue.
Natalie Wynn explains better than I can. Start here:
https://www.youtube.com/watch?v=1pTPuoGjQsI
I’m somewhat familiar with Wynn’s views. Irrespective of what I think she gets right or wrong, it strikes me as implausible in the extreme that she’s provided the kind of arguments or body of evidence (on Youtube or anywhere else) that anyone should take as equivalent in force to the arguments and evidence we have refuting 19th century race science.
In any case, we are now well beyond the subject matter constraints Justin set out at the top of this thread.
Mid-Career,
Doesn’t the sheer fact that Gender Critical Feminism is:
A widely held belief among the general publicNot obviously resting on empirically false claimsDefended as vigorously by its proponents as any other philosophical position about which scholars disagreeA position that is held far far more widely among academic philosophers than the “indefensible 19th century race science” you want to analogize it to (https://survey2020.philpeople.org/survey/results/4950)Show us that GCF is, like almost any philosophical position, a substantive philosophical position worth investigating? GCF and their opponents seem to disagree over matters of empirical, normative, and methodological position but this doesn’t make it unusual and the same debates are seen in philosophy of race, free will, even meta-ethics?
This doesn’t look much like 19th century race science (resting as it does on falsifiable assumptions about essentially inherited traits that just happen to match onto our current notions of race) and looks everything like standard practice.
When I read this material I see a lot of stuff that I agree with, a lot that seems implausible, and a lot that seems conceptually confused. Am I speaking of debates in free-will about agent-causation? Whether psychopathy sheds light on motivational internalism or externalism? Whether all of our social/political frameworks that used to tie to natal sex should now be tied to subjectively determined gender identity?
It could any of these! That we want to close off debate about any of these is confusing to me.
Yes, if they had the same conception of gender as others, then they wouldn’t disagree with those others about how to conceive of gender.
I’m mostly not participating in this discussion since I think it has reached diminishing returns, but this is worth commenting on.
In pretty much any circumstances, I have no interest in naming names here. I think there are theoretical exceptions: if, say, members of the Delegates of the University Press (who oversee Oxford), or other very senior academics or leadership figures in Oxford, intervened with the Press, that seems to me worth knowing. But I don’t regard that as likely (naif that I am) and I see absolutely no value in naming people in almost any realistic circumstance. (I cordially dislike the targeting of individuals that characterizes too much online discourse today. Name-and-shame is a horribly uncontrollable and dangerous weapon that I oppose in almost all circumstances.)
As Steven French says, UK law offers robust privacy, and data controllers have to balance that privacy against freedom of information. I expect OUP to do a careful job of that (if indeed they provide any information at all), but in the unlikely event that they end up providing the identities of external actors, I have no intention of making that public in situations short of the extremes I mention above – which, for the avoidance of doubt, would definitely include protecting the identities of academics, junior or otherwise, organized or otherwise, who contacted the Press to pressure them into not publishing something, notwithstanding the fact that I think that behavior would be seriously problematic.
Not a statement of any sort, but I just want to express my appreciation for you doing this. Besides what’s at stake, I appreciate being reminded the public side of our job.
There are two competing claims being tossed around about what is a normal part of the book manuscript review process. Is it a “very common story” with book manuscripts that have already had a proposal accepted to get to the stage of receiving feedback from multiple people, where the nature of those reviews is mixed, and then to have the book rejected without the author’s chance to first make revisions in light of those mixed reviews?
a: Those who are crying foul play think that Byrne should have been given the chance to update his ms as it is a normal part of the book publishing process (considering his initial proposal was accepted and the reviews mixed), and they suggest the best explanation the normal process was interrupted is because of the unpopular content of the ms, which points to censorship.
b: Those who think there was no foul play think the review process was normal (ie it is not uncommon not to be asked to revise the ms in light of mixed reviews) and therefore the rejection was not an act of censorship based on the content of the ms. The rejection was made purely on “academic” reasons, and thus Byrne and company read too much into the rejection.
(There is of course c: that the review process was not normal, there was censorship, and the latter was justified. And d: The review process was not normal, but there still was no censorship or foul play. I’m leaving these to the side for the sake of getting clear on the normal review process question.)
So: was the review process normal (for a book ms whose proposal had already been accepted) or not? We need more and better data on this. And in search of better data, I suggest we do not rely solely on the anecdata of a few high-profile individuals. While that anecdata is valuable in this inquiry, it is hardly sufficient.
I’m not quite sure why the frequency or commonality of rejected books after review is given such a significant role here. We know that it can happen, and has happened before. Now, which percentage would assuage those that are worried about academic freedom? 1%? 10%? .1%?
The reasoning is: the less common this kind of rejection is, the more likely it seems that censorship played a role in the rejection; the more common this kind of rejection is, the less likely it seems that censorship played a role in the rejection.
I also think we should be as specific as possible when discussing the details of the rejection in this case for the sake of clarity and accuracy. This is not simply a book ms that was rejected after review. I’d guess it’s probably common for a book to be rejected after review if the reviews are all or mostly negative. But this is a case of a book ms being rejected with allegedly mixed reviews.
If we were able to provide reliable statistical evidence that, for example, 40-60% of book ms that received mixed reviews were rejected and not given the chance to be revised, then it just would not be rational to defend option a above. There would no longer be much reason to think Byrne’s manuscript rejection was a likely case of censorship.
I’m reasonably confident that even if such evidence could be produced, the discussion would immediately shift to how often books with mixed reviews *that are written by people of similar stature* get rejected. Those who think this is an exceptional case will be able to narrow it down further to maintain that it is exceptional. They’ve already moved on from “this never happens” to “this sometimes happens but”.
I wrote in the other thread, it comes down to the suspicion of foul play resting on the presupposition that the work deserves publication. This won’t be settled without seeing the actual work.
And even then, chances are dim for a consensus, since the “exceptional” side also does not agree with standard evaluation criteria. Witness how Byrne seems to think that a reviewer’s expertise is reason to discount the review (?!).
It is true that anyone who has an ironclad prior conviction that Byrne’s book was good/bad and that there was/wasn’t some impropriety will be unswayed by further evidence. But this is a fact about ironclad prior convictions in general, and normally we don’t evaluate whether evidence would be helpful in terms of whether it could sway people with ironclad prior convictions, since, quite generally, it can’t.
Nonetheless, for everyone else, if they are attempting to form a genuine assessment of what happened then the first thing they should do is try to be relevantly informed. I share C L’s question and find it odd to argue against people who want to be informed trying to become so (?)
Is that actually a significant amount for them? A cursory internet search says their profits are in the area of $100 million a year.
Don’t know the UK FOI. In the US, most (if not all) similar statutes have a limit on the amount that will be expended, above which the costs shift to the requester (who usually has to post a bond covering the estimated amount). So I think the point is that DW will likely have to pay to get the information.
If this ends up breaking anonymity, I (and I presume other precariously employed members of the profession) wouldn’t be able to give honest reviews for the works of people with political power and professional prestige.
I mean, look at the power on display here! Thanks to the efforts of MIT, Pitt, Chicago (etc) tenured people, somehow it is still OUP that is on trial — not the person whose response to a rejection was to go outside the profession and raise a stink while misrepresenting the facts of the matter.
For all this talk about “professional norms”, which norm is THAT conforming to?
“while misrepresenting the facts of the matter.”
How has Byrne misrepresented the facts of the matter?
Justin recounted the events in the post on which we are commenting, c’mon.
As far as I can tell, the only reasons to believe Byrne mischaracterized the facts are:
(a) He was quoted as saying that OUP wouldn’t publish the book “for the sole reason that ‘the book does not treat the subject in sufficiently serious and respectful way'” and that “[n]o errors in the manuscript were identified”;
(b) He recently said there were negative and positive reviews of the ms;
(c) The reviews could include alternate reasons for why OUP wouldn’t publish the book, and could include errors in the ms that reviewers identified.
However, his characterization of the reiviews indicates that none included complaints that could not be addressed in a revised version. He said that one review “contained some clear mistakes, so [he] wasn’t sure what to make of it.” Of another, he said it would have been “[e]asy enough to show that the reviewr’s complaints were all wrong (according to [him]!).” Of the others, he suggests there were no serious issues that would prevent publication, except the “wrong tone” comment. Given the purported book publishing norms, Byrne’s comments are compatible with the view that OUP wouldn’t publish his book for the sole reason…and that no errors were identified (depending on what is meant by ‘error’). The fact that he uses ‘error’, rather than ‘complaint’ or ‘issue’ or some such indicates, to my mind, that he means something stronger, such as error in scholarship.
A charitable interpretation of all the above would allow that Byrne didn’t misrepresent the facts. To settle disputes on the matter, we would need to know contents of the reviews.
To be clear, I am not saying Byrne *intentionally* misrepresented the facts. I agree with Justin that the picture today is a very different one than the one many might have had from Byrne’s original representation of his case.
It is a remarkable piece of apologia to conflate “no errors were identified” with “the author did not agree with the criticism of his work”. I’m not really interested in engaging with this.
On the one hand, we might say that Byrne’s initial description was misleading, because he wrote:
A couple of weeks later, I heard that OUP would not be publishing Trouble with Gender… for the sole reason that ‘the book does not treat the subject in a sufficiently serious and respectful way.’ No errors in the manuscript were identified and… no revisions were allowed.
And that makes it sound like he got virtually no feedback, that the negative feedback he did get contained no criticisms other than the one he shared with us. This turns out not to have been the case (given his later more detailed description of the feedback, after people rallying to support him sought further information from the press), and the question suddenly shifted to whether the criticisms of the manuscript offered, and reasons given against publication in the negative reports were confused criticisms or meritorious reasons not to publish. It seems pretty straightforward that the description here is misleading, in that he appears to be saying those other criticisms don’t exist, and later it comes out that they do.
And I think many of Byrne’s proponents, if they were to reflect on what they thought when reading the initial description, would cop to having thought that this is what Byrne was claiming, or else they might not have said things like:
Because unless Alex Byrne is actually lying (in a way which will clearly be quickly identified and cause significant professional harm to him) then the way his book was handled was radically outside the norms of academic publishing. Byrne’s account implies that he received no detailed peer review commentary…
On the other hand, perhaps it is unfair to call it misleading, since not everyone was misled by Byrne’s initial description.
Another natural interpretation of
for the sole reason that ‘the book does not treat the subject in a sufficiently serious and respectful way.’ No errors in the manuscript were identified and… no revisions were allowed.
is that no errors in the manuscript were identified as a reason for rejection, implying not misrepresentation, but some lack of precision. If it was misrepresentation, why would he later straightforwardly describe the general content of the reviews? In any case, the main question, as I understand it, is not the content of the reviews, or even, whether the book should be published at all, but rather, why was there no opportunity for revision.
Well, I did literally quote one of the most vocal and prolific Byrne-defenders in the previous thread stating what they took to be a clear-cut implication of Byrne’s framing (in fact, it seems that, by entailment, they are accusing him of lying; n.b. I am not).
I think lots of other Byrne defenders had a similar understanding. I’ll point out that I personally didn’t have that understanding, but not because of how Byrne framed his discussion, more because I was operating with the assumption that OUP ran their book decision in the usual way in this case.
Yes, obviously you did literally quote an interpretation by a “Byrne-defender”. Perhaps I didn’t make my point clear, which was to point out an alternative to that interpretation, and to the interpretation offered just above:
It is a remarkable piece of apologia to conflate “no errors were identified” with “the author did not agree with the criticism of his work”.
… and also to the vaguely accusatory implication of “Byrne’s account of the process omitted certain relevant details”, in the OP. None of these are needed.
I agree with all this. I’ll add:
It’s one thing to say that his description misled, which might have been his intent, but might have been due to others’ misunderstanding. It’s another thing to say that he (or his description) incompletely represented the facts, which involves omission but not necessarily inaccuracy or intentional omission. And it’s altogether different to say that Byrne himself misrepresented the facts (intentionally or otherwise), which indicates he inaccurately represented them, and involves a mistake on his part.
What’s most appropriate in this case? Not sure. Because I don’t know much about the person or the situation, my default is to say he incompletely represented the facts, so as to avoid seeming to attack the character of the potentially innocent. For most commentators here, this is likely most appropriate.
The facts were misrepresented. The person who proffered the representation was Byrne. If you want to suggest this was faultless, feel free.
Personally, I am most concerned with the people who clearly operated under the assumption that he was not given referee reports, and have seamlessly transitioned to accept that he was, without a moment’s pause to wonder if they should exercise more caution in, let’s say, parsing his representations.
We have no reason to believe that facts were misrepresented. Byrne described the reason for the rejection (the book was not serious/respectful). Presumably, this reason would have been given by OUP, since it is OUP that gives reasons for rejection (not reviewers, though of course reviewers provide some of the data on which the publisher’s decision might be based). Byrne also said no errors were identified. This is only a misrepresentation if there were, in fact, errors. Were there errors? I don’t know and neither does anyone else aside from the parties directly involved. Perhaps we will find out, thanks to Wallace’s efforts.
If you disagree that “Byrne’s account implies that he received no detailed peer review commentary” you will have to take that up with David Wallace, as he is the one who made that assertion.
Just above you—not Wallace—claimed Byrne “proffered ” a misrepresentation. That’s what I was referring to.
IF it breaks anonymity, sure. But (I sincerely ask) why think that it would? The request itself very explicitly demands that anonymity be retained. Is the thought that the content and writing style of the reports would give people away, or something like that?
Or just simple carelessness in the redaction of personal information, so that a cabal of (no doubt highly motivated) internet sleuths could put it together. Or, worse, *think* they did put it together and proceed to hurl abuse at an innocent bystander.
Is the risk worth it? It sets a remarkable precedent that if you have enough clout, you can get your manuscript rejection re-tried in the court of public opinion.
Thanks for clarifying. Although I think we disagree on other aspects of this controversy, I agree with you that this is a serious risk.
Anyone can make a FOIA request. Prestige is not a requirement. Why think Wallace is setting a precedent?
Isn’t one of the aims of the FOIA request to establish exactly what the facts are so that we can *then* establish whether Byrne misrepresented them? In the absence of the full body of relevant evidence, which we don’t yet have, we simply cannot yet know one way or another whether Byrne mispresented the facts.
Just jumping in here one last time to say that I’m not engaging in a hair-splitting debate on the differences between misrepresentation, incomplete representation, omission, imprecision, merely implicated falsities, or whether more information is still needed … with the same people who eagerly jumped to conclusions about OUP’s alleged misconduct without any similar concern, taking the most damning interpretation of what Byrne said at face value, and now are trying to backpedal.
There are now facts on the table that everyone agrees on and I am happy to let these facts stand on their own.
These facts, as they stand, are evidence for the same conclusion I’ve been advocating from the start: it was wrong to be credulous about Byrne’s claims.
I don’t care whether this was due to omission, insinuation, imprecision, or falsification. I don’t care whether this was intentional, an honest mistake, or due to conspiratorial thinking. It was wrong to be credulous. You can keep your apologia.
I hope this isn’t directed to me. If you think I’m backpedaling or was credulous, you’d be incorrect. Go back and look at my comments.
A person who participates in what gets called “gender critical theory,” which means theorizing about gender in a way which insists on a gender binary that is mostly biologically determined, and argues against the inclusion of trans women into the category of women (and trans men into the category of men, and so on.)
I think the way that they would put it would be to say that they are critical of the whole *concept* of gender, and not insisting on a *gender* binary, but rather on a *sex* binary, and insisting that since gender is a confused concept, the only way to interpret “man” and “woman” is with a mostly biologically determined category. Since there is basically no gender category, and sex is basically biologically immutable, they deny that there are many meaningful ways to be trans.
Even that may be a bit strong. I take that many (at least) GC feminists don’t deny the phenomenon of being transgender as a meaningful (even valuable) social category, they just deny that being a trans-woman is a meaningful way of being a woman (and similarly trans-man/man).
I take it that a GC feminist becomes a gender *critter* just when s/he also happens to be a raccoon or opossum or gopher, or suchlike, which is what I take Moti’s implicit raised eyebrow to be getting at.
Oh, I definitely know what “gender critical” is. I’m sympathetic to the view. I guess I had just never seen proponents referred to as “gender critters.”
The tweet was about an article for an edited volume, which is separate from the book manuscript being discussed here.
Perhaps the article is not the *main* topic of discussion here, but it does pertain to the FOIA request (see point 2 of the request, quoted in the post above). At any rate, I recognize that this isn’t the main topic—hence, my parenthetical—and was only sharing my disappointment that this (certainly still relevant) issue has somehow been lost in the shuffle.
I would like it if referee reports were always public (though not de-anonymized). Perhaps referees would be more likely to take the job seriously and think twice before writing a cursory, uncharitable, thoughtless, nasty review if they knew the report was going to be publicly available.
The journal Philosophies allows reviewers to indicate that they are willing to have their reviews published along with the paper (if/when the paper is published), with or without their name. Here’s an example – you can click the “review reports” button to see the referee reports, one of which is anonymous and two of which have names attached. I think this is pretty neat.
Give Noah the Nobel, I say. Someone who is not me should collect data and write a paper on the philosophy and psychology of “cursory, uncharitable, thoughtless, nasty review[s].” Some of y’all have issues.
Edit: I just realized it. It’s hazing. That’s what it is. Unwarranted cruelty based on survivorship bias mixed with pedantic notions of disciplinary expertise. Toxic AF if you ask me.
I’ll say that I’m one of the people who would be very upset if my correspondence with editors would be made public. We are sometimes sent manuscripts that have real problems and discussing those problems frankly and openly with editors given the understanding that they will filter the comments while removing any clue of the referee’s identity is an important part of this. A few years ago, I was asked to referee a manuscript that, in my opinion, was hastily cobbled together, poorly argued, and seemed to involve non-trivial amounts of plagiarism. Even if the report were ‘scrubbed’, it would probably not be a surprise to the author(s) who wrote this report. It was not an easy conversation to have, particularly because mine is a small subfield and I have a somewhat friendly and collaborative relationship with the author(s). Having some discussions with the editor with the understanding that they would pass on the reasons they deemed relevant and take steps to hide my identity was key to being comfortable doing an honest review of the work without fear of damaging a relationship. In the book review process, authors are often given the opportunity to defend their work with the editor serving as a mediator. My negative review is taken under advisement, authors can respond, and I have no desire to volunteer labor and give an honest opinion if I can’t trust that my contributions will not clue author(s) to my real identity. Why make enemies or ruin friendships for free? I could spend that time with family instead.
I’m pleased that Wallace has not asked for the identities of the authors, but I don’t know that the next person will show this discretion. Moreover, our reports or correspondence might betray our identities in ways that won’t be scrubbed if these materials are made public. I suppose I could have taken steps to eliminate clues. I also could have refrained from providing evidence of plagiarism. I didn’t. And now I feel exposed because I naively thought that confidential correspondence would be confidential.
I would have thought that, after all the discussion of ChatGPT and related technologies that use word embeddings and transformers, it would have been obvious to all that this business about “excluding the names and other identifying information of any external reviewer used by OUP and of any employee of OUP without editorial authority” is a canard. Deanonymizing prose, especially when you’ve got pages and pages of it, is not that hard. It’s especially easy when the universe of possible authors is very small, as it would be in this case (i.e., only academic philosophers, or even just a subset of them). Run a google scholar search for “authorship attribution” and you’ll find a bunch of recent work in this area. If you want to name and shame people, just admit it.
“If you want to name and shame people, just admit it.”
It’s one thing to raise an objection that David Wallace and others who have proposed the release of anonymized discussions might have overlooked the possibility that the identities of the anonymized people might become known.
But how did you reach the conclusion that Wallace and so many others here *want* this to happen (and, more than that, that they will themselves ‘name and shame’ people), and that they should now take the further step of *admitting* it? That’s quite an accusation, but I don’t see the basis.
Moreover, you accuse Wallace and others who agree with his approach here of overlooking something ‘obvious’ that people should surely know, given the recent development of ChatGPT and related technologies. I’m trying to connect the dots and understand the relation. Are you saying that ChatGPT makes it _easier_ for us to identify the writer of anonymized text? You seem to imply that, but I can’t yet see how ChatGPT would help. I’ve done a search for ChatGPT and de-anonymization and the first pages of results just seem to be the problem of information used to _train_ ChatGPT not having been properly anonymized. But I haven’t yet seen anything on how ChatGPT could be used to detect the author of anonymized writing.
Perhaps you have seen some discussions that my cursory glance has missed: If so, I’d be interested in learning more about it and would be grateful if you’d give us a clearer indication of what you’re talking about. How exactly would you use ChatGPT or related technology to figure out who had written an anonymized referree report?
You briefly mention word embeddings and transformers without explaining exactly what process one would use to work out who had written something. I’m admittedly far from a technological expert, and could easily be missing something big, but I just don’t see the idea yet. Transformers, as I understand them, serve to generate new text, not analyze existing text for purposes like authorship attribution. The word embeddings technique *is* (as I understand it) the main technique used for identifying likely authors of a piece of anonymized text, but I don’t yet see how Chat-GPT and related technology could make that process easier. Perhaps you could explain?
If anything, what you say suggests to me a way in which de-anonymizing things like internal reviews could be made more *difficult*. The word embeddings technique, as I understand it, involves looking for patterns in using specific words or parts of speech in the anonymized text and matching those with similar patterns in the known writing of a given author. I don’t know how ChatGPT would help with that, but I _can_ see how a reviewer who wishes to remain unknown even after an anonymized FOI search becomes public could use it to advatage. Before submitting the review, you simply put it into ChatGPT and ask it to reword the whole thing in a generic way. What ChatGPT produces would then presumably make all the same points you made in your review, without leaving behind identifying word and part-of-speech patterns, covering up (it seems) your footprints so as to foil others’ attempt to learn your identity through word embeddings analysis.
But this would make it _more_ difficult, not less, to figure out who wrote reviews as a prelude to attacking them, so it seems far from the ‘obvioius’ problem you have in mind.
Could you please explain, for those of us who might not know as much as you do about ChatGPT and related technology, what these ‘obvious’ uses of that technology are for performing word embeddings analysis, and also how you came to the conclusion that Wallace and others here _intend_ to expose reviewers to the punishment of naming and shaming, but have just not admitted it? Thank you.
Yeah, sorry, I was moving too fast. I mentioned ChatGPT because it uses BERT to create word embeddings, and everybody is now aware of ChatGPT (I assume very few are aware of BERT). BERT stands for “Bidirectional Encoder Representations from Transformers.” Basically, it’s a way of encoding not only the presence of a word/token in a corpus, but also it’s position relative to other words/tokens that come both before it and after it (thus the ‘bidirectional’). From what I can tell, all of the bleeding-edge approaches to authorship attribution use BERT. It would be pretty straightforward to identify an “anonymized” philosopher using these methods based on the prose they submitted to OUP, even if obvious stuff like names and email addresses and institutional affiliations were redacted.
Authorship attribution has been around much longer than BERT, as I’m sure you know. We can often guess with some confidence who are reviewers are just from the criticisms they do and don’t offer. In a relatively small subfield of philosophy, that would likely be enough to at least narrow things down to a list of likely suspects. Throw in advances in NLP that everyone is or should be aware of, and the prospect of genuine anonymization is essentially zilch. Wallace has mentioned a few times recently that he knows a lot of math, e.g., through this work in philosophy of physics. This should not come as a surprise to him.
I’m curious whether you have tested this. E.g., have you written a review of a journal article, and then used BERT to see if the article author would be able to identify you as the review author?
If you know the author is, say, Mark Alfano, Jamie Dreier, or Justin Kalef, the programs can do a good (but not perfect) job (but you have to give them a lot of accurate samples first — a few hundred thousand words maybe). If you know the author is Somebody in Philosophy, not so good.
So maybe be a little circumspect in the Naming and Shaming that Mark Alfano anticipates.
“If you know the author is Somebody in Philosophy, not so good.”
That would have been my assumption, but I’m happy to be educated on this stuff.
Buried amongst the comments bitter
The author of this apophthegm
(Herein ycleped “SCM”):
“The Stygian Caves of philosophy Twitter”
Has plainly earned entitlement
To any such small enlightenment.
And come now knowledge, fair and free
Delivered here new providentially,
By a sage, though one, styled as three,
An honoured unholy trinity.
This is totally irrelevant to the point actually at issue, but you’re not quite right on the technical details here. ChatGPT does not use BERT. BERT is a family of models that involve transformers, the same sort of architecture as GPT models involve, but they’re distinct.
GPT and BERT have different training objectives, and the way the transformer blocks are allowed to direct attention during training are different. GPT is trained on next word prediction — you give it a bunch of tokens, and it generates a probability distribution for the word that comes after them. BERT is trained on *masked* word prediction — you give it a bunch of tokens, with one of them masked out, and it generates a probability distribution for the masked word, which isn’t necessarily at the end of the sequence (it’s trained on another task as well but whatever). Additionally, from the BERT release paper: “Critically, however, the BERT Transformer uses bidirectional self-attention, while the GPT Transformer uses constrained self-attention where every token can only attend to context to its left.” Also, besides that, BERT just is a model (you can go download it), and that model is not part of GPT.
Anyway, GPT doesn’t have bidirectional encoding. As you probably know, BERT models are usually used for tasks that involve sentiment classification or identifying relations between sentences, where the objective is to get some kind of information about a sentence — is it positive or negative, is it written by the same guy as these other sentences, etc. And as you definitely know, GPT models are used for generative tasks.
Everything substantive I might want to say here has been said by others, but I did want to comment on how touched I am by the idea that, since I am competent in various areas of contemporary theoretical physics, I must therefore be expert in every mathematically-informed discipline in the world. My imposter syndrome is banished forever.
Even if the technology were capable of identifying authorship in referee reports, asking that identifying information be excluded, as Wallace has done, suggests that naming and shaming is not one of the objectives. Unless we think David Wallace, who has said he does not know Byrne or Lawford-Smith personally and does not work on these sex/gender issues himself, cynically made the deidentification request to obscure his nefarious naming-and-shaming motive, knowing and expecting that he or others would apply the LLM technologies to identify the authors.
If what Mark says is true, then it is big news. The implication that the identities of the relevant people in the FOI request would be inevitably revealed through this technology, if the request is granted, is the little story here. The big story here is that anonymous peer review is dead. Once it becomes widely known that any author can, within just a few minutes, use this technology to reveal the identity of the anonymous reviewers of their paper everything will change. Many people would just refuse to do reviews anymore. Almost every junior person would never write a negative review of a paper as they would fear that the author could easily reveal their identity and then hold a grudge against them, which could effect their career progression. Even many tenured scholars would hold back on writing negative reviews because they would prefer that there are not people who hold grudges against them for such a thing.
Mark did not offer any solid evidence that what he claims is true. If he does have such evidence, then Justin should be inviting him to write a guest post here about how authors can now deanonymize their reviewers. This is a very important issue for the profession to discuss and is entirely separate from the debate about the alleged censoring of GC philosophers. Of course, it may be that Mark overreached in his eagerness to score a rhetorical point in the GC debate, and he doesn’t in fact have evidence backing up his view. But if he does have such evidence then we certainly need to be discussing this a lot more.
Perhaps those commenting on this use of the UK’s FOI Act (2000) might care to consider its provisions & in particular the exemptions:
https://www.legislation.gov.uk/ukpga/2000/36/contents
(There’s a slightly different act that covers Scotland because Things Are Different There.)
With regard to anonymity, the Information Commissioner’s Office notes that ‘ The UK GDPR and the DPA 2018 [UK General Data Protection Regulation and the Data Protection Act 2018 respectively] exist to protect people’s right to privacy, whereas the Freedom of Information Act is about getting rid of unnecessary secrecy. These two aims are not necessarily incompatible but there can be a tension between them, and applying them sometimes requires careful judgement.’
I would expect OUP to exercise such careful judgment.
Finally, the Act only applies to public authorities – OUP counts as such because it’s a department of the University of Oxford. Other publishers are not such authorities.
Maybe this issue is another good reason to move towards more of a post-publication review system. Only use peer review to suggest improvements and make sure it’s not some crackpot and then allow the status of the article be decided by peer rating after the fact.
We certainly have the technology, it circumvents both the problem of inappropriate gatekeeping and people getting insufficient credit for groundbreaking work because the referees at the time couldn’t appreciate it.
And in this model who pays for copy-editing, typesetting, making sure the references are correct, maintaining the website so links don’t expire, making sure the file format is still accessible as software changes, coordinating with index makers so the file is accessible via search, etc? What ‘technology’ do we have that solves these problems – spell checkers?
In the case that started all this, there is also the small matter of printing the work on paper, binding it, and physically distributing it. But in most cases that’s a small expense compared to the ones I listed above.
Unless you want academic publishing to be at the quality and accessibility of blogs from 20 years ago, there are substantial costs involved. And when someone (like Thought) suggests publication fees sufficient to cover those costs, we get a different set of complaints.
Uploading a PDF onto a cloud service and linking it to your homepage, which is presumably searchable and/or linked to your university profile, is free. It’s free for you, it’s free for your readers, and it’s certainly freer, as a medium, than publishing with a press or a journal. The cloud, your homepage, and your university website all maintain themselves through persons and services that, fortunately in this case, you’ll never have to think twice about.
Spelling and grammar checking software (yes, you don’t need more) is free. Formatting software is free. PhilPapers is free. We frequently read and review our peers’ papers, for free. You, as a reader, are free to, and certainly should, check an author’s references when that seems important or relevant. Someone can print your paper if they’d like a hard copy, probably for free.
The arguments against this level of freedom that I have seen rely almost exclusively upon career incentives (which could be changed so that our activities are less contingent upon a predatory industry making millions off of our already free, uncompensated labor) and an unexamined belief in that same predatory industry producing, in its current form, the best possible, highest quality scholarship. I’d like to think I can trust my peers and the public to make that assessment, were my work easily accessible and unproblematically available and open to comment and discussion. It just seems like such a shame to eschew all of these wonderful technologies in favor of platforms that require our employers and laypeople to pay hundreds of dollars that we, the writers, never see, and that are an ever-increasing financial drain on our institutions.
Perhaps, on this model, we wouldn’t even be having the discussion that occasioned this thread!
Yes pdfs from personal sites 25 years ago are all readily accessible now this definitely works as an archiving strategy.
Of course, the need to get stuff published *at all* makes one put a lot more effort into it. And response to commentary by others — commentary produced in a responsible, institutionalized manner — improves the work. (Absent the proviso, improvement is less likely — with social media providing a nice “natural experiment’.)
Wallace made these requests in light of reports from Alex Byrne about how a book manuscript and a chapter for an edited collection were handled by OUP. (Note that the request explicitly asks that the names and other identifying information of reviewers and OUP employees without editorial authority be excluded from the produced material.)
Byrne’s initial description of the process, reported on here, indeed made it seem odd. In particular, Byrne appeared to imply that the manuscript for his book, for which he had a contract with OUP (title: The Trouble with Gender), was not properly reviewed. The point of the Quillette article in which he provided his side of the story is that he believes that the discipline of philosophy is heavily biased against views on gender like his and treats those who deviate from what he sees as political orthodoxy on transgender issues so badly that many are deterred from writing about them; the improper rejection of his book was offered as evidence of this. Byrne wrote:
A couple of weeks later, I heard that OUP would not be publishing Trouble with Gender… for the sole reason that ‘the book does not treat the subject in a sufficiently serious and respectful way.’ No errors in the manuscript were identified and… no revisions were allowed.
It has subsequently come to light that Byrne’s account of the process omitted certain relevant details. In an email to David Wallace, OUP philosophy editor Peter Momtchiloff wrote the following:
You will understand that I can’t discuss the details of a specific review process, as this is a confidential matter. I can confirm, however, that the decision not to proceed with Alex’s book manuscript was editorial in nature. By way of clarification, let me assure you that the manuscript underwent peer review, and Alex was given comments on the manuscript from me and from four expert readers. On the basis of these assessments, my judgement was indeed that the work was not appropriate for publication by OUP. And Alex’s submission for the Handbook also went through a process of review by expert readers.
This makes it seem as if Byrne received more back from OUP than just “the book does not treat the subject in a sufficiently serious and respectful way,” possibly including comments that elaborated on this claim or pointed to evidence for it, as well as other issues with the manuscript. (Although one should note that failure to treat a subject in “a sufficiently serious and respectful way” seems like sufficient grounds for rejection.)
In a comment responding to this new information becoming public and requests for him to share the reviewers’ reports, Byrne wrote:
The expert readers told me various things. Nonetheless, OUP’s stated reason for rejecting my book without allowing me to respond to the reports was exactly what I said it was.
As to the possibility of sharing the comments, I do not think that readers’ reports solicited (and paid for) by a publisher are mine to share. Moreover, one reviewer simply supplied a pdf of the manuscript marked up with some (minor) helpful and constructive comments. Peter did likewise. (Most of those were incorporated in the version that will appear in print.) Another report consisted of Peter’s notes from a discussion with a reviewer. I have nothing to hide (apart from the manuscript, since I want you all to buy my book, if only to commit it to the flames). OUP has my permission to release whatever comments it likes.