The Different Ways Philosophy Journals May Be Good Journals
Who is the best philosopher? What is the best philosophical idea? What’s the best philosophy book ever written? These are, to put it politely, not the best questions.
We all know this. There are so many distinct properties that quite reasonably factor into assessments of philosophers, ideas, and books, and so many reasonable ways to weight these factors, and so many reasonable ways to specify “best” (best at what? best for what?), that any attempt to identify the unqualified “best” philosopher, idea, or book will be an exercise in equivocation and confusion.
What about this question: which is the best philosophy journal? This seems to me of a piece with preceeding ones, yet while most academic philosophers would find those other questions embarrassingly naive, and reluctant to provide answers to them without various qualifiers and cautions, it seems many do take this question about journals seriously.
For example, when Brian Leiter (Chicago) asks the readers of his blog which journals are the best (here, for example), several hundred people, presumably many of them philosophers, proceed to rank them in a survey whose results he posts. I think the most charitable interpretation of what respondents to surveys like this take themselves to be doing is answering the question, “which journals contain a higher proportion of articles I judge to be of high quality”? (Some may be instead answering the question, “which journals contain a higher proportion of articles I judge that most others would judge to be of high quality?”, but I’ll leave aside that possibility and its complications.) If that’s right, the survey results tell us in the aggregate what the respondents think is the likelihood that an article published in a journal will be of high quality; the higher the journal appears in the results, the higher the likelihood an article in it will be one the respondents would judge to be high quality.
Such surveys may be informative. (Perhaps similar information might be gleaned from looking at journal acceptance rates.) However, since the respondents of such surveys don’t specify the criteria by which they’re assessing the articles appearing in the journals, since we don’t know who is choosing to answer such surveys and hence are limited in our ability to guess their criteria, since aggregation can smooth out interesting details, and (to depart for a moment from the most charitable interpretation of respondent behavior) since such surveys can be self-reinforcing in ways that affect how people answer them, they are limited.
It might be helpful, then, to be more explicit about the criteria used to judge philosophy journals. Now of course there are different aspects of philosophy journals to judge (e.g., how well they manage the editorial experience from the author’s point of view, the extent to which their content is accessible, the typesetting), but let’s focus on their content.
That is, when you think about the quality of the works that journals publish in order to form judgments of philosophy journals, what are you thinking of? When you make a judgment about how good a philosophy journal is, what are you paying attention to in regards to the articles that journal publishes?
I’m hoping that the responses to this post will generate criteria and vocabulary by which to make more and more varied assessments of journal quality (perhaps to be used in surveys or other informational resources). A richer, pluralistic, and more transparent understanding of journal quality may be useful to prospective authors who are thinking about where to send their work and to readers and researchers who might be exposed to philosophical work they otherwise would have overlooked, and might possibly reduce backlogs at certain journals currently near the top of monolithic rankings.
(This post comes about as a result of a conversation on social media started by Quill Kukla.)
I think that it’s all about prestige. I don’t think that it’s “which journals contain a higher proportion of articles I judge to be of high quality.” For instance, because of my area of specialization, I don’t find that high of a proportion of articles that I judge to be of high quality in Phil Review. Indeed, I’m much more likely to find such an article in Ethics. But I still rank Phil Review higher than Ethics. And if you think about it, it couldn’t be something like proportion of high-quality articles. Because, I take it, no one reads all the articles in some generalist journal like Phil Review and then assesses what proportion of them are of high quality. What we do instead is think of all the tenure and promotion letters that we’ve read, all the hiring decisions that we’ve been a part of, and all the praise that we’ve known people to get for publishing in various journals. And then we’ll rank as highest those journals that are such that publishing in them most helps one’s chances of being hired or promoted and/or of gaining prestige in the profession. And in all the letters that I’ve read and in all the hiring decisions that I’ve been a part of, having a publication in Phil Review seems to help your chances of being hired or promoted more than any other journal. So that’s the one that I rank highest. And this is despite the fact that I actually don’t read much in that journal, nor do I judge that the average quality of the articles that I do read in that journal to be of higher quality than those that I read in other specialized journals. What’s more, it seems to me that Phil Review’s prestige has just as much to do with how hard it is (and what a pain it is) to get published in it as well as its history in the profession as it does with the quality of its articles. Of course, I’m not denying that it publishes excellent articles. I’m merely pointing out that something like prestige depends on more than just present quality. Prestige is as much a matter of perception as it is of reality.
I was basically going to say what Doug said. I think typically when people ask the unqualified “best journal” question they mean something like which journal is such that the relevant people in the profession would be most impressed by publishing there. And like Doug I don’t think such a judgment best interpreted as a summary of our understanding of the excellence of the papers we have read in that journal. Very few journals are such that I would feel justified in hazarding a view about the average excellence of papers published there, for a lot of reasons.
Here are a few ways of judging the quality of a journal that come to mind:
(1) has it published a large number of papers I find myself mining for new ideas again and again? Does it publish such papers with some regularity?
(2) same as (1) but with citations instead of idea-mining.
(3) same again, but this time with respect to starting ongoing conversations I participate in.
I think when people respond to those surveys, their main criteria is probably perceived reputation of the journal, where “perceived reputation” probably has some connection to (at least past) quality, but mostly just tracks which journals it would be most beneficial to publish in for CV purposes. This is particularly true for the ranking of the generalist journals. I don’t see how anyone could rank ~20 generalist journals based on the quality of work they’ve been publishing in, say, the last five years. Surely no one reads everything published in the last few years in each issue of the 20 or so journals that these surveys track and then arrives at a judgment of the quality of the journals on that basis (just think how much reading you’d have to do to read everything published in the last five years in Phil Studies alone!). And how many people would be qualified to judge the quality of all the work that appears in the generalist journals anyway? At best, people might generate an impression of a journal based on the quality of work it publishes in that person’s area of research. But that 1) bases the ranking on an under-representative sample and 2) surely doesn’t have very broad coverage for most people. So I think perceived reputation probably explains a lot of the rankings, and that’s also probably why the rankings don’t shift all that much over time. (Of course, perceived reputation can lead to more good work being published in the top journals. If Journal X is perceived to be one of the best journals, people will send their best work there. But I also think that a fair amount of “trendy” work often gets published in top journals, and I don’t mean “trendy” as a compliment).
As for the main question, I’m not really sure there’s anything general that we can say. Despite what I said in my last paragraph, I think it’s probably good to have a sense of the reputation of the journals in the field since (for better or worse) that information can help with career advancement. So that’s one dimension along which we can try to rank journals. It’s also good to have a sense of which journals have good editorial practices (which ones are quick with decisions, which ones desk reject a lot vs. send papers to reviewers a lot, etc.). As far as research goes, at least in my own case, I don’t really pay attention to which journal publishes a person’s work. I just care whether it is relevant to my own or not. Rankings of journals don’t influence the way I conduct my research at all.
And now I see that Doug said basically what I was saying. I agree with Doug.
When journals are ranked using the methodology that Leiter uses — asking professional philosophers to offer their own ranking — I think it is extremely likely that the respondents are not judging journals by quality initially. Instead, I expect that they are first judging journals according to how relevant those journals are to their own research interests. (Considerations of quality follow after the initial judgment of relevance.)
It is striking to me that the topic areas of the top-20 journals are fairly narrow. Why is that? Is it really because the qualitatively “best” articles just happen to be published on topics like mind, epistemology, and metaphysics? I doubt it. Rather, those topics are heavily represented in the top-20 journals just because the people who do the rankings really like those topics. I know that the top-20 journals are largely supposed to be “generalist” journals. But in practice, they are not.
Further, I think the reliance on rankings like these is extremely dangerous for our profession. As long as we keep taking these rankings seriously, then we will never really make significant changes in the way our profession works. There are excellent scholars working on topics that fall outside of the narrow interests of the top-20 journals. Yet, those scholars struggle in the job market because they don’t have many top-20-journal publications.
If we have to rank journals at all, then maybe journals should be ranked by citation rate instead? I know that method isn’t perfect either, but at least it would indicate which journals publish articles that are worth talking about, and possibly it would expand the range of topics that fall into the top-20.
I don’t think any of your points are very fair. First, I think there are plenty of papers by great scholars on off-the-beaten path stuff–C. Thi Nuygen recently published a paper on games in Phil Review! Second, there’s a lot more mind, epistemology, and metaphysics than, say, aesthetics in top journals, because way more people work on those things. Now you might think that’s bad, but it’s kind of silly to complain that generalist journals mainly publish the kind of stuff that tons of people work on (I say this as someone who works in none of the areas you mentioned.) Third, hiring committees for really niche AOSs, like, say, ancient, don’t care about whether your work appears in Phil Review so long as you publish in a good specialist journal (e.g., Apeiron). So, I don’t think this hurts anyone’s job prospects.
oops I guess the Nguyen paper was in Phil Imprint, but still plenty of great scholars doing non-traditional stuff do get published in predtigous journals
He published in a lot of the top journals! The well known games paper is in Phil Review.
Are my points fair? I don’t know. My basic point is that there is a general trend at the top journals that favors some topics over others. Do your points demonstrate that my claim is false? I don’t think so.
First, Thi Nguyen published an essay about games in Phil Review. So? Does that prove that there is no general trend? No. It could be an outlier. Outliers don’t disprove trends.
Second, you suggest that there are more mind, epistemology, and metaphysics essays in top journals because there are more people working on those topics. Okay. But, isn’t it also possible that there are more people working on those topics because they are topics with more venues for publication? (You can think of this as the Euthyphro dilemma of publishing!) My point is that there is pressure within the profession to favor some topics over others, which you can see manifested in many different ways. I remember in grad school that the program directors would often discourage people from studying aesthetics because “you won’t get a job”. I’m sure the same thing happens for people who want to study feminist philosophy and philosophy of race.
Finally, is it true that hiring committees don’t care if a specialist in ancient publishes in the top journals? I think that is an overstatement. Hiring committees don’t require an ancient scholar to publish in top journals. But, if a specialist in ancient publishes in a top journal, of course the hiring committee will notice and likely favor that candidate. Additionally, ancient isn’t a niche topic in the same way that (e.g.) aesthetics is. Most departments have someone in the faculty who specializes in ancient because many think it important that students are exposed to ancient. The same can’t be said for aesthetics (or feminist philosophy, or philosophy of race).
My basic point is that there is a narrowness at the “top” of the profession in publishing. I want to think of ways of broadening our profession. And I suspect that journal rankings contribute to that narrowness.
This seems to put the cart before the horse. There just *are* few jobs in aesthetics and business is booming for feminist philosophy and philosophy of race. The latter is not explained by journal publications but by larger socio-political factors (i.e. it is apparent in other departments), and publications (both in prestige and number) follow the jobs.
Why aesthetics is relatively unpopular/unprestigious is another question (which I doubt is explained primarily by publication venues).
Bioethics and other applied ethics are I think a better example of what Christian has in mind than aesthetics (please correct me if I’m wrong here!). There are a lot more jobs in these fields than there are the most trendy bits of LEMM, there are more people working in them than there are LEMM, and they have a lot more effect outside of our specialty and yet bioethics journals rank very low in Leiter’s [prestige] measuring contests in the same way that bioethics as a specialty does. Take the ethics journals survey. No one who knows the first thing about applied ethics or bioethics thinks that Social Theory and Practice, Public Affairs Quarterly, Journal of Value Inquiry, or Res Publica are better journals than American Journal of Bioethics or the Hastings Center Report. That’s not a knock to the first group of journals, I’ve published in some of them and I think some really excellent work has appeared in all of them (For instance, Robillard and Strawser’s “The Moral Exploitation of Soldiers” which came out in PAQ was one of the best articles I read last year). The point remains that they are not Hastings or AJOB. But Leiter has a huge and pretty open bias against bioethics and applied ethics on top of the bias that runs pretty strong in the more “elite” corners of our profession. So these journals score really low in surveys he runs in the same way that I imagine that any liberal social policy would in a poll Tucker Carlson ran of his viewers. And to turn around Caleb’s point how many bioethics articles have appeared in Phil Review in the past ten years? I gave up after I couldn’t find any in the past three. Compare that with not just LEMM but even some trendy narrow sub-topic in LEMM like grounding, vagueness, or the nature of colors.
FWIW, the ASA has around 700 members (and four divisional plus one annual meeting each year) making it one of the largest (and most active!) philosophical associations. (There are also Canadian, British, and Europeans societies, and the membership doesn’t all overlap). Aesthetics is not nearly as small as people sometimes think it is. That’s not to say there aren’t more people publishing in mind or metaphysics; it’s just to say that I don’t think our intuitions about the relative proportion of active researchers in these fields is very accurate and, thus, neither are our intuitions about their representation in generalist journals
Also, game stuff is not really ‘off the beaten path’: it’s been a hot topic for several years, thanks to work by Thi and others. It’s unusual to see that sort of thing in PhilReview or PhilImprint, sure, but it’s unusual to see *any* aesthetics there. Similarly, you sometimes see some grounding in the pages of the BJA, but not often!
Right! And I just want to add, Thi’s work is really groundbreaking. He’s published a lot lately in some of the top journals. Deservedly so, because his work is brilliant.
But, “groundbreaking” is a really high bar. I wonder if non-central topics get published in top journals when they are groundbreaking, while essays on more central topics don’t have to meet that same high bar.
I’m also willing to admit that my evidence for my worry is pretty flimsy. It is mainly based on years of talking to people working in non-central fields who seem to have similar experiences. I would be really interested to look at submission data and seeing if various topics across the board really get published at proportional rates, or whether some get published more frequently than others.
The criteria people use to rank journals are like a black box, but it doesn’t mean the aggregate rankings do not track something meaningful. I doubt it’s objective excellence or high quality per se. But it does track, at least for the highest ranked generalist and specialist journals, (i) reputation as observed on the market and through word of mouth (metrics that matter for better or worse), as well as some citation practices.
Personally, I use Leiter’s rankings as a heuristic, because they reflect things that peers care about (again, for better or worse) and because they partially match my arguably unreliable perception of where the papers that people tend to talk about tend to be published. But that’s not my only heuristic. I also care a lot about how the journal is run, as far as I know—its editorial process, turnaround times, and the quality of referee reports. It’s hard to get solid intel on all that, but there are those journals that we all know are poorly run and those that are models to emulate.
Since philosophy is so very broad, it’s not possible to say which journal is the best but only (and maybe) which is the best in which area or field. Americans and Brits are likely to consider primarily journals in the “analytic tradition,” but that’s too narrow. So you might ask your question for different categories of philosophy, not philosophy as a whole – whatever that is. Are Kierkegaard and Nietzsche philosophers? Rorty? Frege? The list is endless.
“That is, when you think about the quality of the works that journals publish in order to form judgments of philosophy journals, what are you thinking of? When you make a judgment about how good a philosophy journal is, what are you paying attention to in regards to the articles that journal publishes?”
I have a sampling bias worry here. If the question is “what do you think of the quality of the works that a journal publishes”, the evidence set I’m bringing to bear on the question are just the subset of works by that journal of which I’m familiar. And same for everyone else.
But papers tend to follow a “super star” or scale-free (ish) distribution, which means the vast majority of people will know about the most well-known papers, and very few will know the less well-known papers. So, for the journals with just a few super star papers, those papers will have a disproporionate effect on the aggregate assessment of that journal’s quality. If (modestly) super star papers tend to be “good” compared to the lot, I would think those journals would then rise to the top comparatively, just because (plausibly) they got lucky with a few papers, saying nothing about the rest that they publish.
I think to some extent this is counterbalanced in people’s priors when they know a journal to publish “a lot” vs. “a little” compared to the average journal in a reference class, and also when they know a journal to be “permissive” or “selective” compared to the same. But I don’t know how much this actually helps “fix” the bias.
One thing that I find is that in *most* cases, I don’t really even think about what journal a paper is in. When writing a paper, I’m trying to write the best version of that paper, and when I submit it to a journal, I often don’t have a clear sense of any reason I should choose one journal rather than another, other than general reasons of prestige, and a few mechanical things like word count limits and open access policies. As a result, when a paper is rejected, it’s very easy to only slightly tweak it and send it to another journal. When I referee a paper, I will on rare occasions care about what journal I am refereeing the paper for, but usually, my comments will be about what I think make this paper a good or bad one for its intended audience, regardless of the journal that asked me to referee it. For both of these reasons, it’s hard for journals to be very distinctive from each other in ways other than prestige.
However, there are some journals that manage to be distinctive. Obviously, there are journals with particular topical scope, like Ethics, or the Journal of Philosophical Logic, or Hypatia. There are some journals that manage to turn a word count constraint into a stylistic feature, like Analysis and Thought (and on the other hand, Phil Review and Phil Imprint, for being willing to publish really long papers).
Are there other ways a journal can be distinctive, such that authors will still publish there? If you write something short, and it gets rejected from Analysis and Thought, or write something long, and it gets rejected from Phil Review and Phil Imprint, what do you do?
How often does anyone write something with a particular journal in mind, because of a stylistic or structural feature? Or conversely, how often does someone find the refereeing and publication process at a journal to shape their paper, so that the final product is essentially associated with that journal, rather than just accidentally so?
Following up on your first point, I do wonder whether the focus on journal prestige is kind of a hold over from before the internet generally and PhilPapers specifically, when one would have to go to the library to browse physical copies of journals to find things to read. In such a context, and given limited time and resources, it makes sense to think a lot more about which journals are going to publish better articles and which less so.
I agree, as you said, that when I find an article with an interesting abstract, I am going to read it basically on the condition that I’ve *heard* of the journal. Maybe people are still scoping out the new issues of journals like Phil Review and Ethics just to see what’s being published, but my completely anecdotal guess is that that is largely generational.
A good journal publishes a diversity of views on a topic with a minimum criteria of coherence and clarity. It is also adapted to modern technology like the internet meaning it doesn’t have word limits. Oh wait this rules out almost all current philosophy journals. Wonder why the field is so dead and boring.
Yeah, philosophy would be much less boring if only there were a lot more reeeaaaalllllllllly long papers!
That you think long papers guarantees boring papers shows how dead philosophy is (every heard of books, chapters in collected volume?). Right now philosophy journals are filled with word limited papers that make minor points in a dialectic that is of no interest to anyone. Much rather have longer papers that have something actually interesting to say and move the field forward. But no the current situation is better wading through heaps of short papers that have nothing to say.
Fwiw, many law reviews don’t have page limits (I can think of at least three 100+ page articles off the top of my head). Nonetheless, a great many (though not all) of these articles are extremely boring and spend a ton of time repeating points that are already well known to anyone would bother to read the article in the first place. There’s recently been some pushback to reign in these book length articles. I really don’t think that philosophy is going to become any more interesting by embracing longer articles. In many fields there are already more words published each year than any sane researcher could find the time to read.
Plus, if you have a book’s worth of ideas, you can still write a book. It’s not like the article form is the only one available.
Lol what? You don’t think there’s any room between 8000-12000 on one hand and 100 page novellas? Do you have any idea how many good interesting papers go unpublished or dumbed dow to fit inside word limits. Sometimes it takes more words to publish novel ideas or argue against the orthodoxy (see my first point in the original comment). But philosophy isn’t interested in that. Oh yes and like getting a book deal is as easy or moderately difficult thing to do. The absolute unthinking resistance to my comments shows the depth to which philosophy is committed to maintaining the current dead boring orthodoxy.
Yeah, I have zero faith that many great ideas are being turned away because of a 12k word limit. If someone tells me they have a great idea but they can’t give an adequate summary of it in 12k words, and they can’t get a book deal for it, then I’m betting the idea isn’t that great.
Why do you think 100 page ‘novellas’ are the shortest book length? I know (because I asked them) that University of Michigan Press will run things at 25000 words, and they didn’t act like it was a close call when I asked.
You seem to be forgetting that internet publishing isn’t just for journals, it’s for books too. And just like Kindle Singles can be basically long paper length, so can books with open access presses.
Maybe there will be a gap somewhere – I’m not sure just where a 17000 word thing should go. (Probably a book press, but maybe you’d have a hard time finding one.) But I’m very very sceptical that it can’t be edited to something reasonable, or a good related point made to get it to 22000 and done.
I continue to be surprised at how little some philosophers say they read and how some also then generalize from their own case to the general case.
It slightly worries me how few commenters on topics like these interpret them as co-ordination problems. That a journal is the best because everyone says so, or because of prestige, doesn’t show that philosophers are mindlessly following convention, or biased, or sheep to trendiness, or that there’s an arbitrary self-fulfilling prophecy in place. It shows that there’s a co-ordination problem and the way it’s been solved is by having Phil Review, Nous, Mind, Leiter etc act as Schelling points. It’s not like the journals just have a set of standards and everyone happens to agree Phil Review is the best – the journals are trying to attract the best papers while authors are *also* trying to seek the best journals for their papers. Once some commonly recognised measure of quality is set up, the best papers do, on average, go to certain journals and those get recognised as the best journals because they have the best papers.
That’s not to say things can’t change—Ergo, for example, seems to be moving up the ranks by having a speedy editorial process which lets it sample more papers to find higher quality ones, increasing its average quality—but trying to have everyone recognise certain journals as better than they are currently perceived risks misframing certain sources of difficulty, and leads to e.g. commenters focusing on prestige rather than the properties that prestige is tracking i.e. status as a Schelling point. If we only succeed in recognising a new journal as better, the current best papers will then get sent to that journal and we end up where we began. If we only change the standards of what counts as a good paper, then the papers that meet those standards end up getting sent to Phil Review. You would need to change what counts as a good journal AND our paper standards somewhat simultaneously, and I’m more inclined to think that our current standards themselves have evolved to be that way for good reasons which we’re unlikely to abandon or change significantly.
The problem, as I see it, is that bad historical reasons have led us to Schelling points that don’t fulfil certain important desiderata. For example, ideally there would be no journals that are department run in the top 20. Having a single department run and provide all, or most, of the editorial board and associate editors of a top philosophy journal gives way too much power to that department, which may have a department culture with narrow views about which topics, methods, or thinkers should be prioritized. Yet several of the top journals have this structure. Likewise, some journals are dominated by certain powerful individuals (often a founder) who fill the editorial board with their buddies and people who think like them. Again, not something we want in our “best” journals. What we want instead is journals with more “democratic” structures such as those controlled by philosophical societies with membership open to all professional philosophers.
Another problem is that most of the top 20 or so journals in philosophy are based in the US and there are probably only two top 20 journals that are not US or UK. This is way too much power for the US and UK and reflects the dominance they had in professional philosophy 50-100 years ago when academia was not sufficiently developed in other countries. We really need to see a decent number of philosophy journals from other countries, especially some non-Western countries, among the “best” but that will not happen when the “best” is mostly based on historical “prestige” or Schelling points that were established decades ago.
Very interesting. I see the potential problem, but it may not be much of a problem in practice.
(I’m not sure which philosophy journals are the top twenty – I found a list compiled in 2015 by Brian Leiter so I’m going with that.)
I think only Ergo and Imprint still have any founders among their editors. Ergo is, famously, exceptionally democratic. So the domination by a founder doesn’t seem like much of a problem. But I may be misunderstanding your point, since almost all of the founders of the top 20 journals are dead. (Nick Rescher lives!)
I’m not sure what makes a journal ‘based’ in one place rather than another – for example, where is the European Journal of Philosophy based? What about Erkenntnis? Synthese? It seems to me it’s more important what parts of the world are represented among journal editors.
I’m glad you raised these points, though. I was completely convinced by Edward Teach’s comment, but the Schelling model of ‘status’ of journals doesn’t sit well with other things I’ve tended to think about philosophy journals. For example I’ve always figured it was fine for there to be in-house journals, since they seem to publish really good papers… but you’re right that that’s not the end of the story.
Thanks for your thoughts. Having a journal run by a single department is, in my view the biggest of the three problems I raise. I think that it is pretty obvious that there is a significant problem here. I can certainly say more about this, but seeing as you don’t discuss it, I will put this aside for now.
Ideally, I would like to avoid discussing specific journals, but because you mention specific journals this now seems unavoidable. I agree with you that Ergo is very well-run journal. If other journals were more like it, things would be a lot better. However, the other journal you mention is a good example of what I had in mind (as it the Journal of Political Philosophy). I have heard several credible complaints about the influence that certain people have at these journals. Another good example is PPA. If you are unfamiliar with the controversy about this journal from several years ago, look it up on the philosophy blogs. The problem seems to have been a powerful clique of insiders.
Regarding your last point, the European Journal of Philosophy, Erkenntnis, and Synthese, are all based in Western Europe, which with pan-Europeanism and the EU, often functions as a quasi-country. You say:
I agree that this matters as well, but I don’t think that it is “more important”. A US based journal may have several foreign nationals on its editorial board who did grad school in the US and work in the US and be culturally very American despite the presence of these foreign nationals.
I think it’s actually much worse than you suggest in the idea that “I think the most charitable interpretation of what respondents to surveys like this take themselves to be doing is answering the question, “which journals contain a higher proportion of articles I judge to be of high quality”?”
As several people have pointed out, perceived prestige is playing a huge role here. But even if we set that aside to articulate a slightly more intellectually interesting process, it wouldn’t be the one you mention — that would require reading a large and somewhat random proportion of the articles in a given journal, and making quality assessments of each of them. But no one reads journal articles randomly, or cover to cover. (We rarely remember that some of them even have covers.) If someone thinks a journal is a “good” one because of the quality of its papers, this is probably because there are a quite small number of papers that they like, which are in that journal. They don’t read the articles that don’t interest them, and so cannot make any reasonable assessment of the average quality.
This is exactly the core of my comment above, but (as above) I want to stress the *further point*: since this is (agreed) true of each person, and, for each given journal, for each such person, the relevant small set of papers isn’t drawn from anything like uniform distribution over the papers that the journal has published, the journal homes of certain papers are going rise to the top in aggregate assessments of journal quality, *just because those certain papers happened to be published there and not elsewhere*. So, the aggregate journal quality assessments say nothing about the quality of most papers in those journals.
Since very few of us have made any attempt to examine, read, look at or even know about all the journals there are, the answer is almost always going to be in terms of what journals the respondent has noticed, occasionally read or vaguely heard about, from student days onwards. Or else the journals where the respondent has noticed, or even read, an article he/she remembers.
I agree that such surveys are worthless or even damaging.
This question is generational and employment-situational in many respects. My Ok-Boomer generation was largely attitudinally formed academically pre-internet, and so depended on many more local factors like grad school experience and place of hire (if you were lucky enough to get a job)–prestige 30-40 years ago was pretty well set in stone at both ends perhaps even more so than now. (As a young prof approached at a smoker and asked where I got my degree–UT–Knoxville–the response was “Oh, a jock school”.) So in grad school you were highly impressed by your profs what journals mattered, and you can guess what big names prevailed. But then I was hired at a small University of Wisconsin campus with a 4/4 load with a tiny library. It carried Mind, Philosophy, The Journal of Philosophy, and Analysis, recommended by philosophers employed there who preceded me. Well guess what journals I read? And guess what journals I published in? (Well, just Philosophy and Analysis, but eventually 5 papers, though I submitted to all four at some point.) And that’s probably how so many of us Boomers got into thinking about journals and publication. A large measure of my generation had attitudes fixed by access, not just to journals but attitudes about journals.
Thankfully that’s no longer true. Access is very much open now at least to journals themselves, and what in part prompts this OP. But my generation was strongly restrictive in attitudes about what journals mattered, and that carried over into the R1 hires of my day, and influenced attitudes in their grad students no doubt, with similar experiences then extended, etc. etc. Its time to Ok Boomer my generation and get over all that.
Like Alan, I “learned” what journals were well respected by seeing which journals our department kept in their Department library. I was at Western Ontario for my PhD and they had a great collection of the most important journals in philosophy of science. If one is going on the market as a phil. of science, and one is coming out of a programme at a place that is well respected in phil. of science, this is a sensible method. It has proved very useful for directing me to the journals that matter in my sub-field. But this was during the time when physical journals were all there were,
My ranking on The Brooks Blog brought together several information points including Leiter’s survey, the ESF European rankings and Australian rankings grouping journals into A*, A, B and C. I still think it’s pretty good http://the-brooks-blog.blogspot.com/2011/09/journal-rankings-for-philosophy_29.html but 10 years later needs an update