Dear Philosophy Colleagues,

Today Brian Leiter posted what he called “an amusing (and also insightful) e-mail ‘rant’ about the PGR and the recent campaign against rankings,” written by one Michael Bramley, “a longtime reader” of Leiter’s blog. This is clearly some sort of rebuttal directed at criticisms—I’ve made some here on UP@NIGHT—of the PGR, of rankings of graduate programs in philosophy, of Leiter himself. Leiter has been uncharacteristically restrained in the last couple of weeks and has not responded to critics, but since he chose to post Mr. Bramley’s messages under the banner “insightful,” it’s reasonable to suppose that Leiter shares Mr. Bramley’s views. Leiter calls this a “rant,” perhaps a sign of some reservations about Mr. Bramley’s remarks, perhaps a term of affection. But nowhere does Leiter say that he disagrees with Mr. Bramley. He goes out of his way at the end of the post to add a follow-up from Mr. Bramley, and the general line of argument is similar to that of defenders of the PGR.

The post is a hearty brew of straw man and ad hominem arguments. I will not go through the entire thing here, but I must address a few of the most illuminating passages, to get us all started. From the first paragraph of Mr. Bramley’s missive:

Please allow me to express my support over the recent rankings nonsense by venting my frustration at the campaign to remove you from the PGR and the campaign to stop all rankings in philosophy.  A move which, it is obvious, is for the benefit of those who do not score highly and not for the benefit of students.

So, all of those who are concerned about the PGR’s methodological problems—for example, the lack of defined criteria, which in turn leaves judges in the position of introducing their own “philosophies of evaluation” (Leiter’s words) into the survey—and those of us who worry about biases in the rankings and the creation of a halo effect in hiring that is unfair to candidates, are criticizing the PGR out of crass self-interest. While, on the other hand, since this is a defense of Leiter’s rankings, I assume that he, in contrast, acts out of the goodness of his heart. This is not the first time we have heard accusations of this sort. Let’s call it the resentment dismissal, and hope that Nietzsche is not rolling over in his grave at this trivialization of ressentiment.

From the next paragraph:

Talk about the perfect being the enemy of the good.  Plato could not have done a better job of convincing everyone that everything is worthless and shit until and unless we can all apprehend the Form of the Good Ranking System.

Speaking for myself, as a pragmatist, I must say that I don’t generally advance views that are tagged with making the perfect the enemy of the good. And I assume that most, if not all, of my non-pragmatists colleagues are not criticizing the PGR from the realm of The Good, even aspirationally. This is a straw man on fire, a sort of Burning Man without the arts. This is ludicrous. We have suggested doable alternatives to the kind of rankings that we find in the PGR, for example, a comprehensive information site with a sophisticated search engine, which will in effect allow each prospective graduate student to create their own “rankings” based on his or her own interests and needs. Real data would be used in such a site. In addition, as I have noted in other posts, although I would prefer no rankings, if we are to have them, they should not suffer from the kind of obvious methodological problems of the PGR.  The PGR is so far from The Good that those who criticize it have plenty of room to maneuver between it and The Good.

Next, from the third paragraph:

The PGR is largely an informed-opinion poll: what do the philosophy professionals think of certain philosophy departments?  This is interesting and good to know (emphasis added).

I am tempted to lead off here with:    EXTRA, EXTRA, READ ALL ABOUT IT: THE PGR IS AN OPINION POLL!! But let’s look at what in fact it is polling: “what do the philosophy professionals think of certain philosophy departments?” This would be an acceptable statement with the following modification, “what do the CERTAIN, SELECTED, philosophy professionals think of certain philosophy departments?”  I mean, isn’t this precisely the heart of the problem? The PGR presents itself as speaking for the profession as a whole, when at best it speaks for a slice of the profession. We don’t even know if a majority of those in the profession would agree with the idea that it speaks for them. But we surely know that it doesn’t speak for many people. Just listen to all of the critical voices. (To suggest it speaks for the profession is real chutzpah. And I believe that this reflects Leiter’s view. If not, he is more than welcome to say so here or in any other venue.)

Continuing in the third paragraph:

If those for whom the PGR is intended are unable to understand what an opinion poll is, then they should demand a refund from their undergraduate education for having failed to teach them basic critical thinking.

Again, we hear that the PGR is an opinion poll, and we, those opposed to rankings, in failing to understand that it’s an opinion poll haven’t learned basic critical thinking skills. Of course the issue is not whether it is an opinion poll–although I am happy to hear a PGR fan characterize it so honestly–it’s that we think it’s a poorly constructed poll/survey, which, much more importantly, pretends to be more than a mere opinion poll. Opinion polls capture snapshots of views held by members of the group being polled. They are time sensitive. So a disclaimer at the top of the PGR’s rankings might be nice: this is an opinion poll and like any poll its results are confined to the polling period.  But we won’t get this, in part because, let’s be honest here, the PGR doesn’t present itself as an ordinary poll. Why? Because it claims that it is reflecting durable judgments about the “quality” of graduate programs, judgments that are not as variable as mere opinion.

Now the last line from the same paragraph:

And if the professors who oppose it do so because they think opinion polls/reputational surveys do not capture adequately the real picture, then they are free to construct ways to capture this ‘real picture’ that they are so worried about missing with the PGR.

I am not opposed to the PGR because it doesn’t capture the “real picture.” This is not a debate about who gets to present reality. No, the problem with the PGR is that its methods can produce inadequate and distorted information.  As it stands, it misleads readers about its mission.   We can do better—just better!—without seeking the ideal of The Good.

And for now, just a word about the follow-up from Mr. Bramley that Leiter includes at the end of his post.

Before I take any more of your time I must say just this: the PGR is a collection of a large number of informed – expert – opinions of department reputation.  Students will ask their professors for advice about where to go for graduate school.  Unless these professors will refuse to even answer their students’ questions, then professional opinions on departments re. graduate training are legitimate.  So it seems this whole thing amounts to exactly this: ‘by all means have your opinions and even offer them to students – but for the love of humanity, do not put them in one place and record them on paper!’


Yes, unbelievable, indeed! Just look at the “logic” here. It seems that Mr. Bramley—and presumably Leiter himself, since he went out of his way to post this as a follow-up—don’t understand how radically different getting information from individual professors is to the claims of a survey like the PGR. To be more specific, Bramley (& Leiter) see the PGR as just a collection of the opinions of professors in one place. Not only is this an impossible claim—when was the last time you gave advice to a prospective grad student and gave ordinal rankings of his or her choices, as opposed to discussing the pros and cons of programs, etc.?—but it once again misses the most basic point: the PGR does not claim to the be the mere opinions of a bunch of individual philosophers. Its overall and specialty rankings pretend to tell us something about the state of graduate education as a whole. The idea that the PGR seriously marginalizes whole swaths of the profession still hasn’t gotten through to Brian Leiter (or perhaps it is his idée fixe and has been driving the enterprise from the start).

If Leiter doesn’t share Bramley’s views, perhaps he can say more about what it is he finds “insightful” about the messages to which he’s given so much space today.

8 thoughts

  1. Rankings measure employment outcomes, so there’s obvious practical value for anyone who’s either ambitious or risk averse. I think you overestimate the likelihood of happy surprises.

    1. Todd, I’m not sure what you are saying about happy surprises, but the PGR rankings in philosophy don’t measure employment outcomes. It’s a reputational survey confined to part of the philosophy community. They may correlate with employment, but even here we don’t have sufficient data. We need to know what outcomes we are looking to measure, e.g., any job or certain kinds of jobs. Data needs to be collected. And then we have the issue about whether the tail is wagging the dog, that is, the rankings are (unfairly) driving certain outcomes.

      1. OK, so you would accept a form of ranking that relied on better data or a sounder methodology. Until then, people will inevitably rely on Leiter or they’ll rely on folk rankings that have the same effect. Your problem is the human propensity to simplify deliberative effort. That’s how the dog gets wagged. Even if every job seeker were a special snowflake, the cost of omniscience would be excessive. Hiring committees rely on proxies to evaluate candidates and one of those is program rank.

      2. I don’t believe that folk rankings would have the same effect. The PGR has the imprimatur of philosophers. It carries weight, so its flaws are actually magnified. Hiring committees should not rely on rankings. It’s wrong and unnecessary. There is plenty of information in candidate files to make reasonable judgments. Please see some of my posts on these questions.

  2. Well, you can’t argue with folk rankings. Without Leiter you’d have no ground for dispute. In fact, Leiter has graciously painted a huge target on his back for ad hominem attack. Be grateful.

    Let me add that rankings are an ineluctable feature of careerism in the face of rationed opportunities. Here’s Jonathan Alter’s “The Promise” on the president’s belief in this principle: “But it’s safe to say that at some level Obama bought into the idea that top-drawer professionals had gone through a fair sorting process, the same process that had propelled him and Michelle into the Ivy League, and were therefore in some way deserving of their elevated status. Eventually a full quarter of Obama appointees would have some connection (as alumni or faculty) to Harvard, just one of several elite universities represented en masse in the government.”

    A fair sorting process.

    You would say this is unfair or wrong and unnecessary, but it’s not as though all of the ambitious people who attend highly-ranked schools or those who fail to attend highly-ranked schools don’t already know that these are the rules by which ambitious people seek career advancement. Obama and Mrs. Obama — just to give an example — understood those rules when they chose to attend Columbia, Princeton and Harvard.

    You’re asking for a second bite of the apple on the ground that one maladroit guy, Brian Leiter, shouldn’t have so much power. But this isn’t Leiter alone, it’s a distilled consensus that’s reflected in every institutional context where ambitious people compete to advance themselves.

    1. Todd, In all due respect, I think you are arguing against a straw man. As I have made clear in various ways in my posts, I am not naively expecting institutional reputation to disappear. It is the flawed nature of the PGR that is at issue, along with the fact that it appears to have received the imprimatur of the philosophical community, when in fact it has not. Academic departments and disciplines have ways of addressing faulty rankings, but it is the especially pernicious nature of the PGR that must be challenged. I don’t want a second bite of the apple. I just don’t want to be handed a poisoned one and told that it is good for my health.

      1. These are your words in a November 2, 2014 post to New Apps: “I am convinced that rankings in general do more harm than good.” But here you say that “[i]t is the flawed nature of the PGR that is at issue[.]” Which is it?

      2. I really don’t understand. There is no either/or here. As I’ve said many times, I prefer an information site with a sophisticated search engine that will allow individuals and departments to do a “ranking” based on their own criteria. But this in no way implies that all ranking systems are equal, if there must be rankings. Again, the PGR is especially problematic for a number of reasons that I have gone into at length in my posts. You are also taking my words out of context. I was replying to your point about our always being “ranked” in some fashion, and saying in this context that what is at issue is the PGR.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s