Leiter Posts Response to Criticism of Rankings—A Response to the Response

Dear Philosophy Colleagues,

Today Brian Leiter posted what he called “an amusing (and also insightful) e-mail ‘rant’ about the PGR and the recent campaign against rankings,” written by one Michael Bramley, “a longtime reader” of Leiter’s blog. This is clearly some sort of rebuttal directed at criticisms—I’ve made some here on UP@NIGHT—of the PGR, of rankings of graduate programs in philosophy, of Leiter himself. Leiter has been uncharacteristically restrained in the last couple of weeks and has not responded to critics, but since he chose to post Mr. Bramley’s messages under the banner “insightful,” it’s reasonable to suppose that Leiter shares Mr. Bramley’s views. Leiter calls this a “rant,” perhaps a sign of some reservations about Mr. Bramley’s remarks, perhaps a term of affection. But nowhere does Leiter say that he disagrees with Mr. Bramley. He goes out of his way at the end of the post to add a follow-up from Mr. Bramley, and the general line of argument is similar to that of defenders of the PGR.

The post is a hearty brew of straw man and ad hominem arguments. I will not go through the entire thing here, but I must address a few of the most illuminating passages, to get us all started. From the first paragraph of Mr. Bramley’s missive:

Please allow me to express my support over the recent rankings nonsense by venting my frustration at the campaign to remove you from the PGR and the campaign to stop all rankings in philosophy.  A move which, it is obvious, is for the benefit of those who do not score highly and not for the benefit of students.

So, all of those who are concerned about the PGR’s methodological problems—for example, the lack of defined criteria, which in turn leaves judges in the position of introducing their own “philosophies of evaluation” (Leiter’s words) into the survey—and those of us who worry about biases in the rankings and the creation of a halo effect in hiring that is unfair to candidates, are criticizing the PGR out of crass self-interest. While, on the other hand, since this is a defense of Leiter’s rankings, I assume that he, in contrast, acts out of the goodness of his heart. This is not the first time we have heard accusations of this sort. Let’s call it the resentment dismissal, and hope that Nietzsche is not rolling over in his grave at this trivialization of ressentiment.

From the next paragraph:

Talk about the perfect being the enemy of the good.  Plato could not have done a better job of convincing everyone that everything is worthless and shit until and unless we can all apprehend the Form of the Good Ranking System.

Speaking for myself, as a pragmatist, I must say that I don’t generally advance views that are tagged with making the perfect the enemy of the good. And I assume that most, if not all, of my non-pragmatists colleagues are not criticizing the PGR from the realm of The Good, even aspirationally. This is a straw man on fire, a sort of Burning Man without the arts. This is ludicrous. We have suggested doable alternatives to the kind of rankings that we find in the PGR, for example, a comprehensive information site with a sophisticated search engine, which will in effect allow each prospective graduate student to create their own “rankings” based on his or her own interests and needs. Real data would be used in such a site. In addition, as I have noted in other posts, although I would prefer no rankings, if we are to have them, they should not suffer from the kind of obvious methodological problems of the PGR.  The PGR is so far from The Good that those who criticize it have plenty of room to maneuver between it and The Good.

Next, from the third paragraph:

The PGR is largely an informed-opinion poll: what do the philosophy professionals think of certain philosophy departments?  This is interesting and good to know (emphasis added).

I am tempted to lead off here with:    EXTRA, EXTRA, READ ALL ABOUT IT: THE PGR IS AN OPINION POLL!! But let’s look at what in fact it is polling: “what do the philosophy professionals think of certain philosophy departments?” This would be an acceptable statement with the following modification, “what do the CERTAIN, SELECTED, philosophy professionals think of certain philosophy departments?”  I mean, isn’t this precisely the heart of the problem? The PGR presents itself as speaking for the profession as a whole, when at best it speaks for a slice of the profession. We don’t even know if a majority of those in the profession would agree with the idea that it speaks for them. But we surely know that it doesn’t speak for many people. Just listen to all of the critical voices. (To suggest it speaks for the profession is real chutzpah. And I believe that this reflects Leiter’s view. If not, he is more than welcome to say so here or in any other venue.)

Continuing in the third paragraph:

If those for whom the PGR is intended are unable to understand what an opinion poll is, then they should demand a refund from their undergraduate education for having failed to teach them basic critical thinking.

Again, we hear that the PGR is an opinion poll, and we, those opposed to rankings, in failing to understand that it’s an opinion poll haven’t learned basic critical thinking skills. Of course the issue is not whether it is an opinion poll–although I am happy to hear a PGR fan characterize it so honestly–it’s that we think it’s a poorly constructed poll/survey, which, much more importantly, pretends to be more than a mere opinion poll. Opinion polls capture snapshots of views held by members of the group being polled. They are time sensitive. So a disclaimer at the top of the PGR’s rankings might be nice: this is an opinion poll and like any poll its results are confined to the polling period.  But we won’t get this, in part because, let’s be honest here, the PGR doesn’t present itself as an ordinary poll. Why? Because it claims that it is reflecting durable judgments about the “quality” of graduate programs, judgments that are not as variable as mere opinion.

Now the last line from the same paragraph:

And if the professors who oppose it do so because they think opinion polls/reputational surveys do not capture adequately the real picture, then they are free to construct ways to capture this ‘real picture’ that they are so worried about missing with the PGR.

I am not opposed to the PGR because it doesn’t capture the “real picture.” This is not a debate about who gets to present reality. No, the problem with the PGR is that its methods can produce inadequate and distorted information.  As it stands, it misleads readers about its mission.   We can do better—just better!—without seeking the ideal of The Good.

And for now, just a word about the follow-up from Mr. Bramley that Leiter includes at the end of his post.

Before I take any more of your time I must say just this: the PGR is a collection of a large number of informed – expert – opinions of department reputation.  Students will ask their professors for advice about where to go for graduate school.  Unless these professors will refuse to even answer their students’ questions, then professional opinions on departments re. graduate training are legitimate.  So it seems this whole thing amounts to exactly this: ‘by all means have your opinions and even offer them to students – but for the love of humanity, do not put them in one place and record them on paper!’

Unbelievable.

Yes, unbelievable, indeed! Just look at the “logic” here. It seems that Mr. Bramley—and presumably Leiter himself, since he went out of his way to post this as a follow-up—don’t understand how radically different getting information from individual professors is to the claims of a survey like the PGR. To be more specific, Bramley (& Leiter) see the PGR as just a collection of the opinions of professors in one place. Not only is this an impossible claim—when was the last time you gave advice to a prospective grad student and gave ordinal rankings of his or her choices, as opposed to discussing the pros and cons of programs, etc.?—but it once again misses the most basic point: the PGR does not claim to the be the mere opinions of a bunch of individual philosophers. Its overall and specialty rankings pretend to tell us something about the state of graduate education as a whole. The idea that the PGR seriously marginalizes whole swaths of the profession still hasn’t gotten through to Brian Leiter (or perhaps it is his idée fixe and has been driving the enterprise from the start).

If Leiter doesn’t share Bramley’s views, perhaps he can say more about what it is he finds “insightful” about the messages to which he’s given so much space today.