Williams is #1, but....

Let me start by saying I'm perfectly aware of the stupidity of biting the hand that feeds you, yet... I think some commentary is valuable here, if only for the inevitable future rankings that don't place us at #1.

I do think that Williams is the best institution in the country, but it's not because Forbes says so. Here's why. Forbes lists the following parts for its ratings:

• Average salaries of graduates reported by Payscale.com (30%)
• Number of alumni listed in a Forbes/CCAP list of corporate officers (5%)
• Enrollment-adjusted entries in Who's Who in America (10%).
• Freshman-to-sophomore retention rates (5%)
• Student evaluations of classes on the websites RateMyProfessors.com (17.5%) and MyPlan.com (5%). 
• Four-year debt load for a typical student borrower (12.5%)
• Overall student loan default rate (5%). 
• Actual graduation rate (8.75%) 
• Gap between the average rate and a predicted rate, (8.75%). 

More methodology info is here (pdf), along with justifications for each data set that I don't have time to parse tonight. But here are the main problems:

1. The average salary of graduates is, in itself, not an accurate ranking of professional success within one's field, but instead dependent on the fields that grads enter. A college with a lot of pre-Med and pre-finance students, like Williams, will do well here because of the kind of students who come here, not because we educate them espeically better. A college with a undergrad population that's into non-profit work is going to get skewered here. This same critique applies to the corporate offices part, though it's understandable given Forbes's area of focus.
2. Rate my Professor is alien to me because we have our own system, Faktrak, which is (properly) anonymous and only accessible to students. Whatever the sample of feedback put up there for Williams, I don't trust it because it's a poor sample. Same for MyPlan.
3. Debt load and default % will depend on the financial resources of the school and how generous its aid policies are. While resources are important, it would probably be better to measure financial aid directly, instead of this measurement, which penalizes schools that have poorer undergrad populations.

The last thing I'll say is that using these numbers puts pressure on schools to get kids to come back/graduate when that might not be the best thing for some students. I don't want these rankings to be too important, because if they are, it makes the work of education that much more difficult. Williams is the wrong place for some people; they come and realize that a city would have made more sense, or that snow isn't their thing at all. Those are legit reasons not to like it here, but Forbes can't distinguish between those and other reasons.


Still cool, though.