Bob Morse, Champion for Change

The most passive journalist on the planet is at it again today, where he discusses the latest ABA jobs data changes.

U.S. News had been among those urging the ABA to take action to improve the credibility of the important consumer information.

Yes, "urging." See, kids, this is how reform works. Figureheads like Bob Morse are a position to change the system immediately, and yet they do nothing. Then, when the people who actually care about reform show up and make a self-evident point, the coat-riders like Morse join the chorus. Then, when they write the history, they were a full-fledged member of the justified opposition, even though they refused to enact any reform steps for years.

But that's not all the magic from Bob's hand.

If the new ABA rules are implemented, U.S. News will use our own law school statistical surveys in fall 2011 to collect and eventually publish the entire new richer and more detailed set of employment and jobs data from each law school for 2010 J.D. graduates. When we gather this richer data set, we will be able to make a more exact determination of how our ranking methodology will change.

Think about this for a moment. The new ABA guidelines already make clear what information will be collected. Granted, Morse and Co. don't know what the response rates will be or how reliable the state-wide surveys will be, but you'd think they could be working with the methodology prior to getting the data itself. After all, statistical methodology should be theoretically-sound independent of the data.

Is Morse being ambiguous and as commitment-free as possible so he can make methadological adjustments to keep the T-14 the T-14, or the T-6 the T-6? I'll leave you to speculate, but it's hard for a cynic like me to not make that assumption given the nature of the propaganda business.

MBA USNEWS Flawed Numbers, Kansas Applications Down Big, and Duke Finds New Way to Grab Tuition Money

1. Grad School U.S. News Fraud:

From BusinessWeek:

The Gainesville Sun last night reported that the University of Florida Warrington College of Business supplied the magazine with inaccurate job-placement data, resulting in a higher ranking.

An anonymous complaint made through the university’s ethics hotline in August alleged that 37 percent of UF’s 2009 MBA graduating class had jobs at graduation, while 53 percent had jobs three months later. The figures supplied to USNWR indicated 53 percent placement at graduation and 79 percent three months later. UF was ranked 39th in 2009, falling two spots from the year before; in 2010 it fell to 47th.

An investigation found no evidence of willfulness, but you can't help but wonder at the similarities. Does any school ever drop as a result of submitting false numbers? Gee, I wonder why that never happens. In any event, the article gives us another look into the askew, dullard mind of Bob Morse, the laziest journalist/statistician on planet Earth:

[Morse] said he thought data accuracy was not a widespread problem, but it's one that should be addressed by organizations such as the American Bar Association and the MBA-CSC in their capacity as standard-setting bodies, not by the media outlets that conduct rankings.

"It's not my problem" is just about the shittiest possible answer a bona fide journalist or statistician can come up with. People who are actually interested in finding the truth - rather than serving as a press agent for the official organizations that supply them - actually get off their asses and find the truth, ensure accurate data, stop official manipulation where it's suspected, etc. I think at this point any illusion of U.S. News being anything more than a puppet with no journalistic or mathematical intentions is shattered by Morse's own admissions.

2. Kansas Applications Down

From the K.C. Star:

The University of Kansas Law School says its applications declined 23 percent for this year compared with last year.
...
About 850 students applied to the law school this year, down from about 1,100 last year.

They're still far from being underwater (they only admit about 350-400 each year), but this is a state flagship university. If you want to ultimately practice is Kansas or the Kansas City metro area, you can't find a better school to attend. I wonder how the quality of their applicant pool is, and I wonder what this kind of drop signifies for places like Washburn and UMKC that are 2nd-choices for KU applicants. I imagine we start seeing sub-150 LSATs more prevalently and schools have to reach further than they normally would.

3. Duke Innovates in Ways to Leach

From the Nat'l Law Journal:

The school announced this week that it will launch a master of laws program in judicial studies geared specifically to judges....

The LL.M. program will open during the summer of 2012 to 10 to 15 sitting judges. They will take intensive four- to six-week sessions over the course of two summers. The curriculum will include classes and seminars in which both judges and scholars look at the history of the judiciary and judicial decision-making. Participants will examine topics including judicial independence, bias, the selection process and use of specialized courts.

Tuition for this four- to six-week program? $17k. For roughly 1/6 of the normal school year, with a guaranteed small class size and what surely will be the lightest teaching, lowest resource demand in academic history, the school is charging over 40% of its full-year tuition price.

Ca-ching! I'm sure Duke justifies this by some inflated beliefs that the judiciary could use the advanced training, but let's be real: most courts are overworked and, frankly, the judges have better things to do than take classes at Duke for $17k and a worthless L.L.M. If a judge in my district went to this thing and was subject to election, I'd vote against him/her on principle.

But hey, you have to hand it to Duke. If kids are going to start turning down law school to find better uses of their money and time, the law schools are naturally going to move to pushing education for existing practitioners, more than they have already.

Bob Morse Responds, Tries to Push Blame on Law Schools

Well, after last week's New York Times story on how law schools use merit scholarships to boost their USNWR ranking, which get pulled away after the first year. As I noted, this is the expected response to such a system where schools are incentivized to act in a way that doesn't always neatly conform to standards of ethics or what an objectively-good law school would do. All of this is the natural consequence of the USNWR rankings having almost no merit in actually evaluating law schools.

Naturally, Bob Morse has decided to respond to USNWR role in the scholarship game and, as seemingly usual, wants to pretend it's someone else's issue. After a "no duh" paragraph about students understanding risks and the need for disclosure, Bob lays out this gem of a passage (numbers mine for purposes of commentary below):

It's clear that the U.S. News law school rankings have a (1) large impact on law schools and prospective law school students. (2) However, the U.S. News Best Law School rankings are not why students lose their scholarships. In addition, the article (3) implies that the U.S. News rankings are the key factor behind why law schools are offering more merit-based aid and less need-based aid in order to enroll students with higher LSATs and GPAs and, as a result, improve in the rankings. (4) Law schools need to take far more direct responsibility for their policies instead of citing the oft-repeated claims that they are forced into these actions solely because U.S. News exerts so much power over law school behavior.

(1) With the power of national journalism (or are we going to admit that USNWR is no longer "journalism?") comes a grave responsibility in understanding the effects of one's words. That's why it's essential for the journalist to report accurately what happens, not base his conclusions on small sample sizes or unexamined self-reporting. And the journalist has a responsibility not to have a conflict of interest in the resolution of what happens. Bob's admission that his rankings have great effect is an admission that he either has a great amount of responsibility to get it right or that he's not a journalist at all.

(2) Technically, no, Bob, but your rankings are essential in establishing a system where parties act on reliance on insincere (as in, they're getting pulled after one year) scholarship offers - often advertised as three-year deals - that only exist because your magazine irrationally believes the LSAT to be relevant to how good a law school is; and many of those students actually do lose their scholarships. It also causes the rankings to favor richer schools, possibly at the expense of more efficient ones, and it causes schools to shift more money to their merit scholarship balances. Your magazine is a but-for cause of this mess, and disclaiming responsibility because you had nothing to do with Johnny taking the bait and getting a C+ in torts is absurd.

(3) "Implies?" No, Bob. The article "implied" nothing, and to say otherwise is disingenuous weasel-wording.

(4) Why don't you take responsibility as a "journalist" for the power your words have over people's actions, especially those who need metrics to boast of their accomplishments? There are countless steps you could take to prevent potentially-deceptive and abusive behavior from occurring. You could factor scholarship retention rates into your rankings. You could factor need-based scholarships into your rankings. You could collect and disclose scholarship offering and retention rates. You could do surveys comparing students' expected debt totals to their actual debt totals and penalize schools for the difference.

There are all sorts of steps to take to aid USNWR be a detached voice that helps the consumers understand the truth and have the facts necessary to make a decision about what law schools are good. To date, USNWR has done none of them and has persisted in using a formula that has nothing to do with actual law school merit. And yet, when they're called on it, they throw up their hands and say "that's the law schools' fault!"

Well, no, it's not. As a news institution, USNWR has a responsibility not to be a party in things like this. As long as they have the word "news" in the masthead, people like Bob Morse can't abnegate that responsibility by some twisted logic. So Bob, don't throw up your hands. Do your damned job.

Either that, or change the name and admit you're just a puppet for the elite educational lobby.

On The Scholarship Game

Many of you have probably already seen the New York Times article where David Segal discussed the law school scholarship game. I think others covered it fairly well, I just wanted to add a few points of my own.

My over-arching reaction to this piece is simple: What the **** did you expect?

Here's the thing about humans. If you set up a system where parties are perversely incentivized. They're going to act perversely. It doesn't matter if its Catholic pardoners or law school deans. Law schools aren't necessarily more immoral than the next exploitable institution, but the law school system is so warped, it's practically a given that this would happen.

What you have in law is a situation where the US News and World Report rankings became such an obsession in a field where many students fight for a small number of cushy positions. Schools, naturally, want to move up in the rankings. The rankings, it turns out, are so flawed that they look only at incoming classes (and not who's still there at the end of 3L year) and disproportionately weigh the LSAT and undergraduate GPA even though they have very little relevance on how good the actual school is, unless, of course, you make some ridiculous assumptions (e.g., that if a good student goes there, it's automatically a better school regardless of what happens to him while he's there).

The natural response for schools is not to increase the quality of their curriculum, but rather to increase the quality of their matriculant pool and then forget them after the first year. And so it was and is.

Of course, there are two obvious ways to stop this. The ABA could have stepped in and regulated the offering and retention of scholarships as an accreditation metric. After all, if a law school has to have a tenured faculty and a full-blown library, I don't think it's unreasonable to suggest they only offer scholarships under reasonable conditions, such that everyone has a chance to retain their scholarship if they do well with the first-year material (which is not currently the case). At the very least, they could protect students by preventing schools from stacking all the full-ride kids in the same section and by taking other tactics designed to increase the first-year scholarship pool.

The second filter could have been (or be) Bob Morse at US News and World Reports. On one hand, I feel sorry for Mr. Morse in that it's not his fault his magazine's ridiculous and stupid rankings gained such regard among administrators desperate for something good to tell their bosses/student bodies. On the other, however, every time Bob Morse opens his mouth, he sounds dumber and dumber.

Why is merit scholarship retention not part of the U.S. News data haul? “The main reason is that we haven’t thought about it,” said Robert Morse, who oversees the rankings. “It’s not a great answer, but it’s an honest answer.”

Then Mr. Morse thought about it.

“This isn’t meant to be sarcastic,” he said, “but these students are going to law school and they need to learn to read the fine print.”

Except, as the article points out, often there is not any fine print!

And how do you not think about it? The whole point of these rankings is to find the best law school, at least ostensibly. You should be trying to prevent schools from gaming the system through shifty means. You haven't even thought about it? That's the kiss of death for when an American system is in disarray. Some jackass didn't even bother thinking about the fact that his decisions and actions actually have consequences for those working in the system. That's why our primary schools are so decayed. Bob Morse's blindness to how schools act in reliance on his publication's rankings is impressive, the type of journalistic insight one surely develops while staring at a wall and banging bricks against his head for dozens of years.

How can you possibly have any integrity in ranking these places when you have no idea what actually goes on at them? Could you imagine the health bureau saying "we haven't really thought about it" when asked why they didn't check a restaurant's air ducts for mold? Evaluating law schools is this man's business and he admits he hasn't even thought about how they act and how it may jeopardize the authenticity of his rankings. What a clown!

Another major problem I have with the situation is that the law school deans are trying to push theory and technicality over practicality. For example, check out this excerpt about Golden Gate:

[I]n recent years, only the top third of students at Golden Gate wound up with a 3.0 or better, according to Ms. Ramey, the dean....

Ms. Ramey says it is statistically possible for 70 percent of first-year students to maintain a 3.0. She also maintains that Golden Gate 1L’s are well informed about the odds they face in keeping scholarships.

“Students who have done well in college assume they will do well in law school,” she said. They would know better “if they read our materials, if they listen to anything we told them in our admission process, or read our course catalog."

There are two major problems here.

First, the dean knows very well that 70% of the 1L class cannot wind up with a 3.0 or higher. If I'm not mistaken, for that number to happen, the same students who get As would have to get a few B-s. In other words, you can't have the same people set the curve in every class (otherwise, they push more people below the median, which makes it impossible for the below-the-line people to have a 3.0 or higher). Never, in the history of law schools, has there been an entirely different set of people set the curve in each 1L class. Given that reality, focusing on what is technically possible is grossly misleading and makes it sound much easier than it actually is to get a 3.0. It's technically possible I'll wake up in the Playboy mansion with a girl in each arm or that an asteroid will crash into Earth tomorrow at 6:17 am. In most areas of the law, including, so far as I know, consumer law, we focus on what is reasonable rather than on what is possible. It just isn't reasonable to tell kids that up to 70% of them can retain their scholarships knowing full well it'll never happen in a million years.

Second, how in the **** can Dean Ramey claim their materials would apprise someone of the risks of law school?!?! The admissions process is a sales pitch, not a risk factors section of a 10-K filing. In fact, let's check out Golden Gate's JD Prospectus on page 22:

Golden Gate Law awards both
full-tuition Dean’s Scholarships and
partial-tuition Faculty Scholarships
to qualified entering students. All
entering scholarships are awarded
on a three-year basis for full-time
students and four-year basis for parttime
students. There is no separate
application for entering student
scholarships as all students are
considered for scholarships at the
time they are admitted to the JD
program. Entering scholarships are
typically awarded to students whose
application materials demonstrate
outstanding achievement in one or
more of the following areas: academic
record, LSAT scores, writing ability,
professional experience, volunteer
service, or community leadership.
...

Is this a change, or are they being dishonest? The date on the PDF is 2010. According to the New York Times, these scholarships are contingent on a GPA requirement, but, as Dean Raney says, their admissions department makes the odds of retaining clear. You think if they had changed their policy, she would have let David Segal know.

Going to the Student Handbook helps answer these questions, on page 123:

The Law School Admissions Office awards merit scholarships to eligible entering students during the admissions process. Scholarships awarded prior to matriculation are awarded on a three (3) year basis for full-time or on a four (4) year basis for part-time students. The scholarship amount is stated in the student’s admission letter. To maintain eligibility for a scholarship as a continuing student, a student must earn a minimum 3.0 cumulative required course GPA at the end of each evaluation period. [emphasis mine]

Okay - so they consider them 3-year scholarships with a continuing maintenance requirement. Here's my question for Dean Raney: Why is that not in your JD Prospectus? (second question: what 1L actually looks at a course catalog, when a 1Ls classes are predetermined?).

Do you see, Bob Morse, that there WAS NO FINE PRINT on the JD Prospectus? That's how they roll. Golden Gate is effectively selling their entrance scholarships as a 3-year deal when it's actually a 1-year deal with a reevaluation period.

In the ordinary state consumer protection law context, that's fraud with very few questions asked. (I actually think it's borderline fraud to even sell them as opportunities for a 2L scholarship when there's more chances offered than spots available).

Overall, this is a situation that developed from a flawed system where - again - transparency and regulation would help, or at least we could hope.

Consider what happens at Chicago-Kent, the school that offers students less scholarship money ($9,000) if they want it guaranteed, and more ($15,000) if they can clear the 3.25 G.P.A. hurdle. Ninety percent opt for the larger and riskier sum, according to school officials. A “significant” number later lose their scholarships, says the school’s dean, Harold J. Krent.

“The real issue is that students don’t think about this decision in the sophisticated way that you’d like them to,” he added.

Well, Dean Krent, that's nice in theory, but it's hard for them to think about it in a "sophisticated" way with (a) limited information and (b) an assumption, which the law schools prey on, that getting a 3.0/3.2/whatever is easy in law school, since it was easy in undergrad for 90% of law school applicants and its easy in other graduate settings.

What really galls me is that the administrations, ABA, and Morse all seem to have a "the consumer should do his homework/caveat emptor" attitude.

Historically, when we have a system where unsophisticated players get screwed by repeat players who know the rules in and out, we put a filter on the system to ensure a fairer process. We've done it countless times in the business/financial sector, and individual states have all sorts of laws governing industries like used cars, pawn shops, antiques, etc.

What's truly sad is that in an industry designed to teach rational argumentation, fair process, and, hopefully, justice, it took so long for people to wake up to the reality of the system and that those in power still have the attitude that nothing really needs to be changed. People are getting screwed by these type of deals and for the Dean Raneys and Bob Morses of the world to just throw up their hands is unfortunate, at best.

Propagandist of the Week: Bob Morse

This is from Bob Morse's latest defense of the USNWR rankings, titled "Law Schools Report Accurate Data to U.S. News":

Just how honest were law schools when they reported their data to U.S. News for our 2011 Best Law Schools rankings? Each year, we ask law schools to report the same statistical information to us that they report on the American Bar Association's (ABA) annual accreditation questionnaire. Despite some notable exceptions and data errors over the years, it turns out the schools are pretty reliable in their data reporting.

To support this absurd confusion of reliability with honesty (you can reliably lie, can't you?), he points to a correlative study done by Chapman's Tom Bell, which takes the data provided to the ABA and uses the USNWR's methodology to reach the same conclusions that USNWR did. Then Morse drops this absolute gem of propaganda bullshit:

The fact that Professor Bell was able to duplicate our methodology by using law school data he obtained directly from the ABA proves that the U.S. News rankings process is very transparent and can be duplicated using publicly available data. This exercise also establishes that U.S. News is calculating and weighting the ranking variables as stated in the posted methodology. Users of our law school rankings can be confident that the results are correct given the weights and rankings variables that U.S. News has chosen.

Finally, does Bell's study also prove that law schools are being accurate in how they report their statistical profile data to the ABA, the general public, and U.S. News? No [ed.: yet your opening speaks of "honest" schools because _______?], this only proves that law schools are being very careful to report the same data to U.S. News that they report to the ABA for accrediting purposes.

Has anyone, in the history of the USNWR rankings, doubted that USNWR was using data that wasn't identical to that given to the ABA, especially since the ABA information is public and USNWR, being excellent journalists, would surely cross-check their data? Has anyone ever suggested that law schools are not "being very careful to report the same data to U.S. News that they report to the ABA for accrediting purposes?"

As far as I know, they haven't. Instead, all Morse did here was publish a blog entry to say that some professor took the listed methodology applied it to the exact same data set, and came up with scientifically-similar results. What the hell is that supposed to show, again?

Ah, yes, that "the results are correct given the weights and rankings variables that U.S. News has chosen."

For most people, the word "correct" would imply that the substance of the rankings is correct, e.g., that Harvard is a better law school than Georgetown, rather than the process used to form the rankings was scientifically sound. Take a look at certain fragments of this piece in order:

• ...schools are pretty reliable in their data reporting.
• ...the U.S. News rankings process is very transparent...
• ...U.S. News is calculating and weighting the ranking variables as stated...
• ...the results are correct...

The way this was composed is deliberately designed to defend the whole system, including the substantive results. The fact that in the next paragraph he disclaims "but the data may not be good" doesn't change the fact that he spent the rest of the piece defending the results derived from that data.

The first issue is, and always has been, that their methodology is fatally flawed. The second issue is, and always has been, that the numbers presented to them are inaccurate.

The fact that Morse was able to take a complete non-story not even worthy of a blog entry and turn it into a rhetorically-sound defense of the rankings' value is admirable. I'm baffled that a professional writer and statistician can conflate the concepts of reliability, honesty, accuracy and precision all in one post. Of course, he did almost the exact same thing a year and a half ago.

The fact that Professor Bell was able to duplicate our methodology and outcomes using data collected from a different source proves that the U.S. News rankings process is very transparent and accurate.

Professor Bell concludes:

"For now, I'll just offer this happy observation: The close fit between USN&WR's scores and the model's scores suggests that law schools did not try game the rankings by telling USN&WR one thing and the ABA (the source of much of the data used in my model) another. Even a skeptic of law school rankings can find something to like in that."

Again, a complete non-story that does not justify the claim made ("the U.S. News rankings process is very . . . accurate") as read by lay people. It's not a "happy" observation and there is no hint that a "skeptic . . . can find something to like" since no ground of skepticism is weakened. The law schools' gaming of the rankings does not take place in presenting different data to the USNWR and the ABA. No one has ever thought that. What law school in its right mind would lie to a journalist when there's contradictory public information available? Do you think law school administrators are dopes?

On the bright side, at least this year's article had a one-line disclaimer that was somehow supposed to alleviate a plausible interpretation of the other four paragraphs. That still doesn't change the fact that it's shameless propaganda calculated to make the rankings seem "accurate," "honest," and "transparent," which seems odd given that just last week Bob was telling us not to take them so seriously.