Truth can be hard to pin down. Who, for example, first uttered the phrase from which the headline above is taken? It appears that Mark Twain popularized it, and he attributed it to Benjamin Disraeli, but there is debate about whether this is accurate. Regardless of its origin, the point of the phrase is apt: assigning statistics to matters tends to wrap them in a presumptive validity, when in fact numbers can be subject to interpretation or debate, or be flat out wrong. So it is with the many statistics that journalists and others are offering up these days about law schools. We’ve decided to post this item to discuss the data as it relates to NYU Law, in particular, and to correct it when it is in error.
Professor Paul Campos and his tainted numbers
In his March 8 Inside the Law School Scam blog post, Colorado Law professor Paul Campos points out discrepancies in the number of graduates law schools report as being hired by the nation’s largest law firms—based on a list compiled by the National Law Journal (the NLJ 250)—and what he credits as the accurate number reported by the NLJ itself. Campos stumbles right out of the gate, stating that the NALP reporting deadline for class of 2011 employment data was in February, so “we can assume the schools chose not to post the most recent data they have.” Actually, the NALP deadline for 2011 data is March 16, and we plan to post those statistics later this month.
Focusing on 2010 data, Campos notes (accurately) that NYU reported 296 of our 2010 graduates going to work for law firms, with 91% going to firms with more than 250 attorneys. He then estimates that NYU Law would have placed 273 of these graduates at NLJ 250 firms. “In fact, the school placed 209,” he writes. In fact? The source of Campos’s certitude is a chart the NLJ publishes each year that purports to show how many first-year associates each law school sends to the NLJ 250. To gather its data, the NLJ contacts each of the NLJ 250 law firms. But not all release this information. NLJ editor-in-chief David Brown told NYU Law that in this year's survey, the results of which he just published, 71 of these 250 firms provided no school-specific 2011 hiring data. And, if a firm didn’t participate and a law school declined to say how many graduates it sent to that firm, the NLJ simply recorded that as a zero. “If we didn’t have information from the law firm or the law school, we didn’t publish information we didn’t have,” Brown said. Presto: a sizeable group of entry-level lawyers vanish into the ether.
How did this disappearing act affect NYU’s numbers? It happens that quite a few of the firms that do not release data to the NLJ are major New York-based firms that typically hire a lot of NYU Law grads. We have strong, longstanding relationships with these firms, and if they did not provide the data, we similarly did not disclose it. During its data-gathering process this year, the NLJ sent us a list of NLJ 250 firms that had reported hiring NYU Law 2011 graduates, including the number at each firm. But 21 NLJ 250 firms that hired a total of 58 of our 2011 graduates were missing from the list, and, when asked, the NLJ told us that was because these firms did not provide information. For the 2010 data that Campos cites, five of these firms alone hired 49 NYU Law 2010 graduates, erasing almost the entire discrepancy Campos cites.
Multiple other issues and ambiguities plague the NLJ’s data collection process and Campos’s consequent claims about data discrepancy. A few examples:
• The economic crisis led a number of firms to defer start dates for 2009 and 2010 graduates. It’s not clear when, or even if, firms include these graduates in data they report to the NLJ, and certainly firms handle this inconsistently. For instance, we recorded four 2011 NYU Law graduates going to one major firm, while the firm itself told NLJ the number was 15—a figure that had to have included deferred hires and perhaps graduates from previous years who completed judicial clerkships prior to starting at the firm.
• In other instances, firms may consider graduates who took fellowships or judicial clerkships as second-year associates, meaning they were never included in the first-year associate count for any year.
• The NLJ did not include “law clerks” in their numbers, because it says its aim is to include only associates in partnership-track positions. But some law firms list partnership-track associates as “law clerks” until they are admitted to the bar.
• Campos ignores the fact that NYU Law's statistics include graduates taking jobs with major British-based firms with large U.S. offices, which are not even included in the NLJ 250.
Having trouble knowing what to believe? We have a proposal for Paul Campos: come audit our numbers. We’ll show you a list of all NLJ 250 firms to which we sent associates in 2010 and 2011. Pick a reasonably sized sample from that group, and compare them to firm-verifiable data. Then let us, and the world, know what you find.
Inaccuracy wrapped in innuendo inside omission: The New York Post weighs in
On March 11, the New York Post published a story titled “NY law schools inflate job figures: critics.” Who are the critics the Post relies on? Paul Campos and the National Law Journal, of course. So, see above. But the problems didn’t end there. The reporter, Christine Parker, states that NYU Law “revised its job numbers after the Post contacted it.” That sounds like we modified information on our website as a result of her inquiries. Not so. We did add new information to our employment statistics page early in her reporting process—information we began tabulating in December and were preparing to post even before we knew she was doing a story. While she cited data from our website highlighting the number of 2010 graduates who were initially employed in positions funded by NYU Law, she didn’t mention the following, despite being told:
• All of these individuals performed full-time legal work for public service organizations.
• In addition to paying these graduates a $2,000 per month stipend, NYU Law, through its Loan Repayment Assistance Program (LRAP), also paid their qualified law school loan debt—an average of about $1,000 per month.
• The overwhelming majority of these graduates converted their law school-funded positions into permanent legal employment.
We make no apologies for the support we extend to our graduates, and find it ironic that a number of those alleging deficiencies in disclosure by law schools are themselves presenting information that is erroneous and incomplete.
Posted on March 14, 2012