Lower California bar exam cut score yields modest increases in pass rates and decline in repeaters, but increase in attorneys appears to slow

Earlier this year, I looked at the change in California’s bar exam cut score (from 144 to 139), and what it looked like for the October 2020 bar exam. Unsurprisingly, bar passage rates jumped, especially among repeaters.

The July 2021 exam is now the third exam with the lower cut score. It’s yielded a couple of results (statistics here) that are interesting, in my view. First, repeaters have declined significantly, and pass rates for repeaters have returned to a fairly low level. Second, there has not been a significant uptick in new attorneys admitted to California this year.

Among first-time test-takers of the general bar exam who graduated from ABA-approved law schools, we’ve seen a relatively steady decline in overall test-takers. When the cut score was lowered in 2020, we saw a jump in passers. (The July 2020 exam was moved to October due to Covid-19 concerns.) The pass rate jumped from 71.7% in July 2019 to 82.4% in October 2020. It fell slightly to 78.9% this year (but still an increase, albeit a bit more modest compared to 2019). But all that’s much better than the 60-something% pass rates in July 2016, 2017, & 2018.

View fullsize

Over to repeaters. We saw a surge in repeaters in October 2020 among ABA-accredited schools, and a surge in those passing. Repeaters increased from 1368 in July 2019 to 1645 in October 2020. And the repeater pass rate climbed, too, from 34.6% to 49.7%. But this July, we saw a sharp decline to just 839 test-takers, and the pass rate fell to just 26.1%.

The decline in repeaters is no surprise—as more people pass the first time, we’d expect repeaters to drop, too. But the decline in the pass rate, I think, also shows that the lower cut score is sweeping in those most likely to ultimately pass. Those who are not passing with the new, lower cut score may be increasingly distant in scores, at the lower end of the bell curve if you will.

View fullsize

One potential benefit of the new, lower cut score would be more attorneys admitted to California, which would increase availability of attorneys and, potentially, lower costs and increase access to justice for those unable to secure representation. But it’s not clear that’s been the effect, at least not yet. Certainly, in 2020, we saw a surge in passers. But this year yielded 5568 attorneys between the February and July 2021 bar exams, across all law schools, the general bar exam and the attorneys exam, you name it. That’s down from 6906 last year, and it’s even below the 5825 in 2019.

View fullsize

I want to be careful here. It’s of course not all the bar exam. The economy of California, the legal market of California, cost of living concerns—there are many reasons why attorneys might be inclined to take the bar exam elsewhere. Additionally, it’s not clear how many attorneys who took the bar exam in 2020 did so to take a secondary bar (e.g., to be admitted in a state outside of the state where they primarily practice) out of convenience.

In short, there are many reasons why the legal market in California may behave outside of the bar exam. And undoubtedly, the surge in passers in 2020 is great news for law schools, for students facing debt and the delayed practice of law, for those who would ultimately pass but now are admitted on the first attempt and need not spend more time studying, and so on. Many of those who would have been admitted in 2021 on a repeat are put into the 2020 first-time bucket, essentially an acceleration of admitting attorneys that will work its way out through fewer repeaters passing later.

But it’s worth watching to see whether there’s a demonstrable increase in licensed and practicing attorneys in the state of California after the lowering of the cut score. On that, we’ll have to wait and see.

Multistate Bar Exam scores drop but remain consistent with scores since 2014

It was difficult to project much about the bar exam last fall given the pandemic. Jurisdictions made many changes to how they administered the exam. When the MBE scores were released last year, usually a harbinger of overall pass rates, we saw just 5700 July 2020 test-takers, down from 45,000 or so in a typical July. Many states developed novel exams; some changed cut scores or offered versions of “diploma privilege.” Early signs in some jurisdictions, however, pointed to dropping passing rates.

Now the MBE scores have been released, and the scores are a drop from July 2019—but still consistent with scores between 2014 and 2019.

View fullsize

I opted to leave the July 2020 MBE information blank in the chart, as it offers little for historical comparison (although it was much higher last year). You can see that scores bottomed out in 2018 at 139.5, so the 140.4 this year is a bit above that. Nevertheless, the decline from July 2019 suggests that bar passage rates continue to be a challenge for law schools and graduating law students. (Of course, the lower MBE mean does not automatically translate to lower bar pass rates, but it does portend that result.) We’ll see what individual jurisdictions continue to reveal in the weeks ahead.

Early signs point to dropping July 2021 bar exam results

Back in 2014, I noted early warning signs of a precipitous drop in bar exam pass rates around the country. The MBE scores were much lower. Part of that may have been attributable to an ExamSoft error, which I ultimately concluded was unlikely, and, at first tentatively blaming the NCBE’s administration of the exam, later backed off that claim as well, attributing the bulk of the decline to a decline in overall student quality as a major part of the answer.

What’s old is new again. July 2021 saw a major ExamSoft error, and now we’re beginning to see a downtick in bar passage rates. From five jurisdictions, and reporting overall results (not first-time results) so far:

Iowa, -12 points: July 2020, 83%; July 2021, 71%

Nebraska, -17 points: July 2020, 89%; July 2021, 72% [Nebraska offered a much smaller second exam in 2020, too]

New Mexico, -18 points: September 2020, 89%; July 2021, 71%

North Dakota, -11 points: July 2020, 76%; July 2021, 65%

South Dakota, +3 points: July 2020, 70%; July 2021, 73%

West Virginia, -19 points: July 2020, 77%; July 2021, 58%

Wyoming, -13 points: July 2020, 85%; July 2021, 72%

It becomes increasingly difficult to compare year-over-year exam results as the format and timing change, and as new variables enter the mix. Some possible variable to consider….

ExamSoft. Here we open with problems attributable to a remotely-administered exam as administered by ExamSoft. Not all of these jurisdictions were remote (e.g., Nebraska was in person). But certainly same-day exam problems are a significant issue. Stress and sleeplessness from the first day could trickle over to a second day, albeit with a more indirect effect. North Carolina, for instance, has already announced it would lower its cut score by 2 points specifically because of software issues. This is not unprecedented—California, for instance, made modest adjustments to the cut score for some test-takers after an earthquake hit testing sites in 2008.

Online learning. Many law students taking the bar exam had nearly half of their law school educational experience shifted largely, if not exclusively, online. It’s not clear what pedagogical effect that had on students in the long term. It would not surprise me that graduating 3Ls “lost” some amount of learning in the pandemic, which later translated to lower bar exam performance.

Pandemic fatigue. Relatedly, one could easily multiple the concerns from the July 2020 administration of the bar exam to prolonged difficulties arising from the coronavirus pandemic. Early summer 2021 may have been some of the better moments for most test-takers, so it’s difficult to know how it may have affected test-takers later in the summer.

Credentials/academic dismissal. These tend to be more individualized assessments at institutions. The incoming class of 2017 was not materially better than the incoming class of 2018—they were, from all I can see, largely comparable. Whether individualized decisions at schools, including academic dismissal rates, affected scores remains to be seen.

In short, I’m cautious about assigning any, all, or none of these responsibility for the decline in scores so far. We may see scores at other states increase, negating the premise of this post. But I’d say these are more likely to be canaries in the coal mine. I’d anticipate lower scores in many states, and a lot of questions about the cause. I wouldn’t rule out these possibilities so far, and I hope to see clear-eyed analysis of the possible sources for declining rates.

This post has been updated as results come in or are corrected.

California's "baby bar" is not harder than the main bar exam

On the news that Kim Kardashian failed the “baby bar” exam in California, an interview clip relayed the following assessment: “This one actually is harder, I hear, than the official bar.”

One reason this mythology persists is to look at the pass rates. On the June 2020 “baby bar,” 145 first-time test-takers had a 27.6% pass rate, and 134 first-time test-takers in November 2020 (where Ms. Kardashian likely took the exam) had a 29.1% pass rate. Compare that, say, to the October 2020 bar exam, which had a first-time pass rate of 73.3% (after California lowered the cut score), or a 63.7% first-time pass rate in July 2019.

On the raw passage statistics alone, it would appear that the baby bar is tougher.

But the quality of test-takers differs dramatically. Recall that every “official” test-taker from a California unaccredited school, or a correspondence course, or a distance learning course, of a fixed-facility course, is required to pass the “baby bar” as a condition of continuing their legal studies. That means 100% of “official” test-takers from these institutions passed the baby bar. That means, if the baby bar is harder than the “official” bar, we would expect pass rates among this cohort to be at or near 100%.

And that’s far from the case. The first-time pass rate in July 2019 among California unaccredited law school graduates was 24.6%—recall, 100% of these test-takers (ultimately) passed the baby bar. In October 2020, it was 37.9%—again, 100% of these test-takers passed the baby bar. Compare that to the 82.4% pass rate among ABA-accredited law school graduates.

Some additional notes from the interview:

She is then shown being told by Jessica Jackson, a human rights attorney and co-founder of #cut50, where Kardashian is interning, that she needed a score of 560, but got a 474.

"That's extremely close on a test that most people are not taking in the middle of a pandemic," Jackson tells Kardashian.

It’s not accurate that “most people” are not taking the “baby bar” during the pandemic. There were 146 first-time test-takers in October 2019, compared with 134 in November 2020. It’s a typically small group. (First-time test-takers at “law offices/judges chambers programs” actually rose from 11 in October 2019 to 13 in November 2020.)

As to whether it’s “extremely close,” scores of at least 540 are close enough for reappraisal to determine whether it merits pass or fail. Ms. Kardashian may, however, pass the next time around.

2020 bar exam saw major swings in pass rates at some institutions

Given all the variance we saw in bar exam administration in 2020, it’s perhaps no surprise to see some significant swings in how individual schools performed on the bar exam. I looked at one metric: a school’s modal bar exam jurisdiction, and how it over- or under-performed relative to the statewide average. That’s the USNWR metric (even though a state’s modal jurisdiction might change year over year). California significantly lowered its cut score, for instance; other states had a version of “diploma privilege,” which assuredly altered what bar exam test-takers looked like. Schools that always do extremely well are not going to be in the top of any list of annual year-over-year changes. And some gain by gaining; others gain by staying still while others perform worse.

I pulled this data from the ABA spreadsheets. The ABA did not include some data for some schools in diploma privilege-heavy jurisdictions, so those schools are omitted. Here are the ten schools that did the best year-over-year in terms of their performance compared to the statewide average:

	2019 pass pct	2019 state pct	2020 pass pct	2020 state pct	Delta
San Francisco	38.7%	69.1%	76.1%	77.9%	28.6
Roger Williams	68.9%	82.1%	70.0%	60.0%	23.16
Northern Kentucky	62.5%	79.5%	85.4%	80.7%	21.75
Southwestern	56.2%	69.1%	82.7%	77.9%	17.84
District of Columbia	52.0%	72.1%	70.6%	75.0%	15.66
Hofstra	60.2%	84.1%	76.6%	85.9%	14.61
Georgetown	90.8%	84.1%	96.2%	75.0%	14.53
Chapman	59.3%	69.1%	80.1%	77.9%	12.13
Oklahoma City	69.5%	84.2%	83.7%	86.6%	11.77
Tulane	76.3%	74.1%	88.9%	75.1%	11.59

Maybe unsurprisingly, a few California schools saw dramatic improvement as the cut score was lowered. Others like Georgetown appeared to find success as the bar rate suffered elsewhere.

Here are the ten schools that saw the biggest declines in year-over-year performance compared to the statewide average:

	2019 pass pct	2019 state pct	2020 pass pct	2020 state pct	Delta
Atlanta's John Marshall	63.1%	65.4%	58.3%	76.1%	-15.6
Chicago-Kent	81.1%	74.1%	74.3%	81.0%	-13.8
Western State	56.7%	69.1%	51.7%	77.9%	-13.7
Belmont	97.1%	81.0%	82.4%	80.0%	-13.7
Ohio Northern	95.0%	79.5%	86.4%	84.3%	-13.4
Tulsa	91.8%	84.2%	81.0%	86.6%	-13.3
Faulkner	81.8%	86.3%	62.2%	79.6%	-13
Pepperdine	80.3%	69.1%	76.1%	77.9%	-13
CUNY	74.5%	84.1%	65.4%	85.9%	-10.9
Lewis & Clark	84.3%	80.9%	79.4%	86.8%	-10.7

Diploma privilege, July bar exam administration, and law school employment outcomes

We saw a lot of variance in how the bar exam was administered in 2020. That assuredly affected employment outcomes for the Class of 2020, specifically the 10-month employment figures publicly released this week. (The underlying data is here.) Employment figures dropped. But the drop was not evenly distributed. And I think we can learn some things from decisions to introduce a version of “diploma privilege,” and decisions to maintain a July administration of the bar exam—I think. Maybe.

Most graduates of most law schools take the bar in the state where the law school is. There are obviously huge outliers (Yale, among others). And most of the “emergency” diploma privilege jurisdictions—Louisiana, Oregon, Utah, and Washington—particularly favored in-state law schools. (Please note, I exclude the District of Columbia, because its “diploma privilege” is really a lengthy supervised practice requirement.) It’s only four jurisdictions with only about 5.1% of all law school graduates. Yes, some of them take other bar exams, and law school graduates in other states may take advantage of diploma privilege here. But we could compare this cohort to graduates of law schools from other states—the 46 others (there are no law schools in Alaska, and I excluded the law schools in Puerto Rico). (Numbers may not evenly match due to rounding.)

	2020 BPR	2019 BPR	Delta	2020 JDA	2019 JDA	Delta
Emergency diploma privilege (4)	69.9%	68.2%	1.7	7.6%	9.5%	-2.0
Others (46)	70.8%	73.2%	-2.4	7.4%	8.3%	-0.9

The raw numbers are below:

	2020 BPR	2020 JDA	2020 Grads	2019 BPR	2019 JDA	2019 Grad
Emergency diploma privilege	1,221	132	1,747	1,143	160	1,677
Others	22,785	2,382	32,179	23,244	2,631	31,745

Bar passage-required (“BPR”) job outcomes (“full-time, long-term,” for this and all other categories) rose 1.7 points between 2019 and 2020 among graduates from schools in these four states, compared to a 2.4-point decline in the rest of the country. The correlation is of limited value, of course, but I think it’s a good place to start considering the effect the bar exam might have on legal employment.

A related component is to compare J.D.-advantage (or “JDA”) positions—i.e., positions for which passing the bar is not a prerequisite. In the four “diploma privilege” jurisdictions, J.D.-advantage position placement declined 2 points, whereas in the rest of the country it declined 0.9 points. (Yes, those JDA figures are small….)

This inverse relationship between BPR and JDA suggests, I think, that diploma privilege did not inherently improve overall job placement at a school; instead, it may have shifted graduates from less desirable positions into more desirable ones—or, more importantly, shifted graduates from non-practice positions into the practice of law. To that end, diploma privilege does exactly what it’s designed to do (if this correlation is sufficient to suggest diploma privilege is doing something…).

But not all four of these states saw equal changes to job placement. Louisiana saw a 0.9-point decline in bar passage-required job placement, while Oregon saw it rise 0.5, Utah rise 7.9, and Washington rise 5.3. Is there something about the West that saw a better job market than the East? Hold that thought while I address another….

States had significantly different timing for their bar exam administration. About 1/4 of graduates came from 22 states that offered a July administration of the bar exam compared to states that did not. (Not all graduates, of course, took the July test—many may have deferred, or taken a later offering.) Some postponed a week to August, like Indiana; others canceled it entirely, like Delaware. Some had it in July just like normal, others offered a July exam and an additional fall exam. Did jurisdictions that had a July bar exam look any different in employment outcomes? I pulled out the four emergency diploma privilege jurisdictions (but I did keep Wisconsin).

	2020 BPR	2019 BPR	Delta	2020 JDA	2019 JDA	Delta
July 2020 bar offered (22)	72.8%	73.6%	-0.8	7.3%	8.5%	-1.2
No July 2020 bar offered (24)	70.1%	73.1%	-3.0	7.4%	8.2%	-0.8

And the raw numbers are below.

	2020 BPR	2020 JDA	2020 Grads	2019 BPR	2019 JDA	2019 Grad
July 2020 bar offered	6,015	604	8,265	6,035	698	8,198
No July 2020 bar offered	16,770	1,778	23,914	17,209	1,933	23,547

These results show that employment looked better in states with a July bar exam. Placement in bar passage-required jobs declined from 73.6% to 72.8%, a 0.8-point drop in these states. Placement in other states, however, declined from 73.1% to 70.1%, nearly a 3-point drop. And we see the opposite trends in J.D.-advantage jobs again, too—J.D. advantage jobs declined 1.2 points in July bar exam states, but only 0.8 points in other states.

There’s a geographic divide, however, between Eastern and Western states. Western states (16 in total, about 1/4 of all grads) saw a decline of just 0.9 points in bar passage-required placement, while Eastern states (34, 3/4) saw it fall 2.7 points.

Break those down further into states with a July bar exam or without, and the divide becomes starker still—exacerbated, of course, by New York, which did not have a July bar exam, has the largest potential employment market, and was hit hardest by the pandemic.

	2020 BPR	2019 BPR	Delta	2020 JDA	2019 JDA	Delta
West, July 2020 bar offered (12)	74.0%	73.0%	1.0	7.1%	8.4%	-1.2
East, July 2020 bar offered (12)	72.3%	73.4%	-1.1	7.4%	8.7%	-1.3
West no July 2020 bar offered (6)	68.2%	70.0%	-1.8	6.6%	8.3%	-1.7
East, no July 2020 bar offered (32)	70.6%	73.8%	-3.2	7.7%	8.3%	-0.5

Once more, the raw numbers:

	2020 BPR	2020 JDA	2020 Grads	2019 BPR	2019 JDA	2019 Grad
West, July 2020 bar offered	2,022	195	2,734	2,044	234	2,800
East, July 2020 bar offered	4,613	470	6,384	4,550	537	6,201
West no July 2020 bar offered	4,144	401	6,072	4,278	508	6,110
East, no July 2020 bar offered	13,227	1,448	18,736	13,515	1,512	18,311

It’s only a small cache of data that only reflects a sliver of some of the variables at play in the employment outcomes. But it does appear that diploma privilege or sticking with a July bar exam administration had a positive effect on employment. Of course, we can run back around the correlation-causation fights: were those jurisdictions with July bar exams least affected by the pandemic, in which case legal employment was able to hire more, as opposed to anything about the timing of the bar exams. But it’s some data worth considering when examining the costs of changes to the bar exam.

Algebra and geometry as prerequisites to the bar exam

From the Colorado Supreme Court, 1898:

Applicants who are not members of the bar, as above prescribed, shall present a thirty-count certificate from the regents of the university of the state of New York, or shall satisfy said committee that they graduated from a high school or preparatory school whose standing shall be approved by the committee, or were admitted as regular students to some college or university, approved as aforesaid, or before enter­ing upon said clerkship or attendance at a law school, or within one year thereafter, or before September 13, 1899, they passed an examination before the state superintendent of public instruction, in the following subjects: English lit­erature, civil government, algebra to quadratic equations, plane geometry, general history, history of England, history of the United States, and the written answers to the ques­tions in the above named subjects shall be examined as to spelling, grammar, composition and rhetoric. The said exam­inations shall be conducted in connection with the regular county examinations of teachers.

California bar exam jumps after lowering of cut score

Last summer, I covered the potential changes as the California state bar lowered its cut score from 144 to 139. The October 2020 exam (administered online) was the first such exam. The statistics have been released. A few quick takes, including some comparisons to last year.

First time applicants declined slightly year over year, from 5198 to 4999. (It might be that some of the recent closures or loss of ABA accreditation from some law schools has yielded a decline in prospective test-takers.) Repeaters increased significantly, from 3008 to 3733. Repeaters appeared to be down in most other jurisdictions around the country. But given California’s very high cut score, which yields a high failure rate, students who may have otherwise been inclined not to repeat found value in trying again with a lower cut score.

The first-time pass rate among California ABA-accredited law schools rose from 71.3% to 84%. This is a big jump and good news for many law schools. Part might be the loss of some more marginal formerly-accredited schools in this figure. But the bulk is assuredly because California test-takers are more capable than the typical test-taker, and while 139 is still a relatively high cut score, it swept in a lot of new passers. The out-of-state ABA pass rate rose from 73% to 78%, not as dramatic.

Among California accredited schools, the first-time pass rate rose from 26.2% to 40%, as I suggested would provide an opportunity for such schools to thrive.

The repeater rate also rose significantly, from 26.7% to 43.0%. The raw total of passing repeaters doubled from around 800 to around 1600.

We’ll know more about race, school-specific data, and foreign attorneys when more data is released. We’ll see if longer-term access to justice or attorney discipline rates are affected. But it’s good news for law schools and law students in the state of California.

July 2020 bar test-takers down significantly from recent years, but much smaller drop in those who passed

I wondered what happened from the July 2020 administration of the bar exam compared to previous administrations, in terms of test-takers and overall pass rates. I looked at results from 10 states: Colorado, Iowa, Mississippi, Missouri, Montana, North Carolina, North Dakota, Oklahoma, South Carolina, and West Virginia. These were states that administered a traditional in-person July 2020 bar exam and did not offer some later fall test. I thought this would be the cleanest way to compare recent trends—non-traditional bar exams (like an online option or a new state test), or those that offered another administration later in the fall, might skew the test-taking pool more.

I then looked at the last three years of July test-takers and overall passers. I thought this might even out any irregularities if I looked at just the July 2019 test alone. But, of course, law school graduating classes differ from year to year, both in size and test-taker ability, so it’s only a crude portrait.

In the July 2017, 2018, and 2019 administrations of the bar exam, these 10 jurisdictions averaged 3627 test-takers and 2576 who passed, a 71.0% pass rate.

In July 2020, test-takers were down significantly, 13.9%, down to 3124 (a drop of 503 test-takers). But passers were down just 3.3%, 2492 who passed the exam (a drop of 84 passers over the three-year average).

Complicating this analysis is that North Carolina temporarily reduced its cut score, which might, on the one hand, been designed to offset any disadvantages to preparing for the bar exam during a pandemic, but, on the other hand, may have increased newly-licensed attorneys if bar preparation during the pandemic was not so adversely affected among remaining test-takers. But North Carolina saw a 4.1% increase in passers over its three-year average, one of only two states to see an increase.

A driving factor appears to be that the most likely to fail the bar exam were the most likely to opt out of this administration. That’s particularly the case for those who previously failed a bar exam—repeaters as a percentage of overall test-takers were down significantly according to the NCBE.

For those worried about the pipeline of attorneys more generally and the availability of clients in search of access to justice, the news is at least modestly promising, that while there was a substantial decline in test-takers, the decline in licensed attorneys was much more modest. (This is not to defend other problems with adminstering the bar exam in a pandemic, as I’ve described extensively elsewhere.) It does vary from state to state, however, so it’s worth looking at specific jurisdictions, too. In Mississippi, for instance, passers declined by more than 22% over the three-year average; in contrast, West Virginia and North Carolina saw modest increases. I’ve included the tables below.

	2017-2019 avg		2020		2020 v. avg		2020 v. avg pct
	Takers	Passers	Takers	Passers	Takers	Passers	Takers	Passers
Colorado	740	530	642	499	-98	-31	-13.3%	-5.8%
Iowa	187	151	162	134	-25	-17	-13.2%	-11.1%
Mississippi	169	99	110	77	-59	-22	-35.0%	-22.5%
Missouri	660	525	602	507	-58	-18	-8.8%	-3.4%
Montana	97	80	83	71	-14	-9	-14.7%	-11.3%
North Carolina	833	532	668	554	-165	22	-19.8%	4.1%
North Dakota	72	48	58	44	-14	-4	-19.4%	-8.3%
Oklahoma	288	223	268	215	-20	-8	-6.8%	-3.7%
South Carolina	417	279	385	276	-32	-3	-7.7%	-1.1%
West Virginia	163	109	146	115	-17	6	-10.4%	5.8%