California’s MFT Clinical Exam is broken

California flagBack in May, I wrote about how pass rates on the California MFT Clinical Exam for licensure had fallen off a cliff. At the time, the state’s Board of Behavioral Sciences offered an explanation for why the pass rate might have been higher than expected at the beginning of 2016. However, they had no explanation for why the pass rate since then had fallen so far.

The most recent data on California licensing exam pass rates [page 25] makes clear that the alarmingly low pass rate in the first quarter of 2017 — when just 57% of first-time test-takers passed the MFT Clinical Exam — was not simply an aberration. It truly does appear that the exam is broken.

Here’s the data presented over time, including new data for the second quarter of 2017, and compared against clinical exams for the other master’s level professions:

As you can see, the LPCC Clinical Exam (otherwise known as the National Clinical Mental Health Counseling Examination) had a weird dip in the third quarter of 2016, but that could easily be a function of a low sample size. Just 26 people took that exam in that quarter in California. For MFTs, the sample sizes are comparatively enormous: Almost 4,000 people have taken the MFT Clinical Exam for the first time over the past year. The LPCC exam also quickly returned to more normal performance, unlike the MFT Clinical Exam.

As I mentioned back in May, there really isn’t any one ideal exam pass rate. So it’s not like that number should be anything in particular. But the passing score changes for each new exam cycle, to account for differences in test difficulty. A harder test should have a lower passing score, keeping the pass rate — the percentage of people taking the test who pass it — relatively stable over time. This dramatic drop in pass rates for the MFT Clinical Exam has now gone through the better part of two exam cycles. Taken at face value, the changing pass rate tells us two things.

  1. In the eyes of the exam, almost half of those taking the MFT Clinical Exam for the first time lack the basic professional knowledge necessary to practice independently without being a danger to the public.
  2. The population of first-time test takers able to practice independently without being a danger to the public dropped significantly in less than a year.

Simply put, neither of those is likely true. Neither one of those statements has any other data to support it. I mean, if most examinees came from a single university’s program, and something had gone wrong within that program, the first conclusion might make sense. But California has far too many MFT programs for that explanation to fly. If we had seen a sudden and dramatic uptick in the number of complaints against interns, then the second conclusion might have some support. But there’s no evidence of that either.

The simplest and most logical conclusion here, at least from the data available, is that there is a serious problem with the test in its current form.

Problems like this are highly consequential to those impacted by them. Those who fail the exam must wait 120 days to take it again. Jobs and promotions often hang in the balance.

We tolerate license exams because they at least provide the appearance of public protection. But in this case, it seems as though the MFT Clinical Exam is likely shutting out hundreds of perfectly safe practitioners from the benefits of licensure — benefits that those people would be enjoying if they had simply tested a year earlier.

The state’s Office of Professional Examination Services, which works with the BBS on exam development, will be presenting at this Friday’s BBS meeting in Sacramento. Hopefully they will be able to offer some kind of plausible explanation for this. You can watch a live webcast.

In the meantime, a new exam cycle started August 1. The BBS has not announced the passing score for this cycle yet, and the BBS and OPES know there is some pressure on them to get it right. There also continues to be discussion of moving California over to the National MFT Exam. I’ll be at Friday’s BBS meeting (it’s open to the public) to encourage a fix going forward, and that something be done to make it right for those who failed in the past two quarters — and perhaps shouldn’t have.