Note: The following is an edited excerpt from Saving Psychotherapy: How therapists can bring the talking cure back from the brink. You can buy it on Amazon.
Licensing exams do not assess your effectiveness as a therapist. They arenāt meant to. That bears repeating: License exams do not assess your effectiveness as a therapist. They are a licensing boardās best effort at assessing whether you have the minimal knowledge (not skill, knowledge) to be able to practice independently without being a danger to the public. Thatās all. When therapists decry the fact that license exams are nothing like doing therapy, theyāre right ā and their point isnāt relevant. Exams arenāt supposed to be like therapy. If you want to know how good you are as a therapist, look elsewhere, because exams are not and are not intended to be a barometer of clinical effectiveness. They are a somewhat crude assessment of safety for independent practice.
With that aim in mind, do they work? Do licensing exams make therapists safer?
Thereās remarkably little data to answer that question. We canāt, at this point, randomly assign therapists to either exam or no-exam conditions to see whether one group would ultimately practice any more safely than another. Any kind of alternative study design would raise questions about other variables. So it is quite difficult to determine through research whether license exams “weed out” therapists who are less safe than their peers. From the evidence we do have, it seems the exams do weed out some therapists. But they weed out very, very few. And they probably weed out the wrong ones.
Almost everyone passes
California, my home state, is a good place to look at licensing exam statistics. For one thing, we have a lot of psychotherapists here: More than 75,000 under the governance of the stateās Board of Psychology and Board of Behavioral Sciences (the BBS governs the masterās level professions). For another, these boards are quite transparent about testing data.
In the first half of 2013, 992 people took Californiaās Written Clinical Vignette exam for family therapist licensure.* The overall pass rate for that exam cycle was 82% (816 people passed). At first, that can appear to be an exam that is reasonably doing its job, weeding out almost one in five test takers. But a closer examination of the data ā which, to the stateās credit, is only possible because they make this level of data available ā shows that 86% of people taking the test for the first time passed it. 74% of those making their second attempt passed. And just over half of those on their third attempt or higher passed the test. While thereās some variability in pass rates from one six-month exam cycle to the next, those numbers appear to be pretty typical. They also appear to be similar to the numbers for the stateās exams for other mental health professions.
The fact that the pass rate goes down on subsequent attempts is good; if passing or failing were truly random, then the pass rate would stay the same from one attempt to the next. But notice what happens when we think about how candidates progress through this exam. More than 17 of every 20 examinees pass on their first attempt. About three quarters of those who failed the first time pass on their second try. And more than half of the few still left after two failed attempts get through on the third. The overall outcome of the exam process looks roughly like this:
Yes, some people do give up and drop out of the process along the way. Small numbers simply give up after failing, in some cases after failing just once. Others move out of state, for a variety of reasons, and seek licensure there. Still others exit the process for other reasons. This graph doesnāt capture those early dropouts ā it only shows those who stay in the process ā and by showing a single test cycle, isnāt a perfect representation of the testing and retesting of a single cohort. But those who drop out once theyāve made it to the testing process are a small group, and the graph gets the point across: Of those who stay in the exam process, almost everyone eventually passes. And while the proportions differ somewhat, the same conclusion is true in all the mental health professions, among national exams as well as Californiaās state exams. Hang in the testing process once youāre in it, and you are extremely likely to pass.
And of those who donāt ā who are they? Is it because theyāre unsafe?
Probably not.
Looking at data from teaching and nursing, studies there have shown that license exams do not appear to improve the safety or effectiveness of a profession’s workforce. Instead, license exams appear to disproportionately shut out (1) people who identify as Hispanic and (2) people for whom English is not the language spoken at home.
Though those findings on license exams do come from other professions, and therefore should be taken with a grain of salt when considering how well they might apply to therapists, they should nonetheless be setting off alarms within the professions and within state licensing boards. There is no reason whatsoever to believe that those who identify as Hispanic or those whose native language is not English are inherently less safe as practitioners than non-Hispanic White native-English-speakers.
Furthermore, many states and the professions themselves are actively trying to improve the racial and ethnic diversity of the mental health professions, so that the populations we serve can have quick and easy access to professionals who understand their issues and concerns. If licensing exams donāt appear to make us any safer in our work, and do appear to shut out professionals based on their skin color or native language, then these exams are running counter to licensing boardsā stated mission of consumer protection, not serving it.
Should we be using licensing exams at all?
Again, there is no data that correlates licensing exam performance with safe practice. The connection between the two remains purely theoretical. While exam developers go to tremendous lengths to demonstrate content validity of their exams, there have been no studies of psychotherapists that have correlated exam scores or passing with safe or effective practice down the road. What little data does exist about correlates to exam success shows that licensing exam performance aligns closely with GRE scores and academic performance ā in other words, that the exams most directly assess your skill at studying for and then taking exams.
Even so, there are two kinds of arguments that can be made in favor of the continued use of license exams for the mental health professions. First, the high pass rates on licensing exams can be argued as evidence of their effectiveness, not their failure. Examinees meaningfully prepare for the exams, and do, in fact, use the exams to demonstrate key knowledge in the profession. That most people pass their exams doesnāt change this fact; if anything, it can indicate that we do a good job preparing new professionals for this key hurdle.
Second, it can be argued that even if the exams arenāt actually valuable in terms of public protection, they still hold value in other ways for the mental health professions. There is some general evidence that licensing exams get more difficult as more people seek to enter the work force of a given profession. While this evidence is debatable (it could also indicate that as more people seek to enter a profession, the proportion of people trying to enter who arenāt actually qualified grows), if taken at face value it means that license exams protect the profession from rapid increases in the labor supply, which otherwise would drive down salaries as the supply-demand curve changes. Exams also give the public the appearance of a high professional standard, which has value even if the reality isn’t quite so clear.
Regardless of their overall usefulness, licensing exams are certainly unlikely to go away in the foreseeable future. So the immediate focus should be on making them better at serving their stated purpose.
Making exams better
As long as exams are described as public protection measures (as opposed to, say, tests of general knowledge in the profession), tests for licensure should be more directly tied to the knowledge and skills most clearly necessary for safe practice. Such an exam would likely be limited to legal and ethical requirements and crisis intervention. These have much clearer and more direct links with public safety than questions about the specifics of several different theoretical approaches, most of which the examinee probably doesn’t use in practice and will not be using after they are licensed.
Exams also must allow enough time for even English-language learners to fully consider and complete their exams. Extending the time frame for examinations reduces the likelihood of cultural bias in the results.
With these changes, licensing exams would be much more in line with the public protection goals of state and provincial licensing boards.
In my experience, most therapists are perfectly willing to be tested, even knowing that the process will provoke anxiety with its high stakes. We just want to know that what we’re being tested on is relevant, and that the tests are useful.
Right now, we can’t say either is true.
# # #
* – California is the only state in the US that does not use the National MFT Exam; the structure and content of the stateās MFT Standard Written Exam and MFT Written Clinical Exam, when combined, closely parallel the structure and content of the national exam.
For more information on Saving Psychotherapy: How therapists can bring the talking cure back from the brink, click here. As mentioned at the top, it’s also available on Amazon.
(c) Copyright 2015 Benjamin E. Caldwell. All rights reserved. Reprinted here by permission.