Uncategorized

Insights from 5 1/2 years of California MFT license exam data: A defense of underperforming programs

by

Some MFT programs’ graduates perform poorly on California’s MFT licensing exams. Don’t assume that means the program is of poor quality; there may be good reasons.

Graduation hat1We’ve seen that there are huge differences in performance on the California marriage and family therapy licensing exams based on what graduate program the test-takers attended. We’ve also seen that for-profit MFT programs should not be dismissed simply because they aim to make money; Argosy graduates do particularly well on the exams, while University of Phoenix graduates do not. I’ve said before, though, that there are lots of things to consider when choosing a graduate program in MFT, and that graduates’ exam performance should only be one of many such considerations. Indeed, there are some major problems with putting too much stock in exam data.

If you are looking at graduate programs, and are concerned about your prospective MFT program’s exam pass rate, here are three reasons why you may want to ignore the exam data:

  • Programs can and do improve. Exam data reflects students who graduated years earlier. Remember, it takes the average MFT intern in California more than four years to move from graduation to the licensing exams. That number is a bit lower for graduates of COAMFTE-accredited programs, primarily because they do more practicum hours while still in school. Nonetheless, if you are looking at MFT licensing exam data from 2009 and earlier, you will find very little information on anyone who graduated much past 2005, and nothing to tell you which programs have gotten better or worse since then. Consider the recently-COAMFTE-accredited programs at Chapman University and Hope International University. Their national accreditation should arguably make them more appealing (and thus competitive) programs for prospective students and faculty alike. That’s important, and simply is not reflected in currently-available exam data.
  • Programs seek to give students opportunities. Consider for a moment the state’s worst-performing program, according to a table that appeared in Part I of this series: Pacific Oaks College. Based on the pass-rate statistic alone, one might presume that the Pacific Oaks program is not very good. But that conclusion can’t safely be made from that data. Pacific Oaks, over the past few years, has specifically sought to provide opportunities to historically underserved populations, creating cohorts specifically for African-American Family Therapy and Latino Family Therapy. (This outreach is vital: Lots of evidence suggests that the mental health workforce is not meeting the needs of minority populations, either in California or around the US.) Students in these cohorts may lack the family, economic, and social support, as well as the earlier educational opportunities, that other students often have. Pacific Oaks goes to great length to remediate these earlier deficits, and may be doing more, with less, than programs who start with more economically- and educationally-advantaged students. When financial and accreditation concerns threatened to close the Pacific Oaks in 2009, I was one of many who stood up in defense of keeping the program alive, and have no reservations about having done so.
  • Programs have no control over what students do after graduation. A program can really only control what happens from the time students are in the program to the time they graduate — and even then, programs have limited control over how well their students prepare themselves. A great supervisor can help an MFT Intern/Associate make up for deficiencies in their education, and help get them ready for licensing exams. Poor supervision may leave the Intern/Associate on their own to prepare, or even offer incorrect information that ultimately harms one’s chance of passing the exams. And of course, programs have no control over whether their graduates use MFT exam prep programs, although there is little evidence that these prep classes actually impact MFT exam pass rates.

Of course, before you do go dismissing a program’s exam pass rate, take some steps to get reassurance that you are making that decision wisely. If you are considering attending a program whose graduates have not performed well on recent licensing exams, ask the program (1) why, (2) what they’re doing about it, and (3) what evidence they have that they’re getting better. If the program can’t pass that test, then it’s time to wonder whether you would be able to pass yours.

That infidelity-and-income study? Don’t believe it.

by

A recent study, presented at an American Sociological Association conference to a fawning media reception (NPR / Salon), tells us that men who make less than their wives or live-in girlfriends are five times more likely to cheat. It’s bogus. Here’s why.

While commentators have been stumbling over themselves to determine what the study’s findings mean about gender, marriage, and society, no one seems to be bothering to notice that the study itself appears pretty useless. The major conclusion, linking income and infidelity, has a number of problems, not the least of which is that everyone — myself included — who wasn’t at the conference is relying on a press release and subsequent media reports about it. Such reports are notoriously unreliable, often drawing ideas from generous and/or speculative interpretations of the results rather than the study itself. That said, here are three of the reasons I’m particularly skeptical:

  1. Do the math. The National Longitudinal Survey of Youth, upon which the study is based, followed about 9,000 individuals — surely a healthy sample size. But the infidelity study examined only those who were married or with a live-in partner for more than a year, which is a much smaller subset. And of those, only seven percent of men and three percent of women actually fessed up to cheating during the study’s six-year period. So, let’s be generous and say that two-thirds of the NLSY group met the relationship-status criteria (n=6,000). And we’ll presume that roughly half are of each gender (3,000 men and 3,000 women). That leaves us with about 210 men who have fessed up to infidelity in this survey. Of those, it is not clear from the media reports how many were in situations where the male earned less than his partner; other recent research suggests about a third, or fewer than 80 of those reporting infidelity, were in such a relationship. And remember, we’re being generous because we do not have the actual numbers. To be sure, 210 male cheaters is still a decent sample, and it could be enough to draw meaningful conclusions about links between infidelity and income (among other factors). But it still is not a lot. In fact, it probably is a lot less than the number of participants in the survey who actually cheated. Remember…
     
    Updated 2010-08-20: LiveScience.com (which has more details on the methodology, and as an added bonus, commentary from Stephanie Coontz) is reporting that only 3.8 percent of men, and 1.4 percent of women, admitted to cheating in the study. That’s not exactly true; on average, 3.8 percent of men and 1.4 percent of women admitted to cheating in any given year of the six-year study, at least according to the press release.

  2. …People lie. A major income discrepancy in the relationship may be a good reason for men to simply be more honest about their cheating. Sure, you could argue, if the wife/girlfriend finds out then the gravy train ends. But if the man is in a relationship for the money, and not emotionally committed, why on earth would he lie to an anonymous survey about his cheating? There is little incentive to, and there is no cognitive dissonance to resolve over telling the truth. On the other hand, if he is emotionally engaged, and is in the relationship for reasons other than money, he may find it safer (and more palatable) to hide any previous infidelity. If all that sounds awfully speculative, well, that’s the point. People lie on studies like this, and we do not always know who will be most likely to lie or why. Yet commenters (and, too often, the researchers themselves, as seems to be the case here) treat the findings as truth in spite of their huge flaws, and then seek to divine an explanation.

  3. Account for other factors, like age, education, and religion, and the income-infidelity link vanishes. That inconvenient fact is actually in the press release, but of course, no one is paying attention to it. Does earning more than your man make him more likely to cheat? the chatterers are asking. In a word, no — the income issue appears to (at best, and even this has big holes) correlate with, but not be a cause for, cheating.

The trouble with any study of undesirable behavior that relies on self-reports is that it is impossible to know what we’re really studying — the behavior itself, or the act of reporting it. Only a more carefully (and expensively) constructed study could parse that out. In the meantime, move on. Nothing new to see here.

Blogging the AAMFT Conference: Sunday

by

The AAMFT Annual Conference wrapped up Sunday morning, as the last die-hards snuck in one last 3-hour workshop before drifting up to the Sacramento airport and back home. I was a presenter on Sunday morning, but only if you use the word “presenter” loosely; I led a panel discussion on licensing exams.

Before you go into the automatic response (which is, I believe, “zzzzzzz”), you should know there was one significant piece of news to come out of it: California’s Board of Behavioral Sciences and the Association of Marital and Family Therapy Regulatory Boards, who develop the California and National MFT Exams respectively, have agreed to collaborate on their next occupational analyses. It’s a small step but a significant one, as it will at least allow for an apples-to-apples comparison of the practice of MFT in California versus around the rest of the country. That kind of comparison would be necessary in any discussion of California eventually moving toward use of the National MFT Exam — something I believe in very strongly.

===

As is the case every year, I leave the conference energized, with many new research ideas, new potential collaborators, and new respect for the amazing work taking place in our field. I also leave Sacramento with renewed appreciation for just how clinically strong our Alliant students are and will be; it wasn’t at all unusual to see the best questions in any workshop — and often the best answers as well — coming from people with orange “Alliant” stickers on their badges. Great work, everyone. See you in Atlanta next September.