Relationship education: Scott Stanley responds

Stanley, one of the developers of the popular PREP program, offers a more optimistic view of the Building Strong Families study’s Oklahoma City data.                                                                                                                                                                                                                                                                                                            

People in Bad MergentheimI posted last week about the disappointing 3-year results of the Building Strong Families study, a major study of relationship education involving more than 5,000 families at nine sites around the country.

Scott Stanley, one of the developers of the PREP relationship education course and a (quite deservedly) well-respected name in relationship and marriage research, wrote me in response, offering a more optimistic look at the site-specific data from the study’s Oklahoma City location. He kindly granted permission for me to post his letter, in its entirety, here. The emphasis added is mine and I’ve plugged in some links, but otherwise left his writing unchanged:

Dear Ben,

Just read your post on BSF in OK. You raise important points like those raised by others.

I wanted to argue the other side a bit, particularly about the Oklahoma finding at 36 months in the BSF study. I won’t take the space to do it fully in this comment, but I would refer your readers to a blog post I did on the 36 month findings in light of a similar critique by Andrew Cherlin in the last week of 2012. The piece (my blog piece) links to Cherlin’s critique and several other interesting pieces that your readers may appreciate. Andrew Cherlin has read my piece on his critique, and I believe he has changed his mind about the importance of the stability finding in OK at 3 years out. (see links here and below)

The key point is simply this: As I explain in my piece, what you refer to a small difference in family stability is actually a 20% increase in family stability for these most vulnerable families 3 years later (49% in the program group vs. 41% in the control group, of families where the children born lived continuously with both parents over the three year follow-up period, which is a 20% improvement of the intervention group over the control group).

What you, as so many others, are likely unaware of is that the evidence for most government social programs (or other programs for that matter) is dismal (I also make this point in my piece). I give some suggestions for how people can look such things up in my piece as well.

This OK BSF finding is actually rather extraordinary, and it is unlike findings in most all evaluations of government programs, especially evaluations of initial efforts–it is significant, relatively large (this is not a small difference on such an outcome in a policy study), and it is on what is easily argued the single most important type indicator relevant to this study context and family policy. It is also not at all unusual to get earlier impacts on things that can change more immediately that may (as in this case) set up longer term, more important outcomes.

You may not be impressed with that finding, but there are those who know government evaluations quite well who find it remarkable. However, it is true that among people who have been critics of these fledgling efforts, the desire has clearly mostly been to focus on the pooled results across sites where the story is not positive.

In addition to the points I make in my blog in commenting on Andrew Cherlin’s piece, I would add these points here about the finding you dismiss: This impact on a hard indicator of an outcome (family stability) of prime interest three years after the intervention (1) happened in the only program to deliver a substantial dose of intervention to a high percentage of couples in the intervention group, (2) happened in the only program to have many significant impacts on relationship quality variables at the intermediate stage of evaluation, and (3) happened in the only site that also had a stability findings at the prior time point.

The link to my comment on Andrew Cherlin’s piece is here:
[There is also a full disclosure comment at the end of that blog piece.]

The link to a resent presentation where you can see Andrew Cherlin’s comment on the OK finding is here:

# # #

My thanks to Dr. Stanley for his thoughtful email and the permission to repost it here.

Your questions and comments are similarly welcomed. You can post them in the comments below, by email to ben[at]bencaldwell[dot]com, or on my Twitter feed.