What is social validity?
In the behavior analytic community, the concept of social validity revolves around the importance and acceptability of our practices to everyone impacted by what we do. The traditional definition of social validity emphasizes the goals, procedures, and outcomes; basically, every aspect of what we do from start to finish with clients should be framed in this lens of acceptability. In clinical behavior analysis, this usually involves including the families AND the client in all decisions being made about their child’s treatment programming to increase the likelihood that caregivers will follow through with the recommendations being proposed by the BCBA.
Why is it important to OBM practitioners?
In my opinion, social validity is everything in the practice of organizational behavior management. In organizations, if you implement any kind of intervention that isn’t socially valid, the performers will be very unlikely to follow through on the procedures that you worked so hard on. Months of hard work will, at best, fall to the wayside, and at worst, upset employees and negatively impact culture. By engaging in best practices related to assessing and monitoring social validity, we drastically improve the likelihood that our OBM interventions will sustain over time and generalize beyond to people directly impacted by the initial intervention.
Another interesting perspective on social validity and its relationship with OBM is that research has found that best practices in social validity are not implemented consistently by ABA practitioners in clinical interventions. Snodgrass et al (2018) noted that less than 30 single-case studies in six major disability-focused scientific journals implemented all three components of social validity assessment. Sounds like an issue with researchers and practitioners alike not following through on procedures that are essential for ethical and effective treatment, right? I wonder if there is a science available to determine barriers to implementation of procedures, and how to remove them, hmm?
Who do we involve?
- The employees – This is the first person that usually comes to mind when behavior analysts think about social validity, and rightfully so. When interventions are put in place to impact front-line workers, they MUST be involved. When I talk about this, I usually describe this as we should be doing things with and for our employees, not to our employees.
- The leaders – In good OBM practice, the vast majority of the work being done during our interventions should be performed by leaders within the company, not by the practitioner themselves. However, I’ve seen many cases in which no one has asked the leaders, managers, and supervisors how they feel about what is being implemented, when in reality they’re likely to be responsible for the hardest aspects of a successful and sustainable OBM intervention.
- The organizational culture – Ah, culture. One of the most difficult concepts for behavior analysts to wrap their heads around. One of my mentors early on taught me that culture is, simply put, “how we do things around here”. I consider this, less simply, “the patterns of behavior that are reinforced in our organization”. This is for better or for worse, realistically, as not all the behaviors that persist in an organization are what we’d like to see… So, if you put an intervention in place, how likely is the current organizational culture to reinforce the behaviors needed to be successful?
- Society – Moving further from the people impacted directly by the intervention, is what we are looking to implement socially acceptable by the broader community? This changes across geographic locations, across different ethnicities and cultures. OBM practitioners should strive to be globally conscious in what we implement, especially since we often consult outside of our own countries and cultures.
- The OBM practitioner – And, finally, factor yourself into the equation. Will you feel good about what you are implementing? Do these projects align with your personal mission and values, and any ethical guidelines? Much of what we implement is influenced by others, such as organizational leadership, so it is crucial that we don’t blindly implement programs that we are questioning ourselves.
When do we assess?
Now that we’ve established who we should assess for social validity, the question becomes, “when?” When do we reach out to these folks to check whether our interventions are socially valid? This is actually best outlined in the original definition of the term: we should be assessing the goals, the procedures, and the outcomes. Given that structure, it should make the “when” fairly clear:
- Goals – When do we establish the goals of the project? At the beginning! So before we push forward with any project, you should touch base with everyone involved to make sure the goals you’re setting are feasible, and remove any barriers that another level of the social validity equation might not be aware of. This is crucial, and often forgotten! No matter how well you implement your OBM intervention, if the goals aren’t possible to reach, or just plain aren’t important to the people impacted, don’t expect to get the results you’re working for.
- Procedures – When will we know best if what we are asking people to do actually works for them? During the project, of course! Only when the project is happening will people actually contact the contingencies associated with implementation and be able to provide feedback that will help shape the project. OBMers often write in pencil, not pen, as we adjust what we are doing based upon the data coming in. And social validity measures are data all the same. Set up a few touch points to see how people are doing throughout the intervention, and most importantly, give time to adjust your procedures based on their feedback.
- Outcomes – Finally, when will we be able to determine if the outcomes of the project are what people wanted to see, and whether we’ve met the goals we established early on? At the end. This is the most popular measure of social validity, checking in after everything is done. However, by reading the above, you should understand now why this measure, while important, is not the only social validity measure we should be focused on. At this point, it is too late to make adjustments to ensure everyone is happy, and the data are the data at this point. That being said, this check point is the one, from my experience, tied to return customers. If the outcomes are good, people are more likely to ask you to do more work for them.
How do we assess?
How we assess social validity can be variable depending on the type of intervention we’re working on.
Observable Behavior: First, I challenge behavior analysts to operationally define the observable behaviors and outcomes they would assign as indicators of social validity. If someone is satisfied with the intervention that has been put in place, it is likely that the latency to respond will decrease, that tasks won’t take as long, and that errors will be less likely to occur. Perhaps employees begin engaging in “discretionary effort”, volunteering for additional involvement in the projects and initiatives that are promoted by leadership. It’s important to note that each measure of observable behaviors that indicate satisfaction will change from intervention to intervention.
Follow-through and Maintenance: Second, assessing follow-through and maintenance can be an indicator of social validity, in the absence of negative reinforcement or coercion. Typically, if people are willing to continue to support an intervention after the initial push has subsided, the initiative is likely to be socially valid. You should measure how well people follow-through and keep working on the interventions you’ve put in place through spot-checks.
Surveys and Focus Groups: Finally, and most popularly, surveys and focus groups can be fantastic tools to assess social validity. Asking people directly how they feel about the goals, procedures, and outcomes of your OBM work is a great way to get information about how the interventions have been received. In the past, I have typically done focus groups at the beginning of an intervention – small groups of employees gathered to respond to questions and provide input on what we’re planning to do. I ask the group to identify barriers, sometimes structured similarly to the Performance Diagnostic Checklist (PDC) to ensure that I address issues with antecedents, equipment and processes, training, and consequences before I begin an intervention. Then, during the implementation, I will send out a social validity questionnaire to all employees to get additional information on how things are going, if any changes are needed, etc. And finally, when I fade out my services, I send an additional survey on how folks feel about the implementation, what we did well, and what we could do better next time. In a future blog, I’ll give deeper into how to write good social validity surveys in a way a practitioner can have clear action items. Behavior analysts tend to shy away from “feelings”, since they are difficult to measure, but embracing someone’s verbal behaviors, tacts of their own feelings, is essential so that interventions can sustain.
I hope this has helped you better understand the role of social validity, especially in OBM practice. Without happy employees who are willing to do what you need them to do, nothing gets done! Prioritize social validity of employees and clients, and the benefits will be endless: significant outcomes, maintenance, and better retention of both clients and employees. If you have any questions, leave a comment below!