While end-of-course assessments will always be the mainstay of compliance training courses, a behavior change-focused approach to measuring the impact of your training could yield more meaningful results and insights. Find out more in this blog from Principal Consultant, Liz Hornby.
On the whole, regulators are not prescriptive about how the effectiveness of compliance training is measured and demonstrated. They simply ask that it is effective. This has resulted in an over-reliance on end-of-course multiple choice assessments.
Time for a New Approach to Measuring Compliance Training
But the increased regulatory focus on culture and conduct perhaps provides a good opportunity for a new approach. It is acknowledged by regulators and practitioners alike that it is hard to demonstrate behavioral change through a multiple choice assessment alone.
In a recent speech on the role of culture in the investment management sector, the FCA’s Chief Executive, Andrew Bailey, emphasized the increasing role that culture would play in the FCA's supervisory approach. We therefore need to adapt our assessment methodologies to match.
That means training effectiveness should not just be focused on measuring knowledge transfer, but on how successful the training has been in supporting a culture where people can, and will, make the right decisions.
In the rest of this blog post, we’ll explore how to design compliance training to target behavior and culture change and the best ways to measure this approach.
Measurable Behavior Change Starts With the Right Design Approach
If your aim is to target behavior change and measure your success on that basis, then behavior change related objectives have to be defined at the outset of your course design.
This means identifying the changes in behavior that you want to see and including them alongside the more traditional knowledge-focused learning objectives. It is impossible to measure behavioral change effectively unless you know what your starting point is and what success looks like. Your training design will then be shaped around encouraging the behaviors you’re specifically hoping to influence.
At LEO Learning, we tend to look at the work of behavior theorists, such as B.J. Fogg, to influence the way we design our training.
B.J. Fogg’s model proposes that three things must be present in order for behavior change to occur: triggers (a prompt to change), ability (i.e. the skills or knowledge to change) and then the motivation to actually make the change.
When it comes to training design, we might translate this model into the following framework:
- Triggers: what will trigger my staff to make changes to their behavior
- Ability: what knowledge or skills do my staff need in order to be able to change their behaviours
- Motivation: what is going to motivate staff to actually make changes to their behavior.
Here’s a simple example of how to apply this model to a training course about speaking up/whistleblowing:
- Triggers: these are essentially the red flags (e.g. the suspicion or concern) that should prompt action in staff
- Ability: this is the knowledge learners will need to know to take the correct course of action (e.g ring the whistleblowing line)
- Motivation: staff seeing it as their responsibility to take action and being willing to do so.
How to Measure Behavior Change in Compliance Training
End-of-course assessments can still be used to measure how successfully your learners have taken on board knowledge, but measuring whether changes in behavior have taken place can involve a range of different approaches.
The key to measuring behavior change is collecting robust evidence that clearly demonstrates that staff have changed their actions and the way that they make decisions.
These metrics should be discussed and agreed during the course design phase. Although it can be tempting to rely on easily accessible data, it’s crucial that any metric has a clear connection to the training outcomes and is sustained over the long term. For example, an increase in Outside Business Interest notifications from staff may indicate an increased awareness of the process rather than a better understanding of how to make good decisions when faced with potential conflicts of interest.
When it comes to measuring behaviors in the workplace, managers play a key role in gathering evidence as they’re best placed to observe staff behaviors on a day-to-day basis. They can also get a feel for changes in the attitudes of their staff which give an indication of changes in the culture of the team and the wider organization. While this is qualitative analysis, it’s an important partner to more quantitative data.
Another more recent approach to measurement is to harness new technological standards, like xAPI, which allow you to pull together data on all the different ways your staff are performing. xAPI allows organizations to gather data on learner performance within both on and offline training as well as on performance within their daily roles. This can lead to valuable insights on how training has influenced behavior change in the organization as it can demonstrate strong connections between training and staff performance.