We've always known that the majority of learning is an iterative dynamic process that takes place mostly in the workplace and is enabled by practice and constant interaction with other people. By the very nature of its informality, it is essentially a one-to-one process that impacts the individual but not the workgroup as a whole.
The evidence of its effectiveness is vital because it can be used to persuade your organization to invest in the emerging tools—and behavior change—that will enable the wisdom of one person to be shared not just with the individual alongside them, but with the whole organization.
In a recent webinar, our team of learning consultants explored with the senior data analyst from the InterContinental Hotels & Resorts Group’s (IHG) how they are measuring the impact of learning.
The IHG Measurement Story
IHG, one the largest global luxury hotel brands, manages more than 5,500 hotels in 100 countries. The company’s strategic objective is to double the pace of growth by 2020 and with 400,000 staff, this will put enormous pressure on the global L&D department to keep knowledge and skills up to date.
To ensure IHG get the best value from the investment they’re making in training, it was decided to introduce a comprehensive impact measurement strategy. They sought LEO Learning’s help to put the strategy in place. This included co-designing and delivering a learning program for IHG’s 100 strong global Learning & Development team, to help them understand how to measure impact and how to use the tools and techniques required to gather the critical data.
How Do You Measure Learner Engagement?
Measuring learner engagement is crucial when it comes to looking for correlations between learners who are invested in their training, and their subsequent impact on business performance.
But how do you actually measure something that is arguably subjective? For most L&D departments, the focus is generally on measuring attendance and satisfaction scores. This is often dismissed as ‘happy sheets’. In fact ‘Level 1’ data can be incredibly useful as we move towards learning architectures that combine instructor-led training and informal peer-to-peer learning.
Examples of IHG’s Learner Engagement Metrics
One of the stories from IHG bears this out. You will hear in the webinar that IHG already had a small learning analytics team in place. They had been focusing on measuring the impact of one of the key programs: a learning and development course for the hotel group’s general managers.
The design of the solution was ground-breaking because it used a learning experience platform that had recently been installed. This enabled the use of video-based peer-to-peer knowledge sharing. General managers were encouraged to record on camera the initiatives they had put in place that made a difference to guest satisfaction ratings and revenue.
The platform also delivered videos that had been produced by the training team and subject matter experts. The analysts looked at the number of views each video had garnered and it emerged that the most popular were those recorded and shared by the general managers themselves.
This insight emerged from a comprehensive set of data that included learner information such as:
- Learner registration/attendance
- Learner progress
- Learner satisfaction
- Self-assessment (pre- and post-workshop)
- Training credits
In addition, they also tracked engagement in the social elements of the program and responses to the user-generated content. Measures included:
- Likes
- Comments
- Shares
By combining this, it allowed for a thorough bank of data to be built, providing a rounded picture of the degree to which the general managers engaged with the content.
The graphs showing which ‘learning nuggets’ the GMs had viewed and the spikes in interest in their own videos quickly became evident. The data was fed back to the general managers to encourage them to tell stories of what has worked well to improve customer satisfaction.
They also collected the critical business performance data for the hotels where the general managers worked and this included revenue and guest satisfaction.
Relating the Learning to Business Impact
With the data assembled in Excel spreadsheets, they then looked for correlations. The results were fascinating. As described earlier, it showed that the general manager videos were the most popular. It also revealed clearly that the improvement in revenue and guest satisfaction was greatest for those general managers who engaged most with the program.
The analysis did not claim that the L&D program was the only cause of the improvement. It was also noted that those general managers who didn’t engage with the training at all also showed a small improvement in business KPIs, so there were clearly other factors at play that were driving the improvement.
However, they could say with confidence that there was a correlation between the program and improved performance – and one of the key ingredients was the learning content provided by the general managers themselves.
Embracing the Power of Peer-To-Peer Engagement
As learning platforms become more user-focused, the introduction of Learning Experience Platforms such as LTG’s Instilled, is enabling a growing emphasis on user interaction.
This means that user-generated content can be uploaded, viewed and discussed between peers. Functionality such as comments, shares and likes encourages users to interact using techniques that are familiar from social media platforms such as Facebook and YouTube.
Interaction between peers can prove to be a powerful engagement tool—it offers learners the benefits of social learning, such as questioning, benchmarking and learning from colleagues.
An additional factor that can make user-generated content more engaging is the feeling of empowerment that comes with being able to upload unique content. It’s a chance for individuals to express opinions on topics in which they have expertise.
Peer-to-peer interactions can be engineered into the design phase of any learning program by creating authentic, meaningful scenarios and tasks that drive conversation between groups of learners.
Success Breeds Confidence
Although the measurement activity at that time within IHG was focused on just a few programs, the data they collected gave people confidence that there would be great value in scaling up the activity.
The impact measurement team, with LEO Learning’s assistance, has now standardized the processes and tools, and is embedding measurement in the day-to-day task of designing and delivering great learning.
In the future, they will be using data to refine the design and, in turn, improving the contribution the L&D team can make towards IHG’s target of doubling growth by next year.