John Helmer reviews the CIPD Learning and Development surveys 2007-2010 to see what they have to say about technology and learning
‘Elearning is the learning and talent development practice that has increased the most, with six in ten (62%) organisations saying they use it more than in 2009.’
The CIPD Learning and Development survey has been an annual event for the past twelve years. I’ve been reading it fairly closely for the last four of those, chiefly with an eye to what it might have to say about technology-supported learning.
It is interesting to see how the view of elearning has changed over that time. Contrast the following statements from successive surveys:
- ‘Only 2% cite elearning as the most effective way to learn’ (2007)
- ‘Perhaps surprisingly, elearning is seen as ‘most effective’ by only 7% of organisations, yet 57% of organisations now use this form of development in some capacity.’ (2008)
- ‘Although only 7% of respondents deem elearning to be one of the most effective learning and development practices, 42% say that they have actually used it more in the last two years.’ (2009)
- ‘It is interesting that although the uptake of elearning continues to rise, it is not highly rated for its effectiveness [12% rated it the most effective way to learn], compared with other practices.’ (2010)
OK, I was kidding. The CIPD line on elearning has changed hardly at all over those four years. It’s the number that has shifted. In 2008 only 2% thought elearning the most effective means of learning, and by 2010 this had increased to 12%. Not a mind-blowing endorsement, perhaps, but significant enough to be worth noting, you would have thought.
Instead, the tone of successive survey reports has consistently taken a glass-half-empty view. 2008’s report devoted an entire section to elearning, which revealed, among other findings, that the membership had largely ‘got’ and accepted the concept of blended learning as a way of using elearning in combination with face-to-face elements of the training mix. The CIPD’s take on this was that, ‘although elearning has grown and is expected to do so further … its effectiveness as a stand-alone tool is not yet proven’. Arguably, by 2008, the success or failure of technology-supported forms of learning – or any other forms of learning – as stand-alone tools was not really the issue. We were in a blended universe by then. It was becoming widely accepted that the best way to get results was through effectively combining different modes of learning delivery, both online and offline.
The CIPD criticism seemed to miss the point, and by now you might have started to get the idea that the organisation was slightly biased against the use of technology for learning.
How many organisations use elearning?
One of the chief things you might have expected to learn from any survey that paid even lip service to the idea of elearning as a legitimate means of learning delivery was how many organisations were actually doing it. But this has not always been easy to ascertain from these reports.
In 2007 (base: 663) the survey said that 14% of organisations used elearning frequently, 34% occasionally, 28% rarely, and 24% never. This gives a total of 76% who used some elearning (although the figure is not explicitly stated).
In 2008 (base: 729) the headline figure given for elearning use was 57%. However, elsewhere in the same survey report respondents were asked how their use of elearning had changed in the last two years, 47% saying they used it more, 2% that they used it less, 20% saying they used it the same, and 26% saying they didn’t use it or no longer used it. This would give a total figure of 74% who used some e-learning – a figure more in line with the previous year’s, although it contradicted the headline figure.
In 2009 (base: 859) a very similar question elicited a very similar result: 42% had used it more, 26% stayed the same, 3% used it less and 26% did not use it or no longer used it. A total of 74% used some elearning, therefore, though again the figure was not explicitly stated.
In 2010 (base: 724) the question was not asked.
With a bit of digging – and taking the 2008 57% figure as a blip – it is possible to deduce that the proportion of organisations using some kind of elearning has remained fairly static over the period since 2007 at roughly three-quarters. What has changed is that they are doing more of it, and that more of them are convinced of its effectiveness. These are conclusions you might come to by extrapolating from the figures, however: it is not a picture that necessarily comes across from the CIPD commentary, which seems at pains to downplay the significance of elearning.
Comparing like with like?
It should be said that the questions asked have varied slightly over the four surveys. Also, for 2010, the CIPD has upped the ante somewhat by repositioning the field of enquiry as “Learning & Talent Development’, and by including international respondents.
Nevertheless, the last two years’ surveys are consistent in showing respondents using significantly more elearning year on year (42% in 2009, 62% in 2010) – a fact that continues to cause some perplexity for the reports’ authors, as we have seen above.
Lack of comparative sources
For many of us, the picture painted by these surveys has felt at odds with our own experience of how Learning and Development departments have adopted the new learning technologies. But something that makes life rather difficult for anyone who wants to question this picture is the lack of comparable, quantitative surveys in the field by anyone other than CIPD.
Towards MaturityMind Tools, has amassed a wealth of data over the years on the impact of learning technologies within the workplace. However, valuable though this resource is, it is largely case-study-based, and ‘qualitative’, focusing on the business impact of technology adoption, so cannot be directly compared with the CIPD surveys. In addition, numerous difficulties face those trying to gauge the true impact of technology on Learning and Development practice.
The most recent fruits of Towards Maturity’s labours is a report from Becta entitled ‘Delivering results with learning technologies in the workplace: evidence for employer efficiencies and tangible business benefits (January 2010)‘. The top-line finding of the report, which contains 50 case studies, is that ‘… employers who measure and report the value of their learning technology investments are achieving both efficiency savings (primarily in time and cost savings) and a variety of business benefits such as increased productivity, sales, staff benefit and learning quality, and reduced carbon footprint’. However, the report has caveats.
Where organisations measure business impact of programmes (and many don’t), the technology element is rarely isolated for scrutiny, the report finds. Further, those who feel they have achieved some competitive advantage through a smart use of new technology are often loath to share the secrets of their success with an audience which will more than likely include their competitors.
LINE’s own experience would add an extra factor that tends to muddy the waters here. Many of the more ambitious, large-scale blended learning programmes LINE gets to work on are not originated and managed from within the Learning and Development department at all, but are part of change programmes with a strategic focus, or emanate from particular business units, or are mounted under the aegis of Organisational Development. In some organisations the latter is seen as not the province of HR, whose developmental responsibilities are often drawn quite narrowly.
All of these restrictions to what can be explicitly referenced, taken together, would seem to indicate that the considerable amount of evidence that Towards Maturity and bodies like it have been able to amass are perhaps the tip of an iceberg.
What can we learn from the US?
The US, being a bigger market, is better served for analysis and research in this area, with well established bodies like Brandon Hall, Bersin & Associates and The E-learning Guild covering the territory in some detail.
Ambient Insight provides regular surveys of the e-learning market in the US and also data about the worldwide supplier market – although this doesn’t go to a level of granularity smaller than ‘Western Europe’, so is of limited use for our purposes. Nevertheless, it is interesting to know that worldwide demand for self-paced elearning products and services (a subset of technology-supported learning) is growing by 12.8% (CAGR) and is estimated to reach $49.6 billion by 2014. Ambient supplies breakdowns of this data by packaged content, custom content development, learning platform and tool hosting services, authoring software and tools, and installed learning platforms. Although recent UK reports by Learning Light (drawing on my own work) and IT Training/BCS are worth a look, no such level of information exists about UK market demand to set against the CIPD surveys.
Perhaps of more relevance and interest is Ambient’s ‘Learning and Performance Technology Research Taxonomy’, which provides a useful starting point for anyone who is having trouble defining exactly what technology-supported learning is in the current organisational context (as all serious researchers know, it’s important to know exactly what it is you’re dealing with before you start framing questions about it). Although there are numerous points of divergence between the UK and the US when it comes to e-learning, not least around language and learning culture, on the whole the similarities far outweigh the differences, and there is much to learn from these sources.
Evidence of this kind of logical analysis – and of the existence of an up-to-date taxonomy of UK Learning and Development practice – is signally lacking from the account given of learning technologies in CIPD surveys, however.
What exactly does the CIPD understand by elearning? And is it perhaps working with an out-of-date definition, given that few practitioners even use the term anymore, preferring to speak of technology-enabled (or -supported) learning?
A worrying disconnect?
Laura Overton’s review of this year’s CIPD survey on behalf of Towards Maturity is diplomatically worded, but speaks of ‘glaring omissions’ in the report when it comes to ‘the transformational use of technology in learning’, and a ‘missed opportunity’ over all.
Part of the problem here is that, as we have seen, when it comes to quantitative surveys of the practitioner Learning and Development community, CIPD’s reports are really the only game in town. Towards Maturity has a different remit, focusing on collecting evidence of business benefits from technology, and is forced to lean on the CIPD’s figures in ‘scoping out’ its field of study. There is scant evidence from the CIPD surveys, however, to show that the Institute has more than a superficial knowledge of the territory that Towards Maturity covers in detail.
There is a worrying disconnect here.
Laura Overton points out that the role of technology in learning is hardly referenced outside the term elearning in the latest CIPD report, and also that while the survey relates that ‘in-house programmes are on the increase and are deemed to be the most successful Learning and Development practice, you would never guess that technology had anything to do with this’. Overton’s view, as someone who does a lot of work directly with organisations (Towards Maturity’s website carries more than 100 employer stories), seems to be that a lot of technology-supported innovation is being allowed to hide in the figures here.
Conclusion
The CIPD might well feel that it is criticised unfairly for the perceived shortcomings of its research in an area (technology-supported learning) which is still fairly niche, and which it evidently believes is not of major importance to its membership.
There are indications, however, that Learning and Development departments, not to mention HR as a whole, lag behind the organisations they serve in this very area. During a roundtable discussion organised by LINE in late 2009, business leaders from global corporates reported that skills issues among those delivering learning and development are now a bigger block to adoption of new learning technologies than either board-level management or learner acceptance.
If this admittedly small group is representative, then the CIPD, as ‘the keepers and maintainers of the one qualification which HR professionals have got to attain’ (as it was described to us by a CIPD member we consulted in the course of preparing this article) would perhaps be serving its membership better by adopting a less dismissive stance towards technology-enabled learning and communications, and by showing itself to be better informed about what is going on at the workface, both here and abroad.
Discover how LEO Learning can help transform your learning strategy and learning content through effective use of the latest learning technology.