In international development we do a lot of trainings and workshops. I’ve lost count of all the trainings I’ve either sat through or delivered myself. But is there any evidence that these initiatives are actually effective? And how can we improve effectiveness in the future?
Does training actually ‘work’?
Reflecting on my own experience as a trainer, over the last few years I’ve had a nagging doubt about the long-term impact of the trainings I’ve delivered. To take one example, with various colleagues I’ve delivered multiple rounds of Market Systems Development training to donors and implementing partners. The training has always been well received and end of training evaluations consistently show high levels of participant satisfaction. However, when I or my colleagues would later engage with the organisations and programmes whose staff we had trained, we were often disappointed by the seemly low application of the key tools and frameworks taught in training. We sometimes found isolated cases of individuals applying some of the tools or insights, but few examples of wholesale adoption within the programme or organisation.
This apparent lack of impact from trainings has re-emerged in several recent research and evaluation assignments we’ve led. For example, at Tandem we recently concluded a research assignment for CDC Group on the most effective use of Technical Assistance and Grants to Financial Institutions. Speaking to several training providers, even they were sceptical that training, by itself, is sufficient to catalyse real change in financial institutions. Here’s one illuminating quote:
“What technical assistance providers typically do is deliver a training. Training is focused on skills and knowledge, passing along knowledge, teaching how to do things. The theory is that once staff have that information, they know how to do it, they will do it. But this is a flawed assumption! Real change requires a big strategic focus shift. Every aspect of your business gets touched… so requires a wider range of technical assistance [than just training].”
Similarly, Tandem recently conducted an evaluation of a knowledge hub designed to support reform in Low and Middle Income Countries. A key pillar of their support is a 5-day workshop, which to date has trained typically mid-level government officials from over 20 countries. By all accounts the workshop is excellent, with very high scores from participants, and clear improvements in participants knowledge when measured with a pre and post questionnaire administered at the end of the workshop. However, for the 17 countries receiving no further follow-up support, the Hub’s own monitoring data recorded only one case of this knowledge actually being applied in practice.
These issues are not unique to development. For example, in a 2010 article in McKinsey Quarterly, only one-quarter of the respondents to a McKinsey survey said their corporate training programmes measurably improved business performance. Similarly, in a recent article in the Harvard Business Review, the author Steve Glaveski notes that despite companies spending $359bn in 2016 on staff Learning & Development (L&D), only 12% of employees apply new skills learned in L&D programmes to their jobs. He refers to pioneering experimental studies of memory in the late 19th Century by German psychologist Hermann Ebbinghaus, culminating with his discovery of “The Forgetting Curve.” Ebbinghaus found that if new information isn’t applied, we forget about 75% of it after just six days!
So, does this mean that training and workshops are a waste of time? And how can we improve the effectiveness of training initiatives in the future?
The importance of follow-up
One theme that emerged in both the CDC research and the knowledge hub evaluation is the importance of follow-up. The training providers we interviewed for CDC pointed to the need to combine training packages with follow-up mentoring, coaching, and other forms of support in order to drive sustained practice change. In the case of the knowledge hub, although only 6% of countries that attended the workshop alone had applied the knowledge learned, this increased to 80% for countries that had attended the workshop and received follow-up Technical Assistance (either virtual or in-country). While there may be some sample selection bias at play – the follow-up TA was directed to those participants most engaged at the workshop and who were therefore more likely to apply the knowledge anyway – in interviews workshop participants confirmed the importance of the follow-up support in helping them to apply the knowledge in practice.
This points to an alternative potential benefit of training: even if the training doesn’t generate much impact by itself, it can be a useful way of identifying motivated and engaged individuals and organisations who will benefit from further support. UNCDF, interviewed as part of the CDC research, calls this process ‘winnowing’. For example, UNCDF start off by providing market engagement training to a wide cross-section of financial institutions. Within the training, UNCDF holds small competitions to solicit ideas. Based on the competitions a smaller number then receive some initial technical assistance around strategic planning, or specific focus areas. The primary goal is to ‘get to know’ the financial institutions and to rank them based on how they perform during the short-term engagements. From this, UNCDF narrows down the list down further, only working with financial institutions with the skills, interest and leadership to move forward.
Engaging the leadership, and reinforcement in the workplace
Reflecting on my own experience of delivering training to programme staff, it seems that we got better results when we trained all the staff on a particular programme together, including the programme leadership, rather than delivering a training to multiple organisations at once, where only one or two individuals (often more junior or mid-level) come from the same organisation.
This chimes with two of the recommendations from the McKinsey article. Firstly, they recommend getting the leaders on board:
“To ensure that the lessons stick when training ends, companies must have meaningful support from the relevant leaders beforehand. This point sounds obvious, but we’ve seen many training programs stall when leaders agree with program goals in principle yet fail to reflect them in their own behaviour, thereby signalling to employees that change isn’t necessary.”
This then enables a second recommendation – reinforcing the new skills in the workplace:
“Participants rarely leave any training program entirely prepared to put new skills into practice. Old habits die hard, after all, so reinforcing and supporting new kinds of behaviour after they are learned is crucial.”
A key element of getting training to stick is providing on-going reinforcement in the workplace, which is much more likely to happen if everyone on a programme, including the leadership, has received the same training, rather than isolated (and more junior) staff. Reflecting this insight, one of our co-created recommendations in the knowledge hub evaluation was for the Hub to more actively engage senior ministry officials, before, during, and after the delivery of Hub training. For example, this might involve engaging with senior officials prior to a workshop to: understand why the senior official is sponsoring attendance by their staff, explain what the attendee will be learning at the workshop and what the benefits will be to the organisation, and agreeing a plan for how the senior official will support the attendee to share and apply the knowledge on return.
Conclusions
Covid-19 put a pause on many training initiatives and workshops (at least the face-to-face ones). Once restrictions begin to lift and we starting planning our next training, before committing all that time and money, it’s worth just taking a pause and thinking: what new behaviours or practice changes are we hoping to bring about, is training really the most effective way to do so, what follow-up support might be needed, and how can we reinforce application in the workplace in those crucial first few weeks – before 75% of what has been taught is forgotten!