A guide to evaluating your online training
If you don’t know how far you’ve come, you can never know the distance you still have to cover
Measurement is the first step to improvement. This is particularly true in the case of online training, where the instant feedback of face-to-face training isn’t necessarily available.
So how can you best evaluate online learning and development? The good news is that because it’s online, measurement of some aspects is much easier than it used to be. But getting an overall idea of effectiveness – in other words getting the big picture - takes a little more organisation.
In this series of guides, we’re going to explain in plain English
- Why you should bother evaluating your online learning and development
- What you can and should measure
- How to measure it – the tools and techniques you should know about
- What to do next – how you can take your evaluation and apply it
As we go along, we’ll throw out some time-saving tips and tricks, a handy checklist and some general frameworks devised by others that you can apply as templates.
Why should you evaluate?
The knottiest question surrounding evaluation concerns what to evaluate. Thinking through the why first makes that question a lot easier to answer.
To make improvements
This is likely to be the chief motivation for looking at your programme both in detail and as a whole. No online training is perfect, and in any case the needs of the organisation and of participants can evolve quite quickly. Improvements are likely to be:
- pedagogical. ie, changes or additions to the structure of a course so that it meets the needs of learners better. This might involve adding new formats and interactivity, improving the way different topics relate to each other, or adapting the ongoing and final assessments of learning impact.
- technical. such as changes to the underlying technology to improve speed or record-keeping. The availability of open Learning Management Systems means that content can be transferred relatively easy and new functions can be added.
- presentational. alterations to the design, layout or wording. This sort of change may seem superficial but should not be underestimated: a good course can be presented poorly but still lose the interest of learners, so usability, colour schemes, button types and all the rest are important considerations.
To spread best practice around your organisation or further afield
Training may have been computerised since the 80s, but systems and ideas are still being worked out as the technology evolves. Trainers and course designers usually leap at opportunities to understand why a programme worked and what they can learn from mistakes.
To validate the resources devoted to developing your online training
Evaluation should be part of the process that feeds into budgeting and business planning. If a training programme has worked then it’s important to show it and devote future resources to similar ideas. Of course, you shouldn’t try to elicit the result you want (eg by guiding survey respondents into reporting satisfaction by means of leading questions, or through limiting choices unrealistically), but you do want to discover “what works” and, if possible, judge the return on investment.
The training expert Donald Kirkpatrick puts the reasons for evaluation in a succinct way:
- to justify the existence and budget of the training department by showing how it contributes to the organizational objectives and goals,
- to decide whether to continue or discontinue training programs, and
- to gain information on how to improve future training sessions.
When you’ve decided why you’re evaluating your programme, you should start getting a clearer idea of what it is you want to measure. That’s something we’ll look at next week.