For organizations in the business of lifelong learning, it’s essential to consider how to best measure the impact and effectiveness of a learning experience. And one of the best ways may lie in a developing trend from which all learning businesses can likely benefit from – assessment-based certificates, or ABCs.
Robin Gault-Winton, a facilitator, trainer, and instructional designer, is the director of learning solutions at Castle and she’s an expert on how assessment-based certificate programs can help increase professional development opportunities and improve learning outcomes.
In this episode of the Leading Learning podcast, Celisa talks with Robin about the benefit of assessment-based certificates, how they differ from certifications, and ways organizations–particularly trade and professional associations–can use them to demonstrate learning effectiveness.
Listen to the Show
Read the Show Notes
00:18 – Thank you to Castle, which is the sponsor of the Leading Learning podcast for the second quarter of 2017. Castle is an accomplished full-services certification and licensure testing company that also offers its clients a variety of learning solutions capabilities. With an expert team of testing and instructional design professionals and a thirty year history of excellence in it’s field, Castle understands what it takes to develop and deliver quality learning and certification programs.
01:13 – Highlighted Resource of the Week – An ABC Program Focuses on Training and Assessing Mastery of a Specific Set of Learning Outcomes – an article published in the Institute for Credentialing Excellence’s quarterly publication (ICE Digest) that is essentially an ABC 101.
01:58 – A preview of what will be covered in this podcast where Celisa interviews Robin Gault-Winton about assessment-based certificates (ABCs).
03:16 – An introduction to Robin and some background information about who she is and what she does.
04:40 – What are assessment-based certificates, or ABCs? Robin shares that an ABC program (at it’s most basic) consists of a learning event, which can occur in any type of delivery format, designed around sound learning performance objectives. Once the person has completed the training they will then have to pass an assessment (can be done using a variety of different formats) that has been designed to measure whether the learning outcomes were achieved. If the person passes the assessment, they are awarded a certificate (can be paper or digital/digital badge) that indicates they met the performance standard.
06:08 – When and how should assessment-based certificates be used? And, as part of your answer, would you also talk about how ABCs differ from certifications? Robin explains that ABCs differ from certifications because certifications (as defined in the testing world) use assessments that measure your competency as a professional as they’re defined by experts in the field – it’s not the mastery of a specific set of learning outcomes that are gained from a single training such as an ABC. People pass a certification assessment because of their years of experience and knowledge that they’ve gained in their professional role. Certifications also usually have requirements for maintaining them such as continued professional development. On the other hand, ABCs are used to indicate a person has achieved additional knowledge or skills in a much more focused area of practice. For most trade and professional associations, ABCs are particularly useful in offering continuing education opportunities.
08:55 – Talk a bit about the “A” of the equation, the assessment piece. How buttoned up does the assessment need to be? Why might an organization care about having training linked to a validated or psychometrically sound assessment tool, rather than a simple survey or feedback process? Robin stresses the importance of an ABC being grounded in best practices for both the assessment and the training components. This ensures that you are both teaching and measuring effectively so that what learners do afterwards is different than what they could do before the ABC. One way to do this is by having an accredited ABC. There are a couple of organizations that do accreditations for ABCs—the Institute for Credentialing Excellence (ICE) and American National Standards Institute (ANSI). Organizations who choose not to be accredited should still follow best practices to create an effective program – this means learning events should reflect strong learning performance outcomes, validated by subject matter experts to their relevance and accuracy. The assessment part should be tied to those same learning outcomes. In order to do that more effectively, the use of psychometrically sound instruments are much more able to demonstrate that linkage.
11:59 – Robin talks about surveys and feedback saying that they may offer data that someone liked the training but they really can’t prove the learning like a validated and linked assessment can.
13:01 – Along similar lines, when might an organization need or want to use proctors for assessment events? What’s the value of proctors for assessments? Robin explains that proctored exams are especially useful when there is an increased need for standardization or test security. A proctored exam ensures that the right person is taking the right exam. They can also help to prevent cheating. All of this is useful in higher stakes situations or if you want to ensure a higher level of certainty about the ABC program itself. Proctored assessments are often part of an accredited program.
14:57 – I think one of the most interesting things about assessment-based certificates is that they have the potential to quantifiably show that the training is effective—the organization offering an ABC has the assessment results data to draw on. Are you seeing organizations using ABCs to demonstrate learning effectiveness? Or, more broadly, what are you seeing savvy organizations do with ABCs? Robin says it can be particularly difficult for trade and professional associations to demonstrate effectiveness because they have less control over the factors that can improve training effectiveness like an organization who are teaching their own employees. Internal companies can tie their trainings to be strategic with the organization and more easily collect pre and post training data to see if it’s actually making an improvement. Trade and professional organizations still have an obligation to provide training that’s effective as possible. An ABC makes that more achievable because it quantifies that an individual has demonstrated mastery of the skills and knowledge.
17:59 – Robin says what she likes to do in her role is to encourage her clients to take ABCs to that higher evaluation level. This is because many of their groups have pretty good connections with their membership so they can understand the data that results within an individual organization and begin collecting data on a broader level. She also tries to let clients know that while an assessment is a really great start to increasing the positive data in learning, the biggest impact on someone actually using newly learned skills is through reinforcement on the job. Again, this can be tricky for trade and professional associations, but even if you don’t have direct interaction with supervisors (who can coach and reinforce), they can create tools for the supervisors to use to promote that ongoing use of what was learned in an ABC.
20:43 – What common mistakes or missteps do you see when organizations are designing, developing, or rolling out ABCs? Robin reveals the biggest mistake she sees is that people don’t fully understand the level of expertise required to do this well. It isn’t just a matter of putting together training and writing some test questions (as most people in the training world are used to). The more deliberately and expertly it’s done, the surer you can be that your ABC is making a difference. Any full service organization (such as Castle) can help you with this—whether it’s developing the entire program from scratch or assisting with the different components. Robin emphasizes that some people may make the mistake of not getting help in fear of losing control of the project but any good company should be a partner in the process.
22:36 – What’s on the horizon for ABCs or, if you prefer, credentialing broadly? Are there any big developments or changes you think we’ll see in the next few years? Robin anticipates data and measurement will become increasingly more important to show effectiveness and organizations involved in training and education are going to have to be able to show this more and more. Because of that, she thinks ABCs will be become an increasingly popular tool. A development Robin would personally like to see in addition to the increased use of tools by supervisors for coaching is staggering the delivery of the assessment itself. The more the content is reviewed, the more likely it is to get into long-term memory, and therefore be used.
25:06 – How do you approach your own lifelong learning? Robin shares a quote by Brian Herbert – “The capacity to learn is a gift; the ability to learn is a skill; the willingness to learn is a choice” and says she tries to be willing to learn in everything she does.
26:10 – How to connect with Robin and/or learn more:
Article about ABCs/ABC resources: http://www.castleworldwide.com/home/an-abc-program-focuses-on-training-and-assessing-mastery-of-a-specific-set-of-learning-outcomes/ (Note this is also the Highlighted Resource of the Week)
Castle Learning Solutions: http://www.castleworldwide.com/home/solutions/custom-learning-solutions/ (has information about their process and sample e-learning in their portfolio)
Email: rgault-winton @ castleworldwide.com
To schedule a 30- to 45-minute consultation with Robin, go to http://www.castleworldwide.com/home/solutions/custom-learning-solutions/consultation.
27:56 – Wrap Up
Thanks again to Castle, a full service certification and licensure testing company that also offers its clients a variety of learning solutions capabilities, for sponsoring this episode of the Leading Learning podcast.
If you are getting value from the Leading Learning podcast, be sure to subscribe by RSS or on iTunes. We’d also appreciate if you give us a rating on iTunes by going to http://www.leadinglearning.com/itunes.
Also, please tell others about the podcast. Go to http://www.leadinglearning.com/share to share information about the podcast via Twitter, or send out a message on another channel of your choosing with a link to http://leadinglearning.tagoras.com/category/podcast.
29:42 – Sign off