Welcome to the completion of week nine as we progress through our 12-week service course, Instructional Design Service Course: Gain Experience For Good. We are entering into the final stages of this course, and at this point, have entered into the Evaluate phase of their design process.
Evaluation Phase
Last week we developed a prototype so we could have a framework of our course in its early stages. This week, the focus is on formative evaluation to gather feedback and further develop the course.
As always, I’m trying to tie elements of our course the business practices and experiences we have here at ThinkingKap, so this was a great opportunity for me to discuss the three stages of evaluation that we use here at ThinkingKap.
As seen below, Evaluation is step six in ThinkingKap’s seven-step process. To ensure the proper feedback at the proper times, we employ formative, summative, and confirmative evaluation.
Before we go further, we should probably define the three types of evaluation. The main difference between the types are the timing of when they occur, and thus, the main focus of the type of feedback they are trying to capture.
Formative Evaluation
Formative evaluation is performed while the product is still under development. This is typically coming in the form of ALPHA reviews, and BETA reviews. This feedback focuses on any bugs, content errors, misalignment with client expectations, cosmetic issues, navigational issues, etc.
Most people are familiar with formative evaluation and naturally use it as part of the process when getting approvals from clients for work performed. This evaluation is typically performed by someone “on the inside” that is familiar with the process do notate whether the in-process product is on target.
Summative Evaluation
Summative evaluation is typically performed when the product is finished, but before it is widely adopted and implemented. This can often take the form of user testing, pilot courses, dry runs, etc. Here the intention is to measure whether the outcomes of a given course are achieved at the end of the course.
Stakeholders (insiders) have been evaluating and modifying the course through ALPHA and BETA review cycles to meet their expectations. This is the stage where end users (outsiders) take the courses and determine whether it hit the mark. This typically focuses on whether the content flow makes sense, if navigation is intuitive, whether content sufficiently prepares users for assessments, etc. Since the product is essentially complete at this stage, the hope is that only minor tweaks are needed and no major course corrections are required prior to mainstream launch.
Confirmative Evaluation
Confirmative evaluation is “newer” than the other two forms and is unfortunately conducted less frequently. This stage of evaluation is conducted after the product has been implemented and in use for a period of time. The main focus on confirmative evaluation is whether the training provided produced a transfer of knowledge in a way that the knowledge is being applied back in the real world. While this is the most difficult (and costly) type of evaluation, it gets at the heart of why the training was created in the first place.
When there is a knowledge or behavioral gap, training is used to bridge that gap. The ultimate measure of the success of the course is whether users have used what was learned to bridge the gap and improve their performance. Because of this, people familiar with both the course itself and the knowledge/behavioral requirements are required to conduct a proper review. The behavior outcomes back on the job need to be analyzed up against the objectives of the course.
This process helps determine if the objectives of the course are still aligned with the required behavioral changes. It also assesses whether the course indeed achieved the behavioral changes desired.
Making Sure You’re On Track
During this stage of the process, you and the client can make sure that not only objectives are being met, but that design and copy are hitting the mark for the desired look and feel of the course. This stage is crucial for ensuring client satisfaction; so don’t skip over it in a hurry. Take the time to work through any issues and deliver the best possible course.