By: Christina Ravaglia
Many products today are built using Lean processes to decide what to build and Agile practices to build them. With Lean, you release your initial minimum viable product quickly, measure your success, then based on what you learn, prioritize improvements that will be most valuable to your users. With Agile, you then deliver these features quickly and improve your product incrementally to continuously deliver value to your users. Trainers can adopt the same practices to deliver content. Repeating the learn and improve steps quickly maximizes your success in building the most valuable training for your audience, and minimizes your need to do rework.
With online help and training tools today, you can incrementally improve your training and documentation products quickly and easily. And with XML, you can single source information and deliver it in multiple ways optimized for different audiences.
How do you measure success?
Metrics that best measure the training team’s success are those that are tied to long-term profit for your company. When planning your training and documentation, always ask yourself what business problem you are solving for your company.
The following table shows examples of metrics that are closely tied to profitability for the personas and user goals from my last blog post. Using metrics tied to profitability is how you demonstrate the ROI of training and documentation to your organization to ensure continued and expanded funding. You want to see better metrics for users who have received training compared to those who haven’t, either by comparing metrics of the two groups (trained and not trained), or by showing improved trends over time for the same group, and highlighting when training for the group started.
What about surveys?
Surveys also provide important insights. If your company has an effective customer experience program, your company will measure customer loyalty to your brand by sending a survey to each customer semi-annually asking, “On a scale from 0-10, How likely is it that you would recommend (our brand) to your friends and family?” This yields the Net Promoter Score®, which measures loyalty to your brand. The survey then asks an open text question and measures the customer journey – by asking the customer to rate their satisfaction with each touchpoint (key interaction) in the customer journey, for example: purchase, setup, product use, training, and support. With this relationship survey, your Customer Experience team can measure the correlation between training satisfaction and likelihood to recommend. When customers who are highly satisfied with training are also more likely to recommend your brand, you’ve demonstrated that training is a driver for the Net Promoter Score, which has been proven have a financial linkage to long term financial growth.
But what about your training surveys? Survey your users right after they have been trained. The training survey should first ask, “Overall how satisfied are you with the training you just received?” This is followed by an open text comment asking why they provided that score. The survey then asks about their satisfaction with specific aspects of the training such as content, depth of information, exercises, handouts, and so on.
Training surveys are important as a diagnostic tool to fine tune the training you’re delivering. You can correlate the satisfaction scores of the training attributes to see what’s driving overall training satisfaction so you can best prioritize improvements in your next iteration. You can also compare performance of delivery options. For instance, see how your training staff compare to each other, or see which learning environments are preferred by your audience. Be sure to segment results by your customer’s financial segments and operational metrics to answer important business questions. What would most improve scores for your high value customers? When do high scores correlate to lower customer churn?
Quizzes and Certification
Providing quizzes throughout the training process helps your audience retain the information. I also advocate for practical certification exams that measure how well your audience can use the information to meet stated goals. For example, can your implementation staff now set up their own system to specification, or can your sales staff answer product questions during a live demo to an assessment team? You can quickly learn which areas of your training or documentation need improvement if you see students consistently showing a weakness in a particular area.
Iterating
Based on your success metric performance trends and what you’ve learned in your surveys and quizzes, create a list of the improvements you want to make to your training as smaller sized user stories that can each be completed in one or two weeks. For example:
- As a salesperson I want to explain the implementation process so a prospect can understand what they will need to do, or
- As an engineer I want to understand the scripts that delete certain files so I can use them.
Now prioritize this list of user stories to create your “product backlog”. Size your effort for the highest priority improvements, then start implementing the highest priority improvements in your product every week or every two weeks. You might want to group the stories together by theme for delivery. Because they are for the same audience, or are aligned to a new release, use the same delivery medium such as a video. Continue to build, measure, and learn weekly or bi-weekly to reprioritize your backlog and ensure continuous delivery of high value content.
—
Christina Ravaglia leads the Product Training and Documentation team for NICE Satmetrix. Her team delivers all product training and help resources to customers and all internal teams. She has held senior product management roles for 13 years and is a certified ScrumMaster.
Net Promoter Score, Net Promoter, and NPS are trademarks of NICE Systems, Inc., Bain and Company, Inc., and Fred Reichheld.