Six veteran CCOs, 5 questions: Improving your compliance training

1. How do you measure the effectiveness of compliance training?
2. How do you make sure your compliance training is engaging?
3. What advice would you give to a practitioner looking to improve a training program?
4. What differentiates a “check the box” training program from one that’s truly effective?
5. What will compliance training look like in 5 years?

Their answers are below.

1. How do you measure the effectiveness of compliance training?

CÉDRIC DUBAR: We observe learners´ reactions during class—do they ask questions? Do they participate? And we ask their opinions, which helps us adapt the training, in real-time if needed. Using dilemmas, quizzes, simple tests, case studies, role play, etc. during the training ensures that a given learning objective is understood.

MATT GALVIN: If the training is to raise awareness, we look for an increase in reports on a certain issue. For example, from our new conflict-of-interest policy and training, we saw a surge in the number of gifts put up for our yearly holiday charity auction that would have otherwise violated the policy. We also borrowed from our marketing teams and measure the net promoter score for people that take our modules—we scored 4.6/5.  

ANA IACOVETTA: We look at direct correlations between the training content and resulting activity levels. For example, if we cover conflicts of interest, do we see an uptick in the number of conflicts disclosures submitted for review?

DIANA JAGIELLA: We measure effectiveness through testing, audience feedback, audits, and company compliance performance. Proficiency testing on key points should be part of every training. Similarly, audience members should be asked to complete a feedback form asking whether they understood the training and how the principles apply to their job.

EILEEN MCCARTHY: We see the ultimate effectiveness of compliance training in our day-to-day interactions across the company—someone sees something that might be off and reaches out to report it. Folks in meetings are comfortable respectfully correcting their peers.

CATHERINE MUSTICO: Aside from the obvious effectiveness of procedures and policies being followed, I measure this by how well the spirit of the regulation is understood. I also find that when they understand, they loop me in early—before there is a problem instead of after.

---

2. How do you make sure your compliance training is engaging?

DUBAR: Avoid boring compliance lectures and keep formal training short to avoid training fatigue. Be engaging by including interactive elements (role play, dilemmas, group projects and discussions, real case scenario, games, etc.). Learners are more interested if we help them solve their problems.

GALVIN: The keys to engagement are relevance, brevity, and interactivity. Our trainings are interactive, animated, choose-your-own-adventure style videos. There are times when all of the possible answers are wrong, others when they’re all right. That way we elevate the experience and make it difficult for people to just click through.

IACOVETTA: Make sure the delivery is fast moving, interactive, relevant, and relatable. Use interactive touchpoints selectively throughout the training as a way to keep them focused—for online training, this can be done by including virtual flip cards or image clicks on the screen; polling the audience works well in live training. 

JAGIELLA: I open by sharing a real company event that illustrates how compliance dilemmas arise in the workplace. The more startling the event, the better. Relatable stories help the audience understand how this disconnect occurs under business pressures.

MCCARTHY: We like to use humor and real life (deidentified) examples—especially if the examples have gotten public media coverage or, if an internal issue, the issue has become known throughout the company. … If the latter, we still deidentify the individuals involved.

MUSTICO: Comedy. Pure and simple. Whether it is inserting into the Business Continuity Plan and Policy a provision for a zombie apocalypse (they can’t kill the Chief Compliance Officer even if I turn) or some other ridiculous scenario, it keeps an otherwise boring subject mildly entertaining.

---

3. What advice would you give to a practitioner looking to improve a training program?

DUBAR: Create a matrix of compliance learning needs based on organizational risk, role/task risk, and learner analysis. Don´t forget to focus on company culture! Break up your materials into small, easily understood pieces; consider the delivery order for your trainings; and integrate interactive elements like storytelling and scenarios etc. Make training repetitive to anchor knowledge.

GALVIN: Show scenarios that depict things specific to your field so your employees feel spoken to directly. And make sure they are brief. When you’re trying to educate, it’s best to give the edited version. Lawyers tend not to excel in this, so we scoured the market and ended up leaving the compliance space. We brought in Labyrinth Training, an interactive design company with a background in storytelling, editing, and entertainment that has made a real impact. 

IACOVETTA: Invest time and resources up front to understand the audience you are trying to reach, and what blockers or biases you need to overcome to get them to be receptive and engaged. Do not assume that because you’ve been working in the same company for many years you know what your audience needs or wants, or what approach will resonate with them today.

JAGIELLA: Improve content relevance. Customize commercially purchased online training to make it germane to your business operations and audience responsibilities. Openly discuss the threat to compliance arising from extreme business pressure, priorities that conflict with stated values, and inconsistent discipline. When faced with a difficult compliance issue, those attendees will be more likely to seek your guidance because you “get it.” Adding some humor, even if somewhat irreverent, keeps the audience with you. 

MCCARTHY: Even the most engaged audience can suffer from training fatigue. Include different types of items to avoid boredom—including questions, pop ups, skits, etc.  It should also be brand compliant (look, feel, use company lingo, if you have them) so the folks taking the training know that it is company specific.

MUSTICO: Resources and attention spans are finite. Make the best use of your training program by applying a risk-based approached based on the areas of deficiency highlighted during any audits and/or exams.

---

4. What differentiates a “check the box” training program from one that’s truly effective?

DUBAR: A training program is effective if learners apply their knowledge in their daily work. An efficient compliance training program is not only about applying compliance knowledge, it is also about understanding company values and culture. Our managers are expected to be role models and promote ethical conduct and perform mandatory “responsible business discussions” with their team members.

GALVIN: Too often, trainings are used as a basis for compliance to try to show that it just made an effort. In contrast, we want to know that our employees came out of the training experience armed to change their own behavior and to know when and how to report troubling behavior they see elsewhere. I like to think that the characters in our trainings are memorable mostly because people don’t want to end up in the same tough spots the characters find themselves in.

IACOVETTA: Often, trouble occurs in situations that cannot be addressed by a linear step-by-step approach or where there is no clear and obvious path to take.  Effective compliance training assures that the audience has the tools necessary to identify potential trouble spots and navigate them effectively by applying the principles and values that underpin specific rules.

JAGIELLA: “Check the box” criticism often stems from tedious, uninspired formats and “dry” content. An effective training program will inspire the audience that compliance is our common mission.

MCCARTHY: The truly effective training resonates with learners in a way that sticks with them. It isn’t too highbrow or too antiseptic. And we want them to know who to call for help if they find themselves in a tricky ethical situation.

MUSTICO: Thinking a little outside the box, by putting a little more focus on the why vs. the what, is important, as is focusing on what they can do, instead of always what they can’t. Also, I’m not aware of any rule that states your compliance manual has to be boring.

---

5. What will compliance training look like in 5 years?

DUBAR: The face of compliance training is changing at high speed thanks to artificial intelligence. AI identifies a learner´s needs, knowledge, and preferences and, thus, can adapt training curriculum and instruction in real-time. Trainings are short and more targeted towards what the learner doesn´t know.

GALVIN: The best companies will bring in expertise from elsewhere—whether that will be behavioral scientists, editors, animators or illustrators. I know memes are popular today, but I am quietly hoping that this is not a long-term trend. My overall hope for compliance training is that it becomes a thing that everyone looks forward to doing, as opposed to being something employees dread. It’s entirely on us in the compliance community to deliver our policies to staff in creative, engaging ways.

IACOVETTA: AI and machine learning will help compliance teams deliver compliance training in a way that adapts in real-time to the specific person taking the training. Machines will be able to account for the person’s job role and scope, level, tenure. The depth of content coverage and difficulty of the training will adjust as the learner progresses through the training.

JAGIELLA: Advances in technology, along with improved understanding of adult learning, will make dynamic, inspirational training the norm, not the exception. We will be able to make every training session an engaging personal experience.

MCCARTHY: I think compliance in five years with have more bells and whistles (VR, maybe holograms!), but it will also be human in a way that the best training is now: stories about people.

MUSTICO: Compliance training will have to evolve to be more customized. There will eventually be an AI system that can learn what the adviser is doing and specifically train to the regulation. If my car can buzz my seat because it detects that my eyes are no longer on the road, it doesn’t seem farfetched for compliance to get a report of how much the trainee actually looked or read the material.