Are you in compliance?

Don't miss out! Sign up today for our weekly newsletters and stay abreast of important GRC-related information and news.


Status message

This is subscriber-only content, you are viewing with temporary unrestricted access. For full access, begin your free, no obligation 5-day trial.

Shop Talk: Metrics for Risk, Compliance

Jaclyn Jaeger | December 8, 2009

Thirteen executives participated in the Nov. 17 roundtable presented by Compliance Week and OpenPages where the discussion centered on what metrics to use when measuring risk and compliance. The roundtable, held at the Plaza Hotel in New York City, was moderated by CW Editor-in-Chief Matt Kelly, and featured Michael Duffy, President of OpenPages. Panelists were encouraged to discuss the challenges they face when measuring risk and what metrics they have employed for top-notch enterprise risk management. The following article provides readers with an in-depth look at their discussion.


The following executives participated in the Nov. 17 roundtable on measuring risk and compliance.

Terri Bourne,

Programme Manager, Ethics & Compliance,

Shell Oil

Laurie Brooks,

VP, Risk Management & CRO,


Michael Duffy,

President & CEO,


Fred Dye,

Director of ERM & Control,

Akamai Technologies

Karen Griffin,

Chief Compliance Officer,

Visa Inc.

Maggie Hayes-Cote,

SVP, Assistant General Counsel-Compliance,

Textron Financial Corp.

Joe LaRosa,

VP, Global Compliance & Legal Affairs,

Avon Products

Frank Manzi,

VP, Internal Control,

Omnicom Group

Robert Metcalf,

VP, Global Ethics & Compliance,

Eli Lilly

Richard Muzikar,

Director, ERM,

Consolidated Edison

Ken Robinson,

VP, Finance, Global Internal Audit,

Procter & Gamble

Matt Tanzer,

VP & Chief Compliance Counsel,


Suzanne Thoeni,

Director, Enterprise Compliance,

Wyndham Worldwide

Corporate compliance officers everywhere know that they need real, practical metrics to measure risk and compliance—and, apparently, the actual chore of measuring isn’t that difficult once you decide what metric to use.

It’s that deciding part that has compliance officers tied in knots.

“There is no shortage of metrics out there,” Kenneth Robinson, vice president of global internal audit at Procter & Gamble, said at a Compliance Week editorial roundtable on the subject. “The key is to find the right ones to measure.”

Robinson was one of 13 compliance and risk executives who attended the Nov. 17 event, co-hosted by Compliance Week and OpenPages. He and his colleagues all said they worry most about ensuring they know what they’re supposed to track, especially to measure the unique risks their individual companies face. The “how” of measuring risk and compliance data is then a much more mechanical process.

Indeed, most of the panelists described a fairly hum-drum list of metrics they use to measure Sarbanes-Oxley compliance—hotline calls, training completion rates, employee certifications for the Code of Conduct, and so forth—precisely because SOX has now been a compliance worry for five years. One executive even said his company measures “you know, all the usual stuff” for SOX compliance.

Metrics for risk were another matter. Panelists agreed that boards and audit committees are more interested in the processes that companies are using to manage their risks than they are in the actual metrics.

“As compliance professionals, it’s our responsibility to provide the appropriate level of detail to keep directors informed about the compliance system to help them carry out their oversight role,” said Karen Griffin, chief compliance officer at Visa. “It’s about striking the right balance and working with them to understand the depth and breadth of the information desired.”

Richard Muzikar, director of enterprise risk management at Consolidated Edison, said much the same. “What we’ve tried to do is improve, create, and enhance processes around risk mitigation,” he said. Rather than provide the audit committee with the metrics, the discussion instead focuses on Con Ed’s top 10 risks and how they’re being managed, he said.

Another executive contended that in many instances, less data is more useful. “We do a huge disservice to the board when we give them all this raw data without proper context, because their job is oversight,” he said.

Michael Duffy, CEO of OpenPages, agreed. “Boards care a lot about consistency in reporting so they can understand the changes over time,” he said. “In many ways, ensuring you have a programmatic approach to reporting is as important as the metrics themselves.”

Metrics vs. Processes

All that being said, companies do still need to create specific metrics to track their risks, operations, and compliance. Some are straightforward. Akamai Technologies, which runs servers and software to help companies deliver Web content more quickly, defines its key operational risk metric as network availability for its customers, said Fred Dye, Akamai’s director of ERM and control.

Consolidated Edison, in contrast, tracks roughly 25 operational risks in its electric, gas, and steam businesses, Muzikar said—right down to the metric of manhole cover explosions per year.

Tyco has a “very large project underway” to track its third-party suppliers and evaluate the risks that each presents, said Matt Tanzer, the industrial giant’s chief compliance counsel. (Tyco has thousands of suppliers all over the world.) “We are measuring the risks to our businesses from those relationships,” he said.

Every Tyco supplier gets a risk score based on more than 100 different criteria, Tanzer explained. Higher scores mean the supplier is a higher risk, and Tyco either does more due diligence or cuts ties. And once Tyco determined which of its suppliers posed high risks, it decided it could live without a good number of them, Tanzer said.

Energy services company PSEG has encouraged employees to consider risk on a holistic level. For any new project the company might undertake, its backers must consider the compliance, strategic, and operational risks and then consult with various corporate functions—supply chain management, environmental health and safety, enterprise risk management—to be sure all those departments believe the risks are properly addressed. All that must happen before the backers can go before PSEG’s capital review committee to get the money to proceed.

Robert Metcalfe, head of ethics and compliance for Eli Lilly, makes a point as Fred Dye of Akamai Technologies looks on.

Terri Bourne of Shell Oil speaks up; at right is Richard Muzikar of Consolidated Edison, at left is Ken Robinson of Procter & Gamble.

OpenPages CEO Michael Duffy offered some advice and best pratices.

“All three risk types interact,” said Laurie Brooks, PSEG’s chief risk officer. “If you look at any one of them by itself in isolation and forget the other two, the project could fail.”

Others said they tie risk mitigation into their Codes of Conduct. Visa, for example, employs a compliance risk assessment focused on its code, which itself derives from the U.S. Sentencing Guidelines. “Linking potential risk with the internal control environment is essential to overall mitigation,” Griffin said. “The program is aligned with the U.S. Sentencing Guidelines and provides reasonable assurance the company is in compliance with legal and regulatory requirements.”

At PSEG, the senior leaders of each department were asked to catalogue their top 10 risks, which generated a list of 97 risks across the whole enterprise, Brooks said. Now she plans to compare that list of 97 risks to the risks PSEG formally discloses in its Form 10-K annual report, “and map all the other risks to it,” she said. Ideally, the goal is to narrow that list of 97 risks down to the top 10, and present those to the board.

And speaking of presenting risks to the board: Panelists offered numerous ways to handle the task. At Consolidated Edison, both operational and administrative risks are evaluated quarterly, and reported to the audit committee on annual basis.

At Eli Lilly & Co., management reports on a quarterly regular basis to both the public policy committee and the corporate compliance committee, “because our board views compliance as a public policy mistakes issue, as well,” said Robert Metcalf, Eli Lilly’s vice president of global ethics and compliance.

Enforcing Ethics

Panelists also discussed the struggle of adhering to ethical standards in the daily scrum of business decisions. Textron Financial has drilled its ethical expectations into employees’ heads by embedding those expectations both into its Code of Conduct and into its performance management system, said Maggie Hayes-Cote, the company’s assistant general counsel for compliance. That helps management reduce risk by holding high-risk employees accountable for their actions, she said.

Automation tools can also play a role. Tanzer said Tyco is currently evaluating the feasibility of continuous monitoring of unusual transactions or payments. He gave one real-life example of an employee who accidentally e-mailed a draft Securities and Exchange Commission filing to someone outside the company before the document was ready. While an “honest mistake,” Tanzer said, Tyco’s monitoring system immediately caught and flagged it. “These kinds of systems can have great utility for all kinds of purposes,” he said.

Another challenge panelists worried about is the swift rise of social media, and how closely corporations should monitor, or control, what their employees say. “This whole area of monitoring is just going to rise to the top,” Robinson said. “Where do you draw the line between the greater risks for the company versus the culture?”

Roundtable panelists also said they are searching for ways to compare compliance and risk measurements against industry averages. Griffin wished for some industry group that could provide pooled metrics by industry, so companies can gauge where they need improvement. “There’s not a lot of really timely benchmark data out there across the full compliment of metrics,” she said.

“At the end of the day, it’s the results of the metrics—the measure of overall effectiveness—that is key as it helps to focus resources and drive improvement,” Griffin said. “It’s important to understand what the data is telling you and how you compare to other companies in the industry.”