One of the major compliance-related announcements of 2015 came last September, when the U.S. Department of Justice hired Hui Chen—former head of compliance for Standard Chartered Bank, ex-assistant general counsel at Pfizer Inc., and former federal prosecutor—for its new position of compliance counsel, effective November 3, 2015. This announcement coincided with new compliance guidance in a speech by assistant attorney general Leslie Caldwell that the new compliance counsel is to apply when evaluating programs for effectiveness.

Both the guidance and the appointment are consistent with the DoJ’s renewed commitment to criminal and civil fraud enforcement, and its intent to be more transparent about how decisions are made regarding prosecution and related matters. They also follow the announcement of a new DoJ enforcement policy (the Yates Memorandum) on the importance of holding culpable executives accountable for corporate wrongdoing. Collectively, these shifts signal that the DoJ will be looking more closely at compliance programs to see if they meet minimum standards or are closer to best practices. This requires companies to actually do compliance (and be able to demonstrate that they are doing so) and not simply establish a paper program to say they have an effective compliance system.

Step in the Right Direction

To the seasoned compliance professional, it’s about time. Since the Federal Sentencing Guidelines for Organizations (FSGO) went into effect in 1991, companies and their compliance officers have sought more specific guidance on how their programs would be evaluated to qualify for effectiveness credit.

Following the 2004 revisions to the FSGO, companies were explicitly called upon to “evaluate periodically the effectiveness of the organization’s compliance and ethics program.” Some organizations have undertaken their own reviews; others have brought in third parties to examine their efforts. While the area of compliance program measurement can vary significantly, emerging research identifies some of the outcomes of an effective program. Academic studies have indicated that an effective compliance program reduces misconduct, increases employee reporting, and reduces the likelihood of retaliation against whistleblowers.

However, the compliance community has questioned whether the federal government is delivering on its promise to recognize genuine compliance efforts. Unfortunately, there is little publicly available information to illustrate that the DoJ considers effective compliance programs and how they do so.

The lack of information on whether and how DoJ takes compliance programs into account in real cases was documented in a 2009 study, “Ethics and Compliance Enforcement Decisions—the Information Gap,” by the Conference Board, which found “there have been very few publicized cases of companies that have received credit under … DoJ … policies for having effective preexisting (i.e., in existence at the time of the offense) compliance and ethics programs.” In 2010, the Ethics Resource Center with participation from the two largest compliance/ethics professional associations—the Society of Corporate Compliance and Ethics and the Ethics and Compliance Officer Association—surveyed in-house compliance professionals to further gauge how DoJ cases are perceived. The overwhelming majority said they believed their company’s program could be strengthened if DoJ made the following kinds of information more available:

Descriptions of cases (without identifying information) in which an organization’s compliance program played a favorable role in an enforcement decision

Information about whether specific aspects of a program (e.g., sufficiency of compliance/ethics training, appropriate position of the compliance/ethics officer) played a role in enforcement

Information about cases where a program contributed to the organization’s receiving some other enforcement-related benefit, such as avoiding having to engage a monitor

One of the duties of the new DoJ compliance counsel will be to provide guidance to prosecutors as they consider the enumerated factors in the United States Attorneys’ Manual, including the existence and effectiveness of any compliance program that a company had in place at the time of any conduct giving rise to the prospect of criminal charges. Chen is to assist in evaluating whether the corporation has taken meaningful remedial action, and to help prosecutors develop appropriate benchmarks and measures for evaluating compliance programs.

There has been some criticism of Chen’s appointment, with commentators stating that federal prosecutors already have enough expertise to examine a compliance program on a more granular level. But is that really so? After all, Caldwell’s use and depiction of “metrics” in her speech underscores a fundamental misunderstanding that many have concerning program evaluation and effectiveness.

Understanding what works best will require investment of time and resources into research as well as the development of metrics and the robust exchange of ideas within and across industry groups. Organizations should work together with industry associations, academics, non-profit think tanks, and other experts to undertake this important task.

The standards and processes by which compliance programs are measured for effectiveness are considerably meager. While the DoJ Prosecution Principles provide some factors for prosecutors to weigh—e.g., “Prosecutors should therefore attempt to determine whether a corporation's compliance program is merely a ‘paper program’ or whether it was designed, implemented, reviewed, and revised, as appropriate, in an effective manner”—there is little substantive guidance that informs prosecutors about what steps they should follow to ensure they arrive at the most complete and accurate answers. This leaves government officials in the position of simply asserting the Justice Potter Stewart maxim, that they “know good programs when we see them.” This may or may not be true, and either way, the government can do better, and companies that make meaningful investments in compliance deserve better.

It is exactly right that a prosecutor should want to know whether the program is “well designed,” whether it was “being applied earnestly and in good faith,” and whether the “compliance program works.” But how is a prosecutor supposed to make these assessments? This kind of guidance was welcome and even groundbreaking when it was written. But best practices for compliance programs have since emerged.

Credit for a compliance program need not be a binary “yes” or “no” as it is under the FSGO. Rather, an exemplary program might warrant treatment different from a barely acceptable one. Otherwise, weak programs may be awarded a passing grade, while good programs will not earn the credit they deserve.

What Is a Metric?

While the Caldwell guidelines and Chen appointment are a step in the right direction, they are far from sufficient. Despite Caldwell labeling the guidelines as metrics, they are perhaps better thought of as more specific hallmarks or indicators of what to look for. Again, what remains conspicuously absent is how the evaluation of these indicators are made.

For example, Caldwell includes the following as metrics that the compliance counsel will apply when working with prosecutors to evaluate programs:

Does the corporation ensure that its directors and managers offer strong support for compliance policies?

Do compliance program staff have stature within the company? Are the compliance teams adequately funded and able to access needed resources?

Are compliance policies clear and in writing? Are the compliance guidelines effectively communicated to employees? Are they easy to find, and do employees get repeated training, including whom to contact with concerns?

The limitations of these factors becomes more clear when going beyond a Yes/No measure that an indicator is in place and going to the next level to incorporate the principles-based nature of the FSGO criteria. Reasonable minds can disagree on what certain principles mean, so some rigorous approach of objective measurement is beneficial. How will an evaluator know if compliance policies are clear and effectively communicated? Will a reviewer do more that read the policies to gauge their readability, etc.

Internal auditors have experience and skills in program measurement and evaluation. They employ a more precise view of metrics as quantifiable measures organizations use to track, monitor and assess the success or failure of various business processes. To be effective, business metrics should be compared to established benchmarks or business objectives. This provides valuable context for the values used in the metric and allows business users to better act on the information they are viewing.

Effective Metrics

Although compliance officers are aware of the need to measure program effectiveness by using metrics, many collect and measure only a small number of basic, one-dimensional metrics, such as the number of calls to the organization’s employee hotline, the number of investigations launched or closed, and the number of training courses completed.

Measuring only the most basic quantitative/objective data may fail to inform management, the board, and prosecutors as to whether the organization’s compliance program is effective. Instead, organizations should seek to develop, collect, and measure creative multi-dimensional metrics.

Beyond the quantitative metrics of the type described above, organizations may also wish to consider developing qualitative indicators but with a means for measurement. Such metrics, while perhaps more difficult to gather, can be helpful in evaluating so-called “soft” or “entity-wide” controls that do not lend themselves easily to quantitative analysis. Such indicators are most easily collected through an internal employee perception survey that seeks to understand employee perceptions, attitudes, and behaviors on the company ethical climate and compliance program initiatives. This kind of survey can measure employee perceptions in certain key areas, such as:

Awareness of laws, regulations, and organizational standards

Effectiveness of communication and training efforts

Whether leaders and managers set a good example of ethical business behavior

Fear of retaliation for reporting unethical behavior including through use of a hotline

Degree of confidence that appropriate corrective action will be taken when reporting concerns

Fielding an employee integrity survey can help compliance professionals identify and understand compliance program strengths and weaknesses that would otherwise prove difficult to detect. Importantly, such a survey can also provide management with the ability to benchmark the effectiveness of compliance initiatives over time, through re-administration of the survey.

There are other techniques of program evaluation that internal audit can recommend. For instance, it is always a good practice to compare a standard metric against another to obtain a deeper insight. Compliance staff can measure the rate of employee hotline calls on a compliance concern in one region of the country versus another. The staff can also compare the rate of calls in the first quarter with call rates in the second quarter to trend over time.

What Does Good Compliance Look Like?

In making the case that they have established an effective compliance program, companies with mature programs can provide evidence that they have consistently implemented the elements of a program as defined in the FSGO. They can show data that employee perceptions and conduct, in those areas subject to quantitative measurement, have improved over time. More compliance officers are using scorecards (or more euphemistically “dashboards”) as a shortcut to giving executives and board members information about what is being accomplished by the compliance program and where the organization is at risk. Company leaders want to readily see the cause-and-effect relationship between compliance and risk reduction.

Understanding what works best will require investment of time and resources into research as well as the development of metrics and the robust exchange of ideas within and across industry groups. Organizations should work together with industry associations, academics, non-profit think tanks, and other experts to undertake this important task.

We have very strong intuitions about all kinds of things—our own abilities, how the economy works, what makes a compliance program effective. But unless we start systematically testing those intuitions, at a certain point we’re not going to improve our programs.

Topics