In continuation of the discussion of compliance program “effectiveness” and the challenges of metrics and measurement, is the concept of benchmarking—an oft misunderstood term. Simply put, it is the process of comparing one’s own business processes and performance to industry standards and peers to determine a relative degree of success.
Occasionally one can still find reference to a board director or company executive stating, “We decided to “benchmark” our compliance program, but actually meaning “We brought in a consultant who linked the elements of the Federal Sentencing Guidelines to our program, gave us a grade, and then talked to us about what’s going well and what could be improved.”
Although we don’t need to get tied up in the semantics, it is important to keep distinct benchmarking from other processes involved in a program evaluation. Benchmarking is a discrete process from the assessment or audit of the effectiveness of a compliance program. While benchmarking can be part of a program evaluation, by itself it does not comprise an evaluation and determination of effectiveness. It does, however, enable organizations to develop plans on how to make improvements or adapt identified leading-edge practices, usually with the aim of increasing some aspect of performance.
Benchmarking already seems to be an implicit feature of a program evaluation, whether conducted by a prosecutor in deciding if a company’s compliance program is so deficient that criminal prosecution may be appropriate, or performed internally when a board direct asks how the company program compares to competitors. In the event of a compliance failure, government investigators are said to compare the organization’s compliance program to those of similar organizations (in terms of size, complexity, industry, geographic footprint, etc.). Companies whose programs are not comparable to those of their peers are more likely to be found ineffective and could be subject to harsher penalties.
Recent developments, including the Department of Justice creating a new position and hiring a compliance counsel, suggests that government authorities will be looking more closely at compliance programs not only to see if they actually exist or meet minimum standards, but whether they are closer to better practices. The charge of the new compliance counsel includes assisting prosecutors in establishing appropriate benchmarks for corporate compliance. According to the DoJ section chief, this means “benchmarking with various companies in a variety of different industries to make sure we have realistic expectations … and tough-but-fair ones in various industries.” This trend of the government to provide more guidance has continued with the DoJ stating it plans to release a set sample questions to give companies an idea what investigators and prosecutors are concerned with.
Types of program benchmarking
Compliance professionals and auditors should monitor this promised guidance from the Justice Department’s Fraud Division on how it proposes to evaluate the existence and effectiveness of individual corporate compliance plans. A recent and detailed “open letter” to the DoJ’s new compliance counsel (published in the Harvard Business Law Review) serves in part to provide recommendations on how the Department should implement this goal of establishing industry-specific benchmarks by which individual programs may be evaluated.
As described in the open letter, some of the principal categories of business benchmarking to consider include:
External benchmarking. This involves analyzing “best in class” outside organizations, providing the opportunity to learn from those perceived to be at the leading edge. This is the type that probably comes most readily to mind, and seems the most intuitively appealing—why not learn from the best? But three caveats are in order.
While benchmarking can be part of a program evaluation, by itself it does not comprise an evaluation and determination of effectiveness. It does, however, enable organizations to develop plans on how to make improvements or adapt identified leading-edge practices.
First, this type of benchmarking can involve implicit decisions about what makes certain organizations best-in-class for certain corporate compliance functions. This is an area of uncertainty, though promising empirical studies are emerging. What works (or what works best) is still not fully known.
Second, the DoJ’s own experiences with corporate investigations have revealed that companies with a general best-in-class reputation (in terms of size or profitability) can still have significant deficiencies in their compliance programs.
Third, solutions that work effectively for very large companies may not be practicable for smaller companies that cannot afford the necessary resources or technology to implement them.
Internal benchmarking. This entails benchmarking businesses or operations from within the same organization (e.g., business units in different countries). At first blush, this type of benchmarking may not seem worthwhile. If a compliance program is found deficient in one business unit within a company, examining how that program works in other business units of that company might seem pointless. However, companies have discovered that in examining a particular process across the organization, they may find significant variations between units or product lines. Such variations can help identify flaws the company should fix or enhancements the company should adopt, and determine whether ongoing monitoring is necessary.
Performance benchmarking. This looks at performance characteristics in relation to key products and services in the same sector. Although the DoJ does not set production performance standards—for example, how many units per hour should be produced—it could use performance benchmarking to identify features of compliance programs that yield quantitatively measurable results. One example would be above-average detection of instances of potential misconduct or numbers of corruption-related Suspicious Activity Reports. In healthcare, this could entail the frequency and accuracy of regular audits of billing and coding processes to ensure that medical services were billed and paid correctly given the scrutiny of government reimbursement.
Strategic benchmarking. This involves examining long-term strategies, for example regarding core competencies, new product, and service development, or improving capabilities for dealing with change. Standard components of compliance programs, such as risk assessment processes and cultural assessment, may come to mind here. The methodology could include strategic approaches for companies developing or improving compliance programs.
Here again the auditor can bring his or her toolbox to assist the compliance professional in benchmarking the program. Methodologies and techniques for benchmarking comprise the evaluative approaches the audit profession has used for systematic program evaluation. The audit practitioner could apply rigorous approaches such as maturity and internal control reliability models with different levels of effectiveness when benchmarking the compliance program.
A feature ideal for benchmarking is assessment of whether the compliance officer has the appropriate autonomy and resources to oversee the compliance program. Because what constitutes appropriate autonomy and resources can vary widely for smaller and larger companies, the general standard and guidance is too general and laden with contingency. For instance, a single paragraph in the FCPA Resource Guide states multiple times that the degree of autonomy and extent of resources devoted to the program will “depend” on circumstances.
Benchmarking on autonomy and resources would be more useful if it concentrates on identifying examples of suitable practices to ensure sufficient oversight, autonomy, and resources in companies of different sizes. For example, while a number of large companies have now separated their risk and compliance functions from their legal departments, smaller companies may need to decide whether and how they can leverage their compliance or legal departments (whichever is currently in place) to be effective overseers of risk and compliance wearing dual hats. Ideally what may emerge is the ability to identify percentage data that could be useful for companies of various sizes. For example, “We found that companies with annual revenues of less than $50 million, those companies that appeared to have effective compliance programs typically devoted between A and B percent of their annual budgets, while companies with annual revenues of $500 million or more appeared to have effective compliance programs typically devoted between C and D percent of their annual budgets.”
Another component for benchmarking is the compliance hotline. Auditors can benchmark the company's hotline data to the vendor's other customers of a similar size and industry, or to another credible external source. If you are only getting a fraction of the industry average number of complaints, there may be problems with the training and communications program. Further analysis can show whether the variance in hotline data (volume, incident mix, use of anonymity) is local or pervasive.
Considering the old adage of the usefulness of measuring call volume, benchmarking can help auditors assess if low volume is indicative of particular issues with the hotline process, by comparing call volume to industry averages. A low call volume can instead be due to employees using other methods to report potential wrongdoing. Before benchmarking your hotline, consider aggregating data from each reporting method to allow for such differences.
No comments yet