Behind the Search for Key Indicators of Audit Quality

Can audit quality be measured? What are the top factors that contribute to it?

Those are questions the Center for Audit Quality and others are attempting to answer in a quest to identify key indicators of audit quality and improve the audit process at U.S. companies.

The CAQ recently asked audit firms to work with audit committees at a handful of public companies to test key factors it has flagged as potential indicators of audit quality. The group is searching for methods of assessing the quality of the external audit with the hope that it will become a standard tool for audit committees to use in evaluating and hiring auditors.

The CAQ has proposed a list of indicators intended for audit committees to use as a yardstick for measuring the quality of work they get from their external audit firms. Although organized a bit differently, the CAQ's proposed framework is based on many of the same themes put forth earlier this year by the International Auditing and Assurance Standards Board as a way to assess and measure audit quality. Major firms each will be approaching their client audit committees asking if they will participate in a year-long pilot test to see if the proposed approach enhances an understanding of audit quality.

The CAQ approach is built on a premise that the best way to assess and seek improvements in audit quality is to enhance the discussion between the audit firm and the audit committee on four key areas that indicate audit quality: the leadership of the audit firm and its tone at the top; the knowledge, experience, and workload of the engagement team; processes and controls to monitor and assess the performance of a given audit engagement; and reports such as regulatory inspection reports, restatements, and withdrawn reports on internal control.

Auditors who participated on the CAQ task force developing the proposed list of indicators welcome the effort to try to come to some agreement over what constitutes audit quality. Today, auditors are judged primarily based on the results of inspections by their regulator, the Public Company Accounting Oversight Board, says Mike Yates, a partner at Crowe Horwath. “Whatever the results of those reports, people will talk about audit quality associated with them,” he says. “The wording in those reports is all about audit failures. It's all bad news. Without any broader context, that's the only thing anyone has to judge audit quality. That and restatements.”

The PCAOB has cautioned against drawing conclusions about the overall quality of any given firm based on inspection findings, because inspectors are following a risk-based approach to searching for audit problems. They are looking specifically in areas where problems are most likely to be found.

“Investors say the only visibility they have into audit quality is PCAOB inspection reports,” says Josh Jones, a partner at EY. “Some firms provide transparency and quality reports that provide insights into how we manage quality, but investors have only a handful of data points that give you insights into audit quality, and those are broad indicators. They don't tell you anything about the particular audit you're investing in.”

Despite the PCAOB's caution, Emre Carr, a financial economist and principal at consulting firm Berkeley Research Group, believes inspection reports function better as a measure of audit quality than any other available metric. “The audit failure rate is a parallel metric of audit quality,” he says. “If you measure by firm, that's arguably a measure of quality.” Knowing the risk-based approach the PCAOB takes with inspections gives the consumer of PCAOB inspection data enough understanding to assess quality accordingly, he says. “As an academic, I interpret findings accordingly. But it is useful when comparing the firms to one another.”

“The wording in those reports is all about audit failures. It's all bad news. Without any broader context, that's the only thing anyone has to judge audit quality.”

—Mike Yates,

Partner,

Crowe Horwath

“It's a first step in a journey,” says Arnie Hanish, audit committee chair for Omeros Corp. and former chief accounting officer for Eli Lilly, who also served on the CAQ task force. “We debated several alternatives and several different metrics. It's very difficult to come up with audit quality indicators that are going to give audit committees a true, black-or-white answer as to what the audit quality is.” As a result, the CAQ approach is built heavily on inspiring dialogue between auditors and audit committees on key indicators, or issues that can point to the likelihood of getting a quality audit.

As an example, data on restatements isn't informative itself without some understanding of what led to a restatement. “If I'm an auditor in 2014, and there's a restatement on 2012 financial statements when someone else was the auditor, that's probably good audit quality,” says Joe Carcello, executive director of the Corporate Governance Center at the University of Tennessee. “I found a problem, and there was a restatement.” A restatement on a firm's own work would not send the same signal, he says.

The idea behind the CAQ approach is to get auditors and audit committees discussing just those types of issues, says Hanish, which doesn't commonly occur. “What causes restatements?” he says. “How frequently do they occur? Is it a client issue or did you miss something? We wouldn't otherwise get into that level of detail at an audit committee meeting with our auditors in regard to their statistics firm-wide or even office-wide.”

Elusive Quality Measures

Carr says he doesn't believe the CAQ approach will lead to an effective system for measuring audit quality for a variety of reasons. The indicators are not specific enough to assess the quality of a specific engagement, and they are difficult or impossible to measure objectively. “When you look at firm leadership and tone-at-the-top, you can see how that relates to quality, but it's only a presumption, and it definitely does not relate to individual audits,” he says. “It's a very broad brush metric.” Likewise, knowing the knowledge or experience level of an engagement team doesn't necessarily assure they will do a quality audit or will operate without any conflict of interest, he says.

POSSIBLE AUDIT QUALITY INDICATORS

In the graph below, the Public Company Accounting Oversight Board shows possible audit quality indicators related to audit results.

Source: PCAOB.

In addition, the indicators are likely to provide little differentiation between audit firms, he says. “All these firms will look like each other on all these dimensions,” he says. “You're going to have very little variation across the Big 4 or engagement teams at the Big 4.”

The PCAOB is working separately on a project of its own to identify indicators of audit quality. Greg Jonas, director of research and analysis for the PCAOB, says the project is looking to determine if audit quality indicators can be established, and if so how they could be used. The board expects to publish a concept release possibly this summer that would solicit feedback on a preliminary list of possible indicators. “The CAQ has come forward with a proposal for engagement teams to privately discuss with audit committees certain qualitative indicators about the audit,” he says. “I expect that our concept release will mention such discussions as one way to utilize audit quality indicators, as well as other ways they can be used.” That might include regulatory purposes or for public disclosure to investors, he says.

Paul Herring, global head of governance and risk management at Grant Thornton, says he would expect the PCAOB concept release to address a broader set of measures than those put forth by the CAQ. “They'll be looking to respond to the needs of a broader set of players in the capital markets,” he says.