PCAOB Inspection Reports at 5: What Can We Learn

After a decade of regulating the audit of public companies in the United States, only one thing is certain about the quality of audits: that even today, nobody is quite sure how good audits actually are.

The Public Company Accounting Oversight Board, formed under the Sarbanes-Oxley Act, continues to adjust its approach to regulating the audit profession, especially the method by which it inspects audits to determine where problems exist that auditors need to fix. That has sent auditors on an odyssey—especially in the last five years—to determine what will satisfy regulators and the public. How can auditors deliver a tough but fair audit at a cost that clients are willing to pay?

“If I’m sitting in Congress or at the Securities and Exchange Commission and I want to see if the auditing profession is getting better or worse, could I figure it out?” asks Joe Carcello, executive director of the corporate governance center at the University of Tennessee and a past member of the PCAOB’s Standing Advisory Group.

It’s a good question. One might start by looking at the results of PCAOB audit inspections, including the failure rates for various audit firms where PCAOB inspectors decided the audit work wasn’t up to par—and at a glance, those numbers don’t paint a flattering picture. But, Carcello cautions, “If you look at the aggregate results, does it mean the profession has gotten worse or the PCAOB has gotten better? It’s of limited usefulness in being able to make any broad general conclusions about audit quality.”

Only for the last five years has the PCAOB published data in its inspection reports to give the public a sense for how many audits the agency examines and how often it finds problems. Among the eight largest audit firms in the United States, the PCAOB inspected 371 audits in 2009 and found fault with 56 of them, or 15 percent—a number nobody found especially alarming at the time (at least not openly). Those firms include the Big 4—Deloitte, EY, KPMG, PwC—plus the second tier of global firms: BDO USA, Crowe Horwath, Grant Thornton, and McGladrey.

The following year, in 2010, the deficiency rate across the profession more than doubled to 34 percent. Inspectors dug into a total of 354 audits across the same group of firms and found problems in 119 of them serious enough to flag inspection reports. In the words of the PCAOB, that means the audit deficiency was of such significance that it appeared to inspectors that the firm didn’t have enough evidence to support the audit opinion.

The pain for audit firms went on from there: 38 percent in 2011, 42 percent in 2012, and 43 percent for 2013, the most recent year for which inspection findings are available.

The PCAOB is careful to caution that its inspection process is not based on random sampling and isn’t meant to provide a scorecard on any given firm in any given year. “The board cautions against extrapolating from the results presented in the public portion of a report to broader conclusions about the frequency of deficiencies throughout the firm's practice,” the PCAOB writes in each of its inspection reports. The inspection process is meant only to sniff out instances where auditors have failed in some fashion to adhere to auditing standards.

While the rates themselves may not serve as a reliable indicator of overall audit performance, they clearly indicate something about audit quality that regulators and auditors alike want to change. Even Chairman James Doty refers to rates with displeasure. “We continue to find unacceptably high rates,” he said at an accounting conference in December 2014.

“However you average it, it’s not changing directions yet from a trend standpoint. It’s going in the wrong direction.”
Gaylen Hansen, Member, PCAOB’s Standing Advisory Group

Board members Jeanette Franzel and Jay Hanson, in discussing results on smaller firms a few years earlier, said they have no illusions that audit failure rates will ever reach zero, although they clearly want the rates to fall. “Certainly the current level is too high,” Franzel said then. Unwilling to pin a number to an acceptable rate, she said simply: “We’re not there yet.”

Gaylen Hansen, a partner at regional firm EKS&H and a former member of the PCAOB’s Standing Advisory Group, says drawing conclusions about overall audit quality is impossible given the way audits are selected for inspection, but the rise in rates should be disturbing to audit firms. “However you average it, it’s not changing directions yet from a trend standpoint,” he says. “It’s going in the wrong direction.”

Cindy Fornelli, executive director of the Center for Audit Quality, says firms spend lots of time and effort addressing PCAOB inspection findings. “Each individual firm and the profession as a whole very much want to see the rate of deficiencies improve year over year,” she says. Alex Schillaci, national managing partner for inspections at Deloitte, says the topic is heavily debated within the profession.

Given the time lag on the release of inspection results, it’s not yet clear whether firms have made any progress in reducing those rates since 2013. The most recent inspection cycle, performed in 2014 on 2013 year-end financial statement audits, will come into focus over the course of this year as the PCAOB publishes individual firm reports.

AUDIT QUALITY INDICATORS

If inspection reports from the Public Company Accounting Oversight Board alone can’t tell the story of what’s happening with audit quality in the United States, then what can? Experts answer that question below.
Having a defined set of agreed-upon indicators against which audit quality could be measured would be helpful, says Cindy Fornelli, executive director at the Center for Audit Quality. The U.S. Treasury Advisory Committee on the Auditing Profession issued its final report in 2008 calling on the PCAOB to define audit quality so the audit profession would have a clear idea of the target it needed to meet. The PCAOB has been picking away at a project for several years trying to define audit quality, but so far has not yet issued the concept release that was promised for early 2014, and again for early 2015.
The CAQ, whose members are audit firms, took a pass at defining what it considers to be the key indicators of audit quality. It focused on evaluating a firm’s leadership and its tone at the top as a starting point, then assessing an engagement team’s knowledge, experience and workload. Also key, the CAQ says, are the processes and controls that are in place to monitor an engagement team’s performance, and auditor reporting.
The International Auditing and Assurance Standards Board also developed its own framework for how audit quality can be assessed, focusing on evaluating an engagement team’s values, ethics, attitudes, knowledge, skill, experience, audit process, quality control procedures, reporting, and interaction with stakeholders.
Absent any clear agreement in the United States over what defines audit quality and what inspection reports say about it, the market has come to rely on other indicators, says Fornelli. Restatements, for example, provide some indication, she says. Restatements spiked in 2006 and dropped off considerably in the following year, holding fairly steady since 2010, even as audit inspection findings have climbed.
Lorraine Malonza, director of accounting policy at Financial Executives International, is a bit puzzled by a decline in restatements at the same time audit findings are rising. “I would have to say yes, audit quality is better, just because the number of restatements is going down,” she says. “We really don’t know, truly, if inspections are leading to better quality audit work.”
Of course, the CAQ also likes to point to its annual investor confidence survey, which has climbed steadily since the turmoil and downturn associated with the 2008 financial crisis.
Alex Schillaci, national managing partner for inspections at Deloitte, says he relies on a number of indicators beyond PCAOB inspection reports to assess the firm’s audit performance, including its own internal inspections, peer reviews, and the firm’s root cause analysis, where audit leaders look at any failures or deficiencies to figure out what went wrong. He says the firm’s overall system of quality control includes ongoing monitoring of engagements to get more real-time indications of problems that might crop up.
Root cause analysis has become a big focus of the PCAOB, says Helen Munter, director of registration and inspections at the PCAOB. Inspectors and the firms alike are taking a close look at inspection findings and learning what kinds of factors are associated with poor audit work.
“Time spent is an important one, along with appropriate supervision and review of the scoping of the audit before the work begins,” says Munter. “Proper planning goes a long way toward having a successful audit.” Inspectors also are looking for indicators of good audit work in inspections where they have no criticisms to offer, she says. “One of the things that stands out is good project management as well as a deep understanding of the client and the industry in which it is operating.”
PCAOB member Jay Hanson has called on the PCAOB to focus more on positive inspection findings as a way of pointing auditors and stakeholders toward audit work worthy of modeling. In a recent speech describing his wish list for changes to audit inspection reporting, Hanson said he hopes to see future reports contain more discussion of root causes, more guidance to audit committees on questions to ask auditors, and more analysis of what negative findings mean. “I am also hopeful that we can move toward providing information about what auditors are doing right rather than focusing our reporting primarily on observed deficiencies, especially in the areas where firms successfully remediate quality control deficiencies,” he said.
—Tammy Whitehouse

At the same December accounting conference, Doty said the board would release some kind of summary information on its most recent round of inspections in the near future. “We are going to be putting something out soon on what we are seeing in the field,” Doty said then. “We can see some improvement, but some are not improving.”

That would be consistent with the historical data. Within individual firms, the trends do not necessarily follow the group averages. At Deloitte, for example, the deficiency rate began at 21 percent in 2009 and jumped to 45 percent and 42 percent the following two years. Then in 2012 the rate dropped back to 25 percent, and 28 percent in 2012 and 2013. PwC can also boast some improvement in its rates, which peaked at 41 percent in 2011 but fell to 32 percent by 2013.

The rates have worsened, however, for EY and KPMG. EY began with a low for all firms in 2009 of 9 percent (the only firm ever to register in single digits) but has risen steadily each year since to 21 percent, then 36 percent, then 48 percent and 49 percent. KPMG began at 13 percent in 2009, rose to 22 percent and 23 percent in the next two years, then to 34 percent in 2012 and 46 percent in 2013.

Among the second tier of firms behind the Big 4, the story is similar. Although the number of audits inspected at McGladrey and Crowe Horwath is much smaller, those firms have shown steady decreases in their rates after rising with the rest of the profession. Grant Thornton topped out at 65 percent in 2012, then fell back to 55 percent in 2013. BDO USA rose steadily from the mid-20s in 2009 and 2010 to a high of 65 percent in 2013.

Deloitte and PwC both say they expect their next published inspection report to show improvement over the prior year. “We do believe the next published report will be indicative of the quality improvements we’ve put into place,” Schillaci says. None of the other major firms were willing to discuss inspection results.

Dan Goelzer, a former member of the PCAOB, says while the statistical data may not reflect a firm’s overall audit performance, it might provide some indication of how firms are performing relative to one another. “The board is applying the same risk focus to the engagement selection process at all firms,” he says. “So maybe there’s some significance in the variations.”

On the other hand, firms may be inspected by different inspection teams, or may have different risks depending on their client mix or sector concentration. “There are a lot of caveats that factor into it, but if a particular firm’s rate is going up or down, it may say something about audit quality,” he says.

The real question is whether the inspection reports provide information that’s useful to corporate audit committees in assessing audit quality, says Phil Wedemeyer, an audit committee chairman for Atwood Oceanics, who is also a former auditor and former staff member at the PCAOB. The PCAOB has given audit committees some guidance on how they can use inspection results, to question their auditors about how the issues raised in reports might apply to their particular audits and how auditors are responding.

“In terms of evaluating the firms for audit quality, I haven’t found a way to use the reports that way,” he says. “They’re all pretty high. The idea of having occurrence rates like this doesn’t help me. If you ask me if 45 percent or 38 percent statistically makes sense, I’d say no. In a lot of areas, the audit is much better than it used to be in terms of rigor. So something just doesn’t hang together here.”