How the SEC Is Using Analytics to Spot Reporting Problems

That continuous monitoring program you're working on? Well the Securities and Exchange Commission is working on one too.

Imagine this: Your company submits its Form 10-K to the SEC and, instead of languishing in a database somewhere, it is immediately analyzed for risky accruals, accounting policies, and disclosures. All of the data gets instantly compared to what all others in your industry have submitted. When the analysis is complete—in minutes, not hours—it goes straight to an examiner with the equivalent of a sticky note that says, “Make sure you take a look at the following areas.”

Sound frightening? It's about year or so away from reality, or at least availability to the staff, says Craig Lewis, SEC chief economist and director of the division of risk, strategy, and financial innovation. In the 18 months since Lewis assumed his current role, one of his major projects has been something he calls the Accounting Quality Model (AQM). “At the highest level of generality, [AQM] is a model that allows us to discern whether a registrant's financial statements stick out from the pack,” Lewis said in a December speech on the topic. While it is still in prototype form, relying on external data providers, it will ultimately hinge on the XBRL tags that every piece of nearly every public company's financial reports are now required to carry. When operational, SEC staff members will be able to create customized reports for any company, or set of companies, using a Web interface.

The model is not necessarily designed to detect fraud, but it is likely to lead to more phone calls and questions from SEC staff to public companies, experts say.  “You may see more detailed accounting comments in the comment letters, asking a company to explain how they got to a certain result rather than the more general question of whether what they're doing is consistent with accounting standards,” says Thomas White, a partner in the Washington, DC office of the law firm WilmerHale and co-chair of the National Conference of Lawyers and Certified Public Accountants. It will also have the potential to seed more enforcement actions. As Lewis puts it: “The goal here is to provide better disclosures and better compliance with accounting standards. And if we happen to catch a few bad guys along the way, no problem.”

Lewis' efforts have already netted a few of those bad guys. The division that Lewis oversees, known in short-hand as “RiskFin,” was created in 2009 and is considered an interdisciplinary think tank for the agency. Within it, Lewis created the office of quantitative research to develop custom analytics such as the AQM. Among its past projects, says Lewis, is a model that helps other departments within the SEC identify hedge fund advisers who are “worthy of further review.” That model led to at least four enforcement cases against hedge fund managers, according to an October SEC report that included an update on the division, and “this success has only fed our ambition for what we can do with sophisticated data-driven monitoring programs,” Lewis said.

“You may see more detailed accounting comments in the comment letters, asking a company to explain how they got to a certain result rather than the more general question of whether what they're doing is consistent with accounting standards.”

—Thomas White,

Partner,

WilmerHale

Companies may be feeling some effects of these new approaches already. In each of the past two fiscal years, the division of corporation finance has reviewed disclosures from 48 percent of public companies that filed with the SEC, up from 36 percent in fiscal year 2007, according to the agency's 2012 financial report. Those two years have also seen record highs in terms of enforcement actions.

At the heart of the accounting quality model is its ability to statistically distinguish non-discretionary accruals from the discretionary ones that may indicate earnings management or fraud, something that has so far eluded academic models. Lewis, who previously spent most of his career as a finance professor, says the discretionary accruals can further be separated into factors that indicate earnings management, versus those that induce it. Those calculated discretionary accruals “are then used to screen firms that appear to be managing earnings most aggressively.”

Still, some question the SEC's ability to screen financials with sophisticated analytics. “It's clear that they're extending the current academic model, which is not very satisfying,” says Mohan Venkatachalam, an accounting professor at Duke University's Fuqua School of Business. He says most academic models are not very accurate, generally flagging a large number of firms that have not done anything wrong and missing batches of others that have gotten in trouble. That's led some academics to look for data outside the financial statements to help give clues about wrongdoing. Venkatachalam, for example, has found that technology-enabled analyses of executives' vocal patterns on earnings calls plus other non-verbal cues can help predict who is trying to hide bad news, as measured by future stock performance.

Big Data, Little Data

While the tool will certainly give the SEC staff an edge, experts privately express relief that the SEC is not, say, bringing news reports and other scattered data into the mix of how it evaluates firms. In fact, the analytics initiative falls short of being a Big Data project in some important ways. For one, it is not pulling in multiple data sources; it is looking solely at information contained within the filings. Second, Lewis notes, the model is not about analyzing mind-boggling amounts of data.

ACCOUNTING QUALITY MODEL

Below is an excerpt from SEC Chief Economist Craig Lewis's speech on the SEC's proposed accounting quality model:

Our Accounting Quality Model extends the traditional approach by allowing discretionary accrual factors to be a part of the estimation. Specifically, we take filings information across all registrants and estimate total accruals as a function of a large set of factors that are proxies for discretionary and non-discretionary components. Further, we decompose the discretionary component into factors that fall into one of two groups: factors that indicate earnings management or factors that induce earnings management. Discretionary accruals are calculated from the model estimates and then used to screen firms that appear to be managing earnings most aggressively.

… our approach necessitates the classification of factors into those that explain either discretionary accruals or non-discretionary accruals. The classification process should be informed by staff experience, intellectual capital, and the substantial accounting literature related to earnings quality and discretionary accruals. … by integrating actual staff experiences and knowledge into the Model, we have a powerful method for identifying those factors that can indicate outliers.

So, the obvious question is, then, what are some of the factors that we take into account when trying to identify outlier discretionary accruals? We can characterize discretionary accruals as different types of risk indicators and risk inducers. Risk indicators are factors that are directly associated with earnings management while risk inducers are factors that are associated with strong firm incentives to manage earnings.

In our model, for example, the choice of accounting policy and firm interactions with independent auditors may be indicative of specific types of earnings management. An accounting policy that could be considered a risk indicator (and consistently measured) would be an accounting policy that results in relatively high reported book earnings, even though ?rms simultaneously select alternative tax treatments that minimize taxable income. Another accounting policy risk indicator might be a high proportion of transactions structured as “off-balance sheet.” Although the vast majority of ?rms use off-balance sheet ?nancing for legitimate business purposes, many of the largest accounting scandals used off-balance sheet activities to hide poor ?nancial performance. In both instances, the metrics associated with accounting policies can be consistently estimated from filings data and compared to peers. Another risk indicator could be the frequency and types of conflicts with independent auditors, as measured by changes in auditors or delays in the release of financial statements or earnings. Again, these risk indicators could be consistently estimated from filings data and compared to peers.

On the other hand, risk inducers are designed to capture managerial incentives to mask poor absolute or relative performance. For example, a firm may be losing market share or it may be less profitable than its competitors. A firm experiencing performance problems, particularly those it considers transient, may induce a response that inflates current earnings numbers in exchange for lower future earnings.

The factor-based approach is a flexible modeling framework that easily accommodates new modeling factors as we add and delete proxies for potential earnings management. The additional flexibility lets us efficiently respond to model feedback and customize the model to suit different missions within the Commission while allowing for sensitivity to the nuances of those differing goals.

Source: SEC.

“The volume of data that we can handle is not a challenge to us or to academics, the challenge is how timely is the data,” says Lewis. What makes this model stand out, he adds, is its access to timely data. Lewis is currently working with other SEC staff to create a 24-hour turnaround time between when the data is submitted to the agency and when it is available to the model.  

Another important distinction between AQM and Big Data is that the SEC's AQM is largely looking at structured data, albeit some of which began as unstructured data (or text) before being tagged with XBRL code. However, that could change, as Lewis and his team are currently considering incorporating some textual analyses of MD&A statements in the model, based on their findings that fraudsters tend to under-disclose some areas relative to peers.

Which, of course, leads to the biggest question: Exactly how likely is it that the model could ferret out a fraud before it happens?

“Would it have picked up Enron? That's not exactly what we find,” says Lewis. While the model will pick up some of the most egregious violators, the agency has discovered in back-testing its model against fraudulent activity it has discovered in the past, many violators will also fall within the norm.

“The model is trying to detect a number of unusual accounting policies. What we found is that not all firms that committed fraud had unusual accounting policies,” he says, adding that the finding “is a little disappointing.”

But in fact nothing the SEC finds should come as a surprise to a corporate filer, notes Vincent Walden, a partner in Ernst & Young's fraud investigations services unit. “Corporates have the luxury of working with transaction details, so in my view, they can do a much more robust job of detecting fraud than anyone on the outside could do.”

According to Walden, however, the SEC's analytical tool is missing an important piece of the fraud puzzle. He says that a big part of proving wrongdoing is demonstrating corrupt intent. “It's very difficult to ascertain corrupt intent in footnotes, especially ones that gone by an auditor. To my mind, corrupt intent is best discerned in micro-data, like journal entries and transaction data, not data in the aggregate.”

Lewis is quick to note that the agency won't be launching any investigations based solely on what the model turns up. Still, companies that want to stay on the right side of the law may do well to start thinking like a computer. “To the extent there are certain tests management can run that you know the SEC is running, you should do it,” says Walden.