JOnline: Evaluating and Selecting Sarbanes-Oxley Software 

 
Download Article

It has been six years since the US Sarbanes-Oxley Act was signed into law, yet many of the software products designed to handle the law’s complex demands are still maturing as accountants and auditors continue to require more sophisticated functions. This article reviews a selection of software products in this market and assesses how well they work.

Complying with the Sarbanes-Oxley Act of 2002 costs the average company US $7.8 million and 70,000 hours of employee time.1 In response to the complex record keeping and auditing requirements of Sarbanes-Oxley, a number of companies have developed software products to automate these tasks. These products can be classified into four major categories:2
  1. Data manipulation software
  2. Document and workflow management software
  3. Risk analysis and risk management software
  4. Control self-assessment and continuous monitoring software

Of these four types of software, the last two categories—products that help accountants respond most directly to Sarbanes-Oxley requirements—will be the focus of this article.

This article reports the results of a study the authors conducted to examine select Sarbanes- Oxley software products in detail. The study included the compilation of a set of Sarbanes- Oxley products that use a COSO-driven, topdown, risk-based approach; the acquisition and hands-on testing of eight major software packages; and a detailed comparison of the many features available in them.

This evaluation is meant to enable Sarbanes- Oxley software users to compare their products with others, enable Sarbanes-Oxley software buyers to purchase software products intelligently and prompt vendors to further develop their products.

Sample Selection and Evaluation Method

The first task was to select software for evaluation.3 Only those software products that were designed to perform a Sarbanes-Oxley compliance audit using a COSO-driven, topdown, risk-based approach were selected. Products that were no longer being marketed were eliminated from the study.

Each of the software vendors on the final short list was asked to participate in the study and provide an evaluation copy of their software. To ensure a meaningful list of products, vendors were also asked to identify their major competitor(s). In most cases, their responses confirmed the importance of the major players already on the short list.

As a result, a final set of eight vendors was created—each having agreed to participate in the study. Some of these products are fairly new to the market and were developed expressly to accommodate Sarbanes-Oxley auditing and reporting requirements.4 Others have been around for some time, were originally designed for risk management and have been modified to handle Sarbanes-Oxley requirements.

The software products included in the evaluation are:
  • ControlCase Compliance Manager
  • Methodware’s Enterprise Risk Assessor
  • OpenPages FCM
  • Paisley Enterprise GRC
  • Protiviti Governance Portal
  • SarbOxPro
  • Sirius Solutions Risk and Controls Management Software
  • SOX Automation’s SOX-DISC

The next step in the investigation was to identify criteria by which to evaluate the software. In this endeavor, the functionality of each product was used to help define both “core capabilities” and “unique features.” This enabled the identification of not only the commonalities among these products, but also those unique features that differentiated a particular package from its peers.

In cases where all software products had the same feature, that criterion was eliminated from further study. For instance, all of the products use a COSO-driven, top-down, risk-based approach, so this criterion is not included in the table.

Finally, in each case, the software was tested without training in order to best judge ease of use. (In general, the authors do not recommend such an approach to end users, who would obviously benefit from such education.)

Results

All of the software products described here can perform the basic evaluation tasks consistent with Sarbanes-Oxley compliance requirements. Similarly, all of them run on PC microcomputers using standard Windows or Vista operating systems. Comparatively speaking, none of them is particularly expensive, especially for mid-sized or large companies. Other common software features include:

  • Public Company Accounting Oversight Board (PCAOB) assertions that can be specified by the general ledger account
  • Processes that can be identified with the general ledger account(s)
  • A prepopulated library of risks for major business processes5
  • The ability to identify key controls
  • The ability to track the history of tests on controls
  • Automatic recording of tests of controls
  • The ability to reuse a single control in many different places, with the software allowing the user to test the control only once
  • Support for multiple entities/locations with either the same controls or different controls
  • The ability to export data to Excel
  • The ability to prevent any user from freely accessing control weaknesses

What distinguishes these products from one another?

Figure 1 (included at the end of the article) provides the results of the inquiry. The software vendors were allowed to check this report for accuracy. However, the opinions contained in the table evaluations remain those of the authors.6

Figure 1 lists products alphabetically in the columns, with no ranking implied by the authors in this summary. Within the body of the table, the term “yes” means that the software quickly and easily performs the task or function indicated, while a “no” means the opposite. (Note: in most cases, the software vendors are willing to customize their products to overcome such deficiencies, but at additional cost to the user.) In a few instances, it was found that the software could perform a task only with some additional effort from the user. In such instances, a rating of “OK” was assigned for this feature. Thus, an “OK” rating is relative to the other products in the lineup.

In figure 1, the criterion “Does it show work papers or provide link to the same?” under Tests of Controls requires further explanation. There is a difference between work papers that are actually embedded in the Sarbanes-Oxley software and work papers that are stored elsewhere and accessed by links. A “yes” indicates that one or the other occurs, and that a user can obtain work papers without concern for how this is done. A “both” indicates that users can choose which manner of storage they prefer. For example, linked work papers might work better for auditors preferring to store such documents elsewhere.

For the Reporting criterion in the table, a rating of “yes” for e-mail notification means that specific users and/or other users will be notified when reports are generated by the software. These may, for example, be forms these users need to complete for their control evaluation tasks—a feature not commonly found in financial accounting software.

Integration is a feature that works differently in this software than in financial accounting software. In financial accounting software, one looks for integration between the ledgers and the journals. In contrast, Sarbanes-Oxley legislation requires an evaluation of controls over financial reporting, and, eventually, a company must assess its overall control process. Consequently, for Sarbanes-Oxley software, integration is needed between the evaluation of the controls and the overall evaluation of the process that the controls are designed to protect. All of these packages use a topdown, risk-based approach and finish by evaluating the controls. Therefore, the authors wanted to see whether the software used those evaluations to help the user evaluate the effectiveness of the overall process or, indeed, whether the overall process was even evaluated. If users had to evaluate the overall process themselves without assistance from the software, integration was rated lower. If the software aided the user in this evaluation, the integration was rated higher. If the software did not accommodate rating the overall process, this is noted in the evaluation. While the integration criterion was primarily aimed at the controls/process evaluations, other integration was also looked for within each software product. Therefore, a package that does not rate the overall process could achieve an acceptable level of integration based on other aspects of its software.

The evaluation table (figure 1) only provides detailed information about the capabilities of the various software products that differed. All of these products have important, useful features in common that, for the sake of brevity, were not included in this table.

Discussion of Results by Product

Figure 1 lists the comparative findings. The following sections discuss each of these products in turn and provide some additional insights that were gathered from the study. (Again, the sample products are presented in alphabetical sequence.)

ControlCase

ControlCase is free for those using the PC-based version. Although all the packages assess process, risk and controls, the software’s remediation module also allows one to document what is going to be done about a problem it has identified. Other packages have similar capabilities, but are not as complete or as well specified as this software. The software also time- and date-stamps documents to provide evidence of the chain of custody for documents that are part of a particular process—a feature particularly important for legal evidence. This product is suitable for evaluating individual controls. One possible concern to the users of this product may be ease of use.

Methodware

Methodware has been available since 1998, and it is a mature, sophisticated and full-featured package. The user interface is different, interesting and (to the authors) visually pleasant. Developed by former employees of Ernst & Young, New Zealand, this software allows users to perform all of their own risk assessments—not just those for Sarbanes- Oxley compliance. The software also performs Monte Carlo simulations, thereby eliminating a need for such assessments by the auditors or the managers. The software also includes a tool that that allows users to build their own business rules and customize the software to suit their firm’s requirements. A unique feature of Methodware is that not all employees need be registered users to participate in Sarbanes-Oxley compliance. E-mail notification with drop-down menus that match the user interface can be sent to nonregistered users for control assessment. These can be completed and sent to registered users for immediate integration into the software. This earns top honors for Methodware when it comes to innovative thinking with respect to product integration and pricing, as fewer registered users reduce costs. This software is suitable for evaluation of controls and the overall process.

OpenPages

OpenPages is one of the leaders in this market. The firm’s president has an accounting degree and this background shows in the software. The company advertises itself as “enterprise software” and this places it at the high end of the spectrum. But, whereas one would expect an enterprise version of software to be difficult to use, this software has an intuitive user interface and is easy to use. The authors particularly like OpenPages’s incident and issues reporting method. Incidents happen in a firm, apart from those related to Sarbanes-Oxley compliance, and this software allows users to document those incidents as they occur, so they can be considered during Sarbanes-Oxley testing. This is important and can be critical to Sarbanes-Oxley tests. The software also comes with an application programming interface (API) that allows OpenPages to synchronize with general ledger software or link with other software applications such as ACL or IDEA. Thus, this software is suitable for evaluation of controls and the overall process.

Paisley

Paisley is a software product that was initially offered as a general risk-based product in 1995. Consequently, this is a more mature product than some of the others in this sample. The president and founder was formerly an internal audit manager and this focus is evident in the software, which has the ability to add new risks and controls. It also has an excellent built-in library of controls. This software is available in six different languages and has two methods of deployment: the user’s desktop or Paisley hosting of the application. This is another enterprise-level software package, yet as with all products, the authors believe there is always room for improvement. The help menus were not particularly helpful and could be improved, although the authors recognize that effective training may eliminate the need for excellent help menus. Also, “out of the box” scoring of the materiality of the process would be a nice feature. This can be done with some customization or user-designed fields, but standardization of such tools would be beneficial.

Protiviti

Protiviti is known for its consulting skills, but the company has also developed its own Sarbanes-Oxley software. The software does an excellent job of identifying risks, and users will be hard pressed to come up with new risks to add to those already built into the software (although it is relatively easy to add others, if necessary). Areas to consider for subsequent versions include user ability to more easily load a trial balance (currently, each balance has to be input by keying in the data), assessment of inherent risk for the overall process and making the reporting function easier to use. Finally, the product’s Sarbanes-Oxley compliance integration could be strengthened. During the demo, it was observed that test results can be marked as ineffective when the underlying tests are marked as effective, because the software does not automate the evaluations conducted at various levels.

SarbOxPro

SarbOxPro is a relatively new product that was developed specifically for evaluating Sarbanes-Oxley compliance for small to mid-sized companies. It is free to users, who can download it from the Internet. It also has a very easy-to-use interface that runs using Microsoft Access. For a relatively trivial price, users can also get training and support for the software, which the authors would highly recommend. The package the authors evaluated lacked some of the bells and whistles of the enterprise-class software packages, but the functionality is sufficient to do a basic Sarbanes-Oxley compliance evaluation. Users will not be able to do workflow analysis, for example, but will be able to do workflow in another application and link to the relevant file within SarbOxPro. Of all of these packages, in the authors opinion, this software does the best job of measuring inherent risk for a process. This is extraordinarily good software for the lower end of the market, and the price cannot be beat. One suggestion for improvement is to better develop the capability for users to develop custom reports.

Sirius Solutions

Sirius Risk and Controls Manager (Sirius RCM) is another product that is built on an Access database. The initial user interface was somewhat simplistic, but users should understand that the actual controls repository and controls-testing portions of the software are more sophisticated. What it does, it does just as well as any of the other lower-end products, and it is relatively easy to use. This software does a better job of documenting the controls than documenting processes. But, users who understand Microsoft Access can modify the software as they see fit—for example, to evaluate controls compliance or perform process evaluations.

SOX Automation

SOX-DISC is a well-designed, competent software product. It is intuitive to use, clearly designed for accountants, and even includes guidelines in each major module to identify the relevant Generally Accepted Accounting Principles (GAAP), standards or guidelines for that module. This is handy for training and staying on top of compliance issues. The risk/ control matrix includes an extensive amount of information with even more reporting behind the matrix—an exceptional design element.

This software is very mature, well conceived and has everything one could ask for in a Sarbanes-Oxley product with one exception: it lacks customization. That is, it has only prespecified screens and reports, which are, nevertheless, very good. But, the vendor appears willing to work with users and should be highly responsive to user needs. A web-based product is under development and should be available soon.

Making a Choice

In one sense, all of the products essentially do the same thing—automate many of the tasks required to comply with the Sarbanes-Oxley Act. For example, they all have the ability to document a process, identify the risks and controls associated with that process, test those controls, and generate reports.

Sarbanes-Oxley legislation is relatively new and, for this reason, Sarbanes-Oxley software is relatively young. Accounting professionals who are used to mature accounting software may be disappointed by some of the features, or lack thereof, in this emerging market. Products that show the best advantages are usually retrofitted products—i.e., products that started as risk management software and then expanded with Sarbanes-Oxley capabilities when the legislation presented the opportunity. Unquestionably, these more mature products have much to offer to the consumer, especially to larger firms looking for a total solution to their risk management problems. But, not every firm requires such a solution, and many firms may want a Sarbanes-Oxley only package that has some of the same features they have come to expect from their financial accounting software packages—that is, good integration, reporting ease and a user-friendly interface.

As with all young software markets, this is a rapidly evolving market and new software versions come out every few months. In the future, the authors expect to see more and better integration, enhanced reporting capabilities, greater flexibility in user abilities to design custom reports, and better and more user-friendly interfaces. Thus, there should develop a complex synergy between the Sarbanes-Oxley vendors and the auditors and accountants involved in Sarbanes-Oxley compliance. This should also, over time, help the Sarbanes-Oxley software market grow and mature.

For now, those involved in Sarbanes-Oxley compliance efforts should find the software discussed here well worth the price. For example, such efforts require evaluating and documenting internal control processes—tasks that these software packages make relatively painless. Also, the relative youthfulness of this market should not detract from the fact that the software can assist users today with a task that most would prefer not to do manually or with software designed for other tasks.

The purpose of this evaluation was to provide information that can assist firms in making their own “best” choices. It also provides criteria and information that should be useful in evaluating competing software that was not included in this evaluation. Additionally, every organization has unique requirements, and the authors do not discount the importance of customer service and training—attributes that were not included in this evaluation.

As with other types of software, picking the best product usually means finding the product with the features required for the particular organization (e.g., a particular language option), rather than picking a product with some kind of overall best score—a rating the authors deliberately chose to avoid making here. Nonetheless, there is a software package available that is suitable for every firm.

Figure 1
Figure 1 cont.

Endnotes

1 Lacy, Sarah; “The Sarbanes-Oxley Software Race,” Business Week Online, 12 July 2005
2 Bagranoff, Nancy A.; Laurie Henry; “Choosing and Using Sarbanes-Oxley Software,” Information Systems Control Journal, vol. 2, 2005, www.isaca.org/archives
3 Ibid. and Brooks, D.; M. Goldman; R. Lanza; 2006 Buyer’s Guide to Audit, Anti-Fraud, and Assurance Software, Ekaros, 2006
4 Shein, Esther; “Thinking Inside the Sarbox,” CFO, 22(5), 2006, p. 32
5 Every vendor in the sample reported that most public entities want to use their own controls and risks, rather than the controls and risks available in these libraries.
6 The authors also note that, because software is frequently updated, a feature that was identified as missing in figure 1 may now be available.

Roberta Ann Barra, Ph.D., CPA
is an assistant professor of accounting at the University of Hawaii at Hilo (USA). She can be reached at roberta.barra@hawaii.edu.

Arline Savage, Ph.D.
CA , is a professor of accounting at Cal Poly San Luis (California, USA). She can be reached at savage@calpoly.edu.

Mark G. Simkin, Ph.D.
is a professor of information systems at the University of Nevada (USA). He can be reached at markgsimkin@yahoo.com.


ISACA Journal, formerly Information Systems Control Journal, is published by ISACA, a nonprofit organization created for the public in 1969. Membership in the association, a voluntary organization serving IT governance professionals, entitles one to receive an annual subscription to the ISACA Journal.

Opinions expressed in the ISACA Journal represent the views of the authors and advertisers. They may differ from policies and official statements of ISACA and/or the IT Governance Institute and their committees, and from opinions endorsed by authors, employers or the editors of this Journal. ISACA Journal does not attest to the originality of authors’ content.

Instructors are permitted to photocopy isolated articles for noncommercial classroom use without fee. For other copying, reprint or republication, permission must be obtained in writing from the association. Where necessary, permission is granted by the copyright owners for those registered with the Copyright Clearance Center (CCC), 27 Congress St., Salem, Mass. 01970, to photocopy articles owned by ISACA, for a flat fee of US $2.50 per article plus 25¢ per page. Send payment to the CCC stating the ISSN (1526-7407), date, volume, and first and last page number of each article. Copying for other than personal use or internal reference, or of articles or columns not owned by the association without express permission of the association or the copyright owner is expressly prohibited.

Subscription Rates:
US: one year (6 issues) $75.00
All international orders: one year (6 issues) $90.00
Remittance must be made in US funds.