Auditing the IT Auditors 

 
Download Article Article in Digital Form

Every time an IT auditor engages in an IT audit/assurance project, at least one person reviews the work. The audit profession in general has developed structures and processes to make sure audits and similar projects are subject to an appropriate degree of review before being released to the sponsor. That process often involves multiple layers of review. The more risk associated with the project, the stricter the review process becomes and the more layers the structure contains.

The review process is designed to make sure the final product is appropriate given the original request, the results of tests and procedures, and technical literature and guidance on the subject matter. Tenure is important for a quality review to occur, and naturally requires a level of skill and knowledge to perform. The bottom line is that someone who is a subject matter expert (SME) in terms of the type of IT audit being performed will be the reviewer. Thus, for a Payment Card Industry (PCI) audit, the reviewer is likely to be certified Payment Card Industry Data Security Standard (PCI DSS), have years of experience, and be knowledgeable of technical literature and guidance on PCI audits. For a financial audit, the reviewer is likely to be a partner who is knowledgeable about IT, knows the client, knows the technical literature and guidance, and has years of experience.

Everyone involved in the project wants a quality review done internally prior to being released.

The questions addressed herein are: What should an IT auditor new to the profession expect in terms of the review process? What types of things are reviewers concerned about? What things would cause a reviewer to send the project back to the IT audit team for changes? How can the IT auditor make sure the review process goes smoothly? What does a reviewer’s checklist look like?

Scope and Risk

The first thing to understand about the review process is that the structure and process are highly correlated with risk. That is, the more risk associated with an IT audit project, the higher the level of scrutiny in the review process and the higher the likelihood of multiple layers of review. Therefore, if a project has a lot of risk associated with it for one reason or another, the IT auditor knows up front that the project will be reviewed more closely and probably by more professionals than otherwise.

Sometimes the type of entity drives that same intense review. Audits of financial institutions, larger entities (e.g., publicly traded companies) or entities highly dependent on IT are often considered in need of a higher level of review.

A level of importance based on other issues may also lead to a higher level of review. For instance, an internal audit of the entity’s perimeter for a high-profile organization could be seen as vitally important in today’s environment and, thus, may lead to a higher level of review.

Generally speaking, even entry-level IT auditors have a sense of this elevated need for review and recognize a project that is likely to take on that process. If risk is relatively high, more caution is needed in tests and procedures, documentation and work papers, and communications and reporting.

Relevant Areas for IT in a Review

A review obviously covers many aspects of the project/audit other than those that are IT-related. The general (non-IT) aspects include planning, technical proficiency of the team members, sufficient independence, methodology and using the work of others, among others. The areas in a review that can be related to IT are described in the following sections.

Technical Training and Proficiency
IT members of an audit or project are treated the same as others in terms of assessing an adequate level of technical abilities and proficiency sufficient for a particular project. One of the differences is the number of subsets of IT that exist and the need to have experts in them from time to time. For instance, a medium-sized company could be employing e-commerce via a web site, using a service organization to take care of payments (subject to PCI compliance and federal and state laws), using cloud services and involving a host of other IT-related influencers. These present the need for several specialty areas of IT that could arise for this single entity.

A second issue is the need to keep skills and abilities adequate and up to date. It goes without saying that once a person chooses to be in the IT audit profession (or any other IT profession), the person is committed to life-long learning. IT changes so rapidly that a person’s skills and abilities can become obsolete in a few years. Thus, one aspect of assessing risk for the IT function in an entity is the level of training and professional education people obtain each year, both on the corporate and individual levels. The same is true for IT auditors.

Controls: ICFR
Controls are always a part of an IT audit or assurance project, but they can be different sets of controls depending on the nature of the project and the audit objectives. Thus, each of the controls aspects that follow may or may not apply to a specific IT audit (i.e., when applicable, use them).

When the audit project involves financial reporting, the internal controls over financial reporting (ICFR) are a critical component of what the reviewer(s) want to examine. Thus, the IT auditor must take care in a financial audit to identify the relevant IT controls for ICFR and make sure to gain a proper understanding of applications, transactions and infrastructure that impact the financials.

Figure 1The reviewer’s checklist1 for this item should generally look something like figure 1. In some manner, the items in figure 1 should be mapped to authoritative guidance and work papers.2 This ensures compliance with standards and a sufficient scope of checklist items and facilitates the review process.

Controls: Entity Level
Another aspect of internal controls that affects the IT auditor is entity-level controls (ELC). These are controls at the higher levels of the entity that can impact the objective(s) of the IT audit or project. These would include board-related controls (e.g., IT governance), executive-related controls (e.g., managing the IT function by the CIO) and other broad controls relevant for the entity.

The reviewer’s checklist for ELC should generally look something like figure 2. Once again, the items usually are mapped to authoritative guidance and work papers. Often, and probably usually, an analysis of the ELC would require a walk-through to see if these controls have been designed properly and are implemented. Testing of these controls may become necessary later in the IT audit theoretically, but are sometimes done simultaneously with the up-front identification and analysis.

Figure 2

Controls: IT General Controls
IT general controls (ITGCs) have been one of the hot topics in IT and controls over the last few years. ITGCs have an effect on financial audits, all internal IT audits, all external IT audits and IT governance.

One of the things IT auditors need as a foundation is a good understanding of what constitutes an ITGC, how they are measured, what their risks are, and how to identify controls and assess their effectiveness. A great beginning is to read the IT Audit Framework (ITAF) from ISACA, section 3630, Auditing ITGCs.3

ITAF describes 17 different areas of ITGC. The Public Company Accounting Oversight Board (PCAOB) describes three. The American Institute of Certified Public Accountants (AICPA) has five. Figure 3 provides these lists. It is fairly easy to map these three taxonomies to each other; thus, there is basically a consensus as to the ITGCs. The difference is in the details of the items.

Figure 3

IT auditors need to understand that a significant weakness in an ITGC not only affects the ITGC, but also everything it affects. An ITGC is considered to be an entity-level control by the PCAOB, and thus it affects almost everything in the IT function and much, if not all, of the information systems, applications and technologies. It is for this reason that the PCAOB and AICPA guidance states that if the ITGC has significant deficiencies, the auditor cannot rely on application controls, which in reality reside beneath the ITGCs.

The reviewer’s checklist for ITGCs would generally look something like figure 4. It is based on authoritative guidance and work papers.

Figure 4

Controls: Application
Application controls are, by definition, automated controls, or hybrid controls,4 which make sure transactions are properly initiated, authorized, recorded, processed and reported. The more the control objective is driven by manual controls being melded, the higher the failure rate of the control. As IT professionals know, a fully automated control does the same thing over and over again. Once it has been tested, and as long as integrated technologies are unchanged, it will keep producing the same results as designed.

The IT auditor needs to know when it is appropriate to include application controls, which controls are relevant, how to test the control, how to assess its level of reliability and assurance provided, and how to fit all of that with the overall audit plan and program. The reviewer will be looking for that kind of information.

Figure 5The reviewer’s checklist for ITGCs would generally look something like figure 5.

Work Papers/Documentation
All auditors understand that the audit tests and procedures, and other aspects of the audit program, must be documented. The body of work papers, in particular, becomes the body of evidence to support the audit report and findings.

In addition, work papers become a key factor in the review process. The reviewer usually has not been privy to the daily information being gathered and handled during the audit process. Therefore, the reviewer is dependent on a body of information to pick up and fully understand, for example, what was done, the results, whether a sufficient body of tests and procedures was done, and whether the proper conclusions were drawn from the body of evidence. In fact, it is not uncommon in the first review for the reviewer, who is generally one level above the team leader internally, to have a question or need something else done to the evidence or need something else done for the audit program. In such a case, this is done with some form of written instructions (i.e., notes). When this happens, the process is pushed back to the team leader to clear up whatever the reviewer requested—usually referred to as “clearing notes” in a financial audit. It then goes back to the same reviewer. The efficiency and effectiveness of this process, including the loop, are highly dependent on the work papers’ quality, completeness and clarity (for reader’s understanding).

Report
Lastly, there is a report. The report is definitely subject to review. The report must be in conformity with the entity’s methodology and any technical requirements. It is probably the most important document to get reviewed.

Process

Some of the process has been explained, but it will flow something like the following:

  • Team leader reviews the documentation, work papers and report before sending it up the review chain.
  • Someone above the team leader reviews the audit. If problems or issues are found, the package is returned to the team leader, who addresses them.
  • Often, the review then goes to yet another person up the review chain—or parallel to it—for a second review.
  • Should the nature of the audit (e.g., risk) or entity require it, a third review might occur.
  • Even when the internal part is complete, sometimes there is an external entity that reviews the audit and the review process.

Conclusion

If the review process is done properly, the resulting product has sufficient quality to withstand scrutiny and analysis by the sponsor and other users, which means it is a quality audit. Therefore, it is valuable for the IT auditor to understand what constitutes a proper review, how that review process is carried out and how to make sure that the process is as trouble-free as possible.

Review is often multilayered—the team leader (e.g., manager) reviews for the team, someone else reviews the team leader (partner or director of IT audit) and additional layers may be needed for special occasions. But each layer is generally looking for the same things: proper scope throughout, proper work papers/documentation, whether the body of work papers support conclusions, and whether the report is written properly (grammar/spelling, enough narrative but not too much), addresses the original audit objectives effectively and is technically correct.

Perhaps the best guidance for the topic of this article is ITAF.5 It contains information on the subjects herein and more. In particular, it can make the review process more efficient and effective since the IT auditors doing the work are empowered to properly scope the audit, develop adequate evidence, address the proper controls and draw the proper conclusions.

To help make the process go smoothly and to avoid “loop backs” to clear up issues from the reviewer, IT auditors need to be diligent in performing their duties. In particular, it is important to self-check the audit when they believe they are done, asking:

  • Were all of the audit objectives addressed?
  • Were the tests and procedures robust enough given the risk involved?
  • Were all the relevant laws and regulations considered?
  • Is the body of evidence sufficient to support the conclusions and findings?
  • Do the work papers tie back to the audit objectives and conclusions properly?

While most newer (to the profession) IT auditors may need some time and experience to adequately address the guidance provided here, they need to at least understand who audits the auditors, how that process works, and how to make that process efficient and effective.

Endnotes

1 The checklists herein are based on checklists that might be used in a financial audit. Thus, they would be somewhat different for an internal IT audit for application controls, infrastructure audits, etc., but the principles should be similar.
2 All checklists have those two same factors: mapped to authoritative literature in some manner (e.g., reviewer mentally scanned the relevant literature requirements) and mapped to workpapers, usually physically on the form.
3 ISACA, IT Assurance Framework (ITAF), www.isaca.org/itaf
4 Hybrid refers to a combination of manual and automated controls to perform the control objective. This type of control is referred to as IT-dependent by the PCAOB and others.
5 Op cit, ISACA

Tommie Singleton, CISA, CGEIT, CPA, is the director of Consulting for Carr Riggs & Ingram, a large regional public accounting firm. His duties involve forensic accounting, business valuation, IT assurance and service organization control engagements. Singleton is responsible for recruiting, training, research, support and quality control for those services and the staff that perform them. He is also a former academic, having taught at several universities from 1991 to 2012. Singleton has published numerous articles, coauthored books and made many presentations on IT auditing and fraud.


Enjoying this article? To read the most current ISACA Journal articles, become a member or subscribe to the Journal.

The ISACA Journal is published by ISACA. Membership in the association, a voluntary organization serving IT governance professionals, entitles one to receive an annual subscription to the ISACA Journal.

Opinions expressed in the ISACA Journal represent the views of the authors and advertisers. They may differ from policies and official statements of ISACA and/or the IT Governance Institute and their committees, and from opinions endorsed by authors’ employers, or the editors of this Journal. ISACA Journal does not attest to the originality of authors’ content.

© 2013 ISACA. All rights reserved.

Instructors are permitted to photocopy isolated articles for noncommercial classroom use without fee. For other copying, reprint or republication, permission must be obtained in writing from the association. Where necessary, permission is granted by the copyright owners for those registered with the Copyright Clearance Center (CCC), 27 Congress St., Salem, MA 01970, to photocopy articles owned by ISACA, for a flat fee of US $2.50 per article plus 25¢ per page. Send payment to the CCC stating the ISSN (1526-7407), date, volume, and first and last page number of each article. Copying for other than personal use or internal reference, or of articles or columns not owned by the association without express permission of the association or the copyright owner is expressly prohibited.