What Every IT Auditor Should Know About Data Analytics 

 
Download Article Article in Digital Form

In the last column in this space,1 the introduction described the current IT environment and reasons why circumstances are driving an increasing demand for subject matter experts (SMEs) in CAATs, data mining and data analytics. Needless to say, there are a variety of needs in business today for effectual data analytics.

For several years, there has been a burgeoning segment of the IT profession known as business intelligence. A range of tools has been developed and used to assist businesses in mining valuable information from its own data to enable it to have more effective strategic and operational insights into executive decision making. Data analytics, as used herein, is similar to business intelligence. Because of the ways data analytics can be used and benefits gained from examining data, there is ample opportunity to apply the same methodology to IT assurance or IT reviews of this growing segment.

This space has also included a series of articles that, when combined, describe the process of using CAATs from data extraction,2 to data transformation3 and now to data analytics. The series is modeled after the data warehouse concept of ETL (extract, transform and load) when building the warehouse database. This article will describe factors for the IT auditor to consider in conducting data analytics.

Designing an Analytic Program

There are several key aspects of designing an effective analytic program. Those aspects can be determined through a series of questions:

  • Who are the key players?
  • What data are needed (tables and fields)?
  • What are the purpose and scope of the testing?
  • Who will get the report?
  • Where are the data residing?
  • How will data be requested and/or obtained?
  • What tools will be used to perform data analytic tests or procedures and why?
  • How will the tools selected be used (processes, tests)?

Answering these questions will enable the IT auditor to design an effective analytic program. Obviously, the IT auditor must have some knowledge of the organization, systems, data files and tools available, as well as the capabilities of those tools.

Analytic Methodology

The IT auditor will need to follow a methodology in getting the data and analyzing the data properly.4 The approach is similar to that of an IT assurance project or, for internal audit, IT reviews.

Scope
As with all assurance and review projects, the process begins with defining the scope of what needs to be done. This can be done by answering these questions:

  • What is the purpose of the audit?
  • What is the objective of the audit’s conclusions?
  • What parameters need to be applied to the data to accomplish that purpose?
  • Where are the data found in the financial or other system?
  • What is the risk (e.g., in data reliability)?
  • What does the scope of the source data need to be in order to meet the objective and address risk?
  • What other information will impact the nature, timing and extent of the procedures to be performed?

After answering these questions, the IT auditor should be able to determine the best approach to take to satisfy the objectives and purpose. Next, a planning meeting, where issues such as the specific procedures and tests can be discussed, should be set. Consideration should be given to relevant data that lie outside the auditee’s systems and data files (e.g., cloud, data center, industry data), and to any issues in getting data, such as usefulness and reliability. Individuals from both IT and the business should be involved, as both perspectives prove beneficial. IT understands how the data look, where they reside and how best to obtain them, while the business can answer questions on what information may be required to meet objectives and the flow of data from a business perspective.

Data Acquisition
After the scoping step is complete, the IT auditor is ready to acquire appropriate data to properly perform testing that meets the objectives set forth.5 Data acquisition can also be referred to as data extraction. Some common elements to consider include:

  • Developing a request for data (a standard request form should be developed and used)
  • Meeting and reviewing the request form with the appropriate person to obtain the data (maybe with someone in IT, accounting and/or the business)
  • Obtaining a sample of the operational transaction processing system (TPS) data (if necessary to ensure proper layout and manageable process)
  • Inspecting pro forma data (where feasible), before the final extraction
  • Timing (associated with scope and purpose)
  • Transferring extracted data from TPS to IT auditor

One key here is about being efficient. Using a standardized request form, for example, should take less time than other alternatives and will reduce time if the auditor chooses to perform this testing again at a future date. The inspect step is critical to efficiency. To ensure that the data being extracted are satisfactory, IT auditors should have some part of the extraction displayed on a screen or partially printed so they can validate the data extraction template and process (which are examples of pro forma data). Going back later—after the discovery that data are incomplete or inaccurate—can be time-consuming.

Data Validation
This step is also critical to a successful data analytics program, just as any other audit evidence. That is, the IT auditor needs sufficient assurance as to the data’s veracity and reliability before performing tests and procedures. That might include understanding, or even testing, controls on the TPS or the IT function (e.g., IT general controls). The goal is to ensure that the data acquired are the precise data on the TPS and they are sufficient for performing the testing required.

That can be done by:

  • Validating balances independent of the data set extracted
  • Reconciling detailed data to report control totals
  • Validating numeric, character and date fields
  • Verifying the time period of the data set (meets scope and purpose)
  • Verifying that all necessary fields in scoping are actually included in the acquired data set

Validating the data definitions of columns and fields is important because some commands on some CAATs require the target field to be “numeric” or a “date” or “character.” This is also important because the data may appear to be numeric, but be defined, or extracted, as “character.”6

Execution of Program
Once data have been imported and validated, the tests can be executed. These tests, generally speaking, enable the IT auditor to:

  • Gain an understanding of the data
  • Perform ad hoc data analysis
  • Run standard scripts
  • Run nonstandard scripts
  • Sample data
  • Run any other analytic program test or procedure

Documentation of Results
Documentation should provide for a clear understanding of the testing purpose, data sources and conclusions reached. The tests should be repeatable. That is, the information contained in the documentation should allow an experienced IT auditor, with no previous experience with the particular testing, the ability to understand and reperform the testing and get the same results.

The good news is that most CAATs provide some automated recording of tests run and their results. It could be a simple matter of copying and pasting to generate much of the documentation.

Review
All work performed should be reviewed to ensure that the testing procedures have been adequately performed and the results analyzed to look for consistency with conclusions documented. The review should be performed by a qualified person (i.e., an SME). Possibilities include:

  • Self-review (not recommended)
  • Technical review (for scripts and should be performed in conjunction with an independent review)
  • Independent review by a team lead (in charge)
  • Independent review by someone not on the team

Retention and Archiving
Retention provides several benefits and fulfills multiple purposes, including:

  • Regulatory requirements
  • Contractual requirements
  • Reperformance needs
  • Litigation (which often includes reperformance)

Also the nature of things to be retained should be carefully considered. Items to consider include:

  • Program files
  • Scripts
  • Macros/automated command tests
  • Data files

Automated command tests refer to the fact that most CAATs allow the IT auditor to save a complicated command/test as a macro-type object in the CAAT for future usage. It typically is efficient to save certain commands/tests and reuse them on future audits or reviews in which a keystroke or click of the mouse will execute a fairly complex command/test.

In addition to what is retained, the length of retention should also be considered.

Application for IT Assurance

Applying all of these concepts in an IT audit or review is relatively straightforward, but creativity always helps find effective uses for the IT auditor. Data analytics can be effective for IT auditors in both planning (e.g., evaluating risks) and field work, for example, in:7

  • Combining logical access files with HR employee master files for authorized users
  • Combining file library settings with data from the change management systems and dates of file changes that can be matched to dates of authorized events
  • Matching ingress with egress records to identify tailgating in physical security logs
  • Using data analytics results to produce reports (e.g., group, summarize, exceptions/detail of devices)
  • Reviewing table or system configuration settings
  • Reviewing system logs for unauthorized access or unusual activities
  • Testing system conversion
  • Testing logical access segregation of duties (e.g., analyzing Active Directory data combined with job descriptions)

Other fruitful areas include: P-card review, testing of automated controls (by reviewing data associated with the process), Payment Card Industry Data Security Standard (PCI DSS) compliance and testing of general financial reporting.

Conclusion

There are a variety of ways data analytics can provide cost-effective benefits to IT audits and reviews. Creativity in spotting opportunities is certainly a plus. This may require an SME to help identify the majority of the opportunities, but any IT auditor who understands CAATs, data and data analytics should be able to find some ways to improve the audit program or review plan by using data analytics.

Endnotes

1 Singleton, Tommie S.; “What Every IT Auditor Should Know About Using Inquiry to Gather Evidence,” ISACA Journal, vol. 4, 2013, www.isaca.org/archives
2 Singleton, Tommie S.; “Data Extraction, A Hindrance to Using CAATs,” ISACA Journal, vol. 6, 2010, www.isaca.org/archives
3 Singleton, Tommie S.; “What Every IT Auditor Should Know About Transforming Data for CAATs,” ISACA Journal, vol. 5, 2013, www.isaca.org/archives
4 Op cit, Singleton vol. 6 2010 and vol. 5 2013. Steps in this process are described in these two articles.
5 A more descriptive narrative on data acquisition can be found in: Singleton, Tommie S.; “Data Extraction, A Hindrance to Using CAATs,” ISACA Journal, vol. 6, 2010, www.isaca.org/archives.
6 There is much more discussion on cleaning up data in Singleton, Tommie S.; “What Every IT Auditor Should Know About Using Inquiry to Gather Evidence,” ISACA Journal, vol. 4, 2013, www.isaca.org/archives.
7 Hoesing, Michael; “Applying Data Analytics to IS Audit,” ISACA Journal, vol. 4, 2010, www.isaca.org/archives

Acknowledgments

A special thanks to Michele Schaeffer of Carr Riggs & Ingram for sharing her expertise on data analytics and her contributions to this article.

Tommie Singleton, CISA, CGEIT, CPA, is the director of consulting for Carr Riggs & Ingram, a large regional public accounting firm. His duties involve forensic accounting, business valuation, IT assurance and service organization control engagements. Singleton is responsible for recruiting, training, research, support and quality control for those services and the staff that perform them. He is also a former academic, having taught at several universities from 1991 to 2012. Singleton has published numerous articles, coauthored books and made many presentations on IT auditing and fraud.


Enjoying this article? To read the most current ISACA Journal articles, become a member or subscribe to the Journal.

The ISACA Journal is published by ISACA. Membership in the association, a voluntary organization serving IT governance professionals, entitles one to receive an annual subscription to the ISACA Journal.

Opinions expressed in the ISACA Journal represent the views of the authors and advertisers. They may differ from policies and official statements of ISACA and/or the IT Governance Institute and their committees, and from opinions endorsed by authors’ employers, or the editors of this Journal. ISACA Journal does not attest to the originality of authors’ content.

© 2013 ISACA. All rights reserved.

Instructors are permitted to photocopy isolated articles for noncommercial classroom use without fee. For other copying, reprint or republication, permission must be obtained in writing from the association. Where necessary, permission is granted by the copyright owners for those registered with the Copyright Clearance Center (CCC), 27 Congress St., Salem, MA 01970, to photocopy articles owned by ISACA, for a flat fee of US $2.50 per article plus 25¢ per page. Send payment to the CCC stating the ISSN (1526-7407), date, volume, and first and last page number of each article. Copying for other than personal use or internal reference, or of articles or columns not owned by the association without express permission of the association or the copyright owner is expressly prohibited.