What Every IT Auditor Should Know About Using Inquiry to Gather Evidence 

 
Download Article Article in Digital Form

Most IT audits—be they assurance-driven, consulting-driven or internally driven—require the IT auditor to assess risk, develop a plan and use that plan to gain evidence about the audit objectives. Inquiry is a procedure commonly used in gathering evidence.

Inquiry Framework

It could be said that there are two kinds of inquiry: personal interviews and questionnaires. There is value in both, and they have some exclusive advantages and disadvantages.

The interview has several advantages over a questionnaire. For instance, in the interview, the auditor can be alert for visual or audible cues about the person’s veracity. Research shows that people who are under stress because they are lying generally show it with body language, speech cues or other behavioral traits. Other information may be gathered from the person’s behaviors and reactions as well.

The interview also has the advantage of the nature of conversations vs. a dry list of boxes to check. For instance, the interviewer can use open-ended questions. Also, sometimes the interviewer or interviewee gets into a flow of conversation. Be it better recall on the part of the interviewee or more effective questions on the part of the interviewer, this flow often generates more pertinent information. When using questionnaires, a flow is just generally not possible. Interviews are also a more effective technique when the nature of the questions call for insights, assessments and other analytically processed information.

The disadvantages of interviews include the availability of both parties, the ability for the auditor to be physically present with the interviewee, distractions that can occur in the interview location, the time it takes to create the interview questions, the time it takes to transcribe an interview and other resource constraints.

A questionnaire has advantages unique to its nature. The process can be made concise, thus limiting the amount of time the auditee must invest in the process of providing answers. It can use standard, professionally developed questions that are known to be effective. For financial audits, there are providers of audit tools, including questionnaires, for each aspect of the audit that needs information gathered. Thus, the time involved in developing the questionnaire can be minimal, but still relatively effective. The IT audit profession is, in fact, replete with questionnaires related to various types of audits, audit procedures or audit objectives.

One key advantage of questionnaires relates to the nature of the questions. If the IT auditor is trying to gather answers to questions about systems, applications and technologies that are fact-based, a questionnaire works quite well.

The disadvantage of a questionnaire can be the loss of the advantages of an interview; for instance, the information gathered in interviews is generally richer. There is also a temptation for the interviewer’s and interviewee’s minds to slip into neutral with planned and canned questionnaires, because of the lack of engagement. There are many things about auditors that make them professional and valuable, but none more important than their analytical thinking and mind-set. To become mindless to questionnaires is to forfeit this valuable asset. Therefore, IT auditors need to resist the temptation to copy a questionnaire from last year (or pull it from the audit methodology provider’s kit), get answers to the questions this year and check off on the audit plan that this step has been completed—something affectionately referred to as SALY (same as last year). Another potential disadvantage is when the auditee fails to complete the questionnaire, misunderstands questions, or otherwise provides incomplete or incorrect information. In the interview, it is likely the auditor would catch such situations and correct them immediately. Thus, the questionnaire is more costly in correcting information errors.

Figure 1There is no one perfect way to gather information using an inquiry approach. Consideration should be given to all approaches, and if the questionnaire/checklist is used, care needs to be taken to keep the auditor’s and interviewee’s mind engaged and to provide an analytical approach to the information being gathered (figure 1).

Inquiry Effectiveness

There are several manners of gathering evidence. One popular framework is: inquiry, observation, examination and inspection/reperformance. These types of tests vary in terms of strength and are generally seen as depicted in figure 2.

Figure 2

Figure 3Thus, the audit objective and the assessed level of risk have an effect on the type of test chosen. In general, the higher the risk, the greater the need for inspection or reperformance as the type of test for gathering audit evidence. Obviously, inquiry is viewed as providing the least amount of assurance and, thus, has a low level of reliance as evidence. There are a variety of reasons why inquiry, or inquiry alone, may be insufficient in developing competent evidence (see figure 3).

For instance, the American Institute of Certified Public Accountants (AICPA) has stated in its technical literature that evidence from inquiry alone is not sufficient in a financial audit. Clearly, an overreliance on inquiry as evidence can lead to an audit with weak quality. Therefore, the IT auditor needs to take care in choosing inquiry as the type of test. Some of the pertinent questions would include: Which audit objectives are suitable for inquiry? How much of the evidence should be via inquiry? Is the type of test and the assessed risk a proper fit?

A specific example in IT may be helpful to demonstrate some of the nuances in inquiry. The following illustrates a situation that is not uncommon: Executives establish standard operating procedures and a sufficiency of controls, then believe those procedures are being done and those controls have operating effectiveness. However, for various reasons, those controls have been changed and are not as effective as designed, and those procedures have been tweaked by well-intentioned (or sometimes malicious) employees and are not functioning as planned.

An example from a real audit involves an interview-type inquiry in which two C-level executives were asked about access controls. When asked who had access to a certain high-risk function, the IT auditors were given a very short list (good news so far). When asked if anyone else could get access to the function, the answer was no one (feeling good about access controls). But when the IT auditors used an inspection test for the access controls—due to the fact that a high level of risk was assessed to this function—they discovered that several other people had access and that a key senior executive actually assigned login credentials and kept a handwritten list of all of them. That fact was unknown to the other executives and apparently to everyone except those who had been granted access—who assumed the credentials process was authorized and standard operating procedure.

Therefore, IT auditors need to be careful about relying on inquiry evidence when obtained from senior managers and executives who may be under the wrong impression about the procedures and controls they designed and believe were implemented. If there is any significance in the difference between design and operations, it could cancel out the assurance that the inquiry appears to provide. This possibility exists in almost every IT audit, regarding some aspect of the entity’s procedures (business processes) and controls. The further the inquiry participant is from front-line employees and processes, the more likely this scenario becomes. Thus, when conducting an inquiry of senior management or executives either via interview or questionnaire, the IT auditor should take care in confirming the inquiry information; that is, less reliance is placed on inquiry as evidence.

Other dangers include the aforementioned temptation to focus on the answers of a questionnaire and mentally disengage. The previous example shows how that can be devastating in the wrong set of circumstances. However, the evidence gathered in the inquiry was not difficult to confirm with another test type.

A set of dangers could be labeled as: things that should be operating effectively but are not. For instance, as mentioned previously, employees have been known to tweak business processes or controls (manual or IT-dependent controls) to make their job easier, but the end result is a detriment to the overall control system or effectiveness of a business process. The danger here is when two things happen:

  • The employee makes unauthorized adjustments and/or changes to business practices or controls and does not communicate that change to the proper authorities.
  • The change is to the detriment of the overall internal control structure or effectiveness of business processes.

Employees can also simply fail to follow standard operating procedures for business processes or for executing controls (particularly manual controls). It might be unreasonable to expect every employee to perfectly execute every business process and every control every time. The question becomes: What is the impact of those failures individually and/or in the aggregate?

Controls, except for possibly automated controls, may suffer from atrophy. Such a state could develop because employees get careless with manual or IT-dependent controls, and thus, they become less effective. It could be that, because of external circumstances, business processes have changed and the needs of the system have changed, but the system of business processes and/or controls has not changed and is now becoming less effective. Even automated controls can suffer from atrophy as a result of updates to IT, vulnerabilities that develop and other similar issues.

An IT example would be access rights. It is not uncommon for an investigation of access rights to reveal a host of risk issues. For example, all IT personnel are sometimes granted administrator rights to keep support simple. However, that is a serious violation of best practices and introduces a high risk factor. The same could be said for database administrator rights. In fact, access rights in general sometimes lack a least-privilege approach.

Another group of dangers could be described as: things that are that should not be. These can be seen as a failure to properly carry out the authorized procedures or controls. A good IT example involves the testing of new technologies, especially applications, where an employee is given elevated access rights for the testing, but once testing is completed, the elevated access rights are not returned to the proper level. Thus, the employee now has access rights greater than he/she should have.

A similar situation exists when employees are terminated. Access rights for a terminated employee should be concluded in correlation to that person’s date of termination.1 Deleting access rights for terminated employees is an area of concern in all IT audits where access rights are in scope.

A third category or group of dangers is nefarious activities. Unfortunately, human nature is such that the business community will never successfully eliminate nefarious activities. In fact, the opposite is true today. Never before has there been more risk, more nefarious activities, more cybersecurity issues than today. Just a casual reading of news or professional literature reveals the common concern in business related to cybercrime and cybersecurity. The greatest threat today comes from the millions of potential intruders who can gain unauthorized access to an entity’s system and databases, combined with the vector of spear phishing, and the level of IT expertise possessed by the modern, sophisticated cybercriminal.

There is also the possibility of an entity’s own employees conducting malicious or nefarious activities against the employer. For instance, in one IT audit, the IT auditor discovered that a key senior manager had managed to edit the login application and have the login credentials bypassed (if-then-else statement, where if employee # = key manager, skip login). This is an example of a backdoor that allows the intruder access to a wide variety of applications and data. Worse, this situation is not some accident or oversight or poor execution of procedures and controls; it is purposeful and likely intended to conduct harmful activity against the entity.

Conclusion

Inquiry has a lot of advantages in an IT audit and has been successfully used millions of times for millions of audits. However, the IT auditor should take into account a couple of factors that can mitigate the reliance upon inquiry evidence. First, care should be taken in the nature of the inquiry—a questionnaire vs. an interview. Second, the IT auditor should take into account factors that could reduce reliance on inquiry evidence (see figure 3). But, in the end, it is similar to what auditors have done since the beginning of audits: Collect the evidence, make sure the evidence is reliable and draw audit conclusions. The slippery issues are those about mistakenly relying on inquiry when, with some careful thought, the IT auditor could search out corroborating, or superior, evidence.

Endnote

1 Sometimes entities will grant the terminated employee access rights to the entity’s email for some period of time in order to transition the contacts and sources of email.

Tommie Singleton, CISA, CGEIT, CPA, is the director of consulting for Carr Riggs & Ingram, a large regional public accounting firm. His duties involve forensic accounting, business valuation, IT assurance and service organization control engagements. Singleton is responsible for recruiting, training, research, support and quality control for those services and the staff that perform them. He is also a former academic, having taught at several universities from 1991 to 2012. Singleton has published numerous articles, coauthored books and made many presentations on IT auditing and fraud.


Enjoying this article? To read the most current ISACA Journal articles, become a member or subscribe to the Journal.

The ISACA Journal is published by ISACA. Membership in the association, a voluntary organization serving IT governance professionals, entitles one to receive an annual subscription to the ISACA Journal.

Opinions expressed in the ISACA Journal represent the views of the authors and advertisers. They may differ from policies and official statements of ISACA and/or the IT Governance Institute and their committees, and from opinions endorsed by authors’ employers, or the editors of this Journal. ISACA Journal does not attest to the originality of authors’ content.

© 2013 ISACA. All rights reserved.

Instructors are permitted to photocopy isolated articles for noncommercial classroom use without fee. For other copying, reprint or republication, permission must be obtained in writing from the association. Where necessary, permission is granted by the copyright owners for those registered with the Copyright Clearance Center (CCC), 27 Congress St., Salem, MA 01970, to photocopy articles owned by ISACA, for a flat fee of US $2.50 per article plus 25¢ per page. Send payment to the CCC stating the ISSN (1526-7407), date, volume, and first and last page number of each article. Copying for other than personal use or internal reference, or of articles or columns not owned by the association without express permission of the association or the copyright owner is expressly prohibited.