Tommie W. Singleton, Ph.D., CISA, CGEIT, CITP, CPA
Using a risk-based approach (RBA) to the IT audit begins with the IT auditor assessing the inherent risk (IR) of the relevant technologies. Some IT risks are generally high, maybe very high, regardless of the industry, type of organization or nature of the individual entity. Some examples of those risks are data transferring between information systems, using a spreadsheet for critical applications and performing customized application development in-house. This article focuses on the last item: change management for custom application development (AppDev).
The next step by the IT auditor is to investigate the control environment to see if the entity has mitigating controls for change management associated with AppDev. The IT auditor needs to assess the control risk (CR) to assess an overall risk associated with AppDev and the audit/review being undertaken. COBIT and other ISACA tools contain a rich set of knowledge and techniques related to this important risk.
This article provides the IT auditor with concepts, techniques, processes and structures that can mitigate the change management risk associated with AppDev. It also provides questions and possible sources of evidence regarding the assurance that mitigating controls could provide.
The COBIT 4.1 process associated with AppDev risk is AI6 Manage change. This process is described as follows:
All changes, including emergency maintenance and patches, relating to infrastructure and applications within the production environment are formally managed in a controlled manner. Changes are logged, assessed and authorized prior to implementation and reviewed against planned outcomes following the implementation. This process assures mitigation of the risks of negatively impacting the stability or integrity of the production environment.1
In COBIT, the control objectives related to the Manage change process are:
These control objectives can be achieved through various practices depending on the capability of the enterprise and the technology involved. One possible set of such practices for AI6 is documented in the COBIT® Control Practicespublication.2 The IT Assurance Guide: Using COBIT®3 provides auditors with guidance on how to assess the adequacy of their enterprises’ design and implementation of their change management processes, based on the COBIT control objective/ control practice content.
Control over the change management process is measured by metrics such as:
Other metrics are suggested for consideration in the COBIT AI6 process content.
The benefits of the COBIT control objectives, control practices, assurance guidance and related metrics examples are that they provide the IT auditor with guidance on appropriate questions to ask in relation to change management processes and activities, suggested sources of evidence of control activities and risk mitigation, and audit procedures to perform.
For instance, in regard to AppDev, is the application programming change:
The IT auditor needs procedures to address these questions and the COBIT-related content referenced previously supports this need. One audit procedure is to pull a sample of AppDev projects for the period under review to see if evidence exists (via inspection or observation) to obtain some assurance that the entity has mitigated the high IR of AppDev by establishing effective control of the enterprise’s change management process by achieving the control objectives described in COBIT AI6.
As a specific focus, the IT auditor should examine enterprise documentation for evidence of the employment of best practices of systems development life cycle (SDLC), which are generally considered mitigating controls for AppDev risks.4
The maturity model for AI6 described in COBIT provides a scale and supporting attributes by which the IT auditor could assess the maturity of an enterprise’s change management process.
In the process of completing the IT audit for AppDev, the IT auditor should remember that change management is enabled by an organizational structure with roles and responsibilities as well as by a process and metrics (measurements). COBIT AI6 acknowledges this multifaceted aspect by supporting process control objectives and performance measurement aspects. The structure aspect specifically related to change management activities is dealt with by the responsible, accountable, consulted and informed (RACI) chart related to AI6. The RACI chart provides guidance on change management roles and responsibilities related to generic change management process activities that support the broader organizational structure considerations addressed by COBIT in another process—PO4.
Another way the entity can mitigate AppDev risks is to have a formal structure to deal with some of the previously mentioned responsibilities and accountabilities that enable process activities to occur in a controlled manner (e.g., authorization, prioritization, alignment with business strategy, entity to whom reports are made).
A Steering CommitteeCOBIT process PO4 Define the IT processes, organization and relationships applies to AppDev controls by providing a necessary formal structure. As part of COBIT’s Plan and Organize (PO) domain, this process is necessarily about the entity as a whole and the general (management) controls over IT.
PO4.3 provides one of the formal structures that is beneficial to mitigating AppDev risks. According to COBIT 4.1, PO4.3 establishes an IT steering committee (or equivalent) composed of executives, business managers and IT management (i.e., it is cross-functional). Its purpose is to:
Other PO4 control objectives such as PO4.5 (IT organizational structure) and PO4.6 (Establishment of roles and responsibilities) should also be taken into consideration.
As can be seen, these purposes fit the risks and needs to direct and control AppDev. Thus, the IT auditor may want to gather information and evidence about the existence of a steering committee and its operating effectiveness.
An Ideal StructureBecause a steering committee is strategic in nature, and because reporting and tracking of AppDev changes is tactical in nature (i.e., lots of things happen in a week’s time and problems need relatively immediate attention), there is a need to consider another level of structure for change control. An ideal structure would be for the BoD to establish a steering committee (or its equivalent) as a cross-functional group responsible for IT projects at the strategic level. This body would, for instance, be responsible for prioritizing and funding IT projects.
But the tactical aspects of AppDev (and other similar IT changes) are probably better suited to a tactical committee that meets more often (probably weekly, but not less than monthly) and is dedicated to solving problems and managing the IT changes hands-on. It may be appropriate for the enterprise to consider using a change management committee to oversee IT-related changes (not just AppDev) from the tactical perspective. The committee should be made up of the business sponsors, representatives of the user groups and the IT function. Tracking the status of changes and resolving conflicts might be better suited at the tactical level than the strategic level (steering committee).
A side benefit of such an organizational approach is that business-unit managers have the opportunity to see changes initiated in other units that have consequences—maybe unintentional—that affect their unit. The change management committee provides the opportunity to vet changes across the business units before certain problems occur.
This proposal is consistent with some other principles and bodies of knowledge in the IT profession. For example, a PMO performs this type of service, function and oversight for IT projects. In Capability Maturity Model Integration (CMMI) from the Software Engineering Institute, there is a hierarchy structure for software development that includes the strategic level (BoD), the middle management level (tactical) and the ground level, where programmers work. So, the change management committee idea could be integrated with a PMO or CMMI-based structure.
AppDev is an area of IT that generally is considered to be high in IR because of the probability of errors or fraud in programs when written and deployed. There need to be some reasonable mitigating controls to provide assurance that changes made to business applications do not adversely impact achievement of business objectives. The COBIT processes AI6 and PO4, and supporting materials, provide specific guidance and information to the IT auditor that can be used to gather evidence and make an assessment about the effectiveness of controls in place.
But beyond just ensuring the adequacy of the process controls that are being, and have been, employed, the auditor should also consider the adequacy of the process and organization to support the business objectives. COBIT materials support such an assessment through the guidance offered in relation to process activities, roles and responsibilities, goals and metrics, maturity levels, control practices and assurance testing.
1 IT Governance Institute, COBIT 4.1, USA, 20072 IT Governance Institute, COBIT Control Practices: Guidance to Achieve Control Objective for Successful IT Governance, 2nd Edition, USA, 20073 IT Governance Institute, IT Assurance Guide: Using COBIT, USA, 20074 For more on SDLC and IT audits, see the IT Audit Basics column: Singleton, Tommie; “Systems Development Life Cycle and IT Audits,” vol. 1, Information Systems Control Journal, 2007.
Tommie W. Singleton, Ph.D., CISA, CGEIT, CITP, CPAis an associate professor of information systems (IS) at the University of Alabama at Birmingham (USA), a Marshall IS Scholar and a director of the Forensic Accounting Program. Prior to obtaining his doctorate in accountancy from the University of Mississippi (USA) in 1995, Singleton was president of a small, value-added dealer of accounting IS using microcomputers. Singleton is also a scholar-in- residence for IT audit and forensic accounting at Carr Riggs Ingram, a large regional public accounting firm in the southeastern US. In 1999, the Alabama Society of CPAs awarded Singleton the 1998-1999 Innovative User of Technology Award. Singleton is the ISACA academic advocate at the University of Alabama at Birmingham. His articles on fraud, IT/IS, IT auditing and IT governance have appeared in numerous publications, including the ISACA Journal.
Enjoying this article? To read the most current ISACA® Journal articles, become a member or subscribe to the Journal.
The ISACA Journal is published by ISACA. Membership in the association, a voluntary organization serving IT governance professionals, entitles one to receive an annual subscription to the ISACA Journal.
Opinions expressed in the ISACA Journal represent the views of the authors and advertisers. They may differ from policies and official statements of ISACA and/or the IT Governance Institute® and their committees, and from opinions endorsed by authors’ employers, or the editors of this Journal. ISACA Journal does not attest to the originality of authors’ content.
© 2011 ISACA. All rights reserved.
Instructors are permitted to photocopy isolated articles for noncommercial classroom use without fee. For other copying, reprint or republication, permission must be obtained in writing from the association. Where necessary, permission is granted by the copyright owners for those registered with the Copyright Clearance Center (CCC), 27 Congress St., Salem, MA 01970, to photocopy articles owned by ISACA, for a flat fee of US $2.50 per article plus 25¢ per page. Send payment to the CCC stating the ISSN (1526-7407), date, volume, and first and last page number of each article. Copying for other than personal use or internal reference, or of articles or columns not owned by the association without express permission of the association or the copyright owner is expressly prohibited.