Demand for digital transformation is increasing as enterprises endeavor to offer more innovative products or services and modernize legacy IT environments. This presents an opportunity for IT auditors to champion the advisory pre-implementation review to senior management. During an advisory pre-implementation review, internal audit provides observations and recommendations to management to enhance the control environment prior to the deployment of a digital solution. The advisory pre-implementation review is a proactive approach that allows for course corrections to be made throughout deployment, as opposed to a reactive response to enable digital transformation that may result in the project team not meeting user expectations.
Unfortunately, senior management may perceive the review as a barrier to their aggressive schedules and milestones, since it may require them to divert resources from key activities to meet with an auditor, explain their project and provide evidence (and follow-up responses). This can be cumbersome for many projects. However, there are innovative approaches that can aid in advisory pre-implementation review communication and execution.
Delivering the Value Proposition
To engage senior management, the IT auditor must deliver a value proposition explaining the benefits of an advisory pre-implementation review. The value proposition should address how new digital technology can be adopted without disrupting the project life cycle by mitigating potential risk and control deficiencies as they relate to project expenditures, product specifications and the delivery timeline.
Digital transformation is pushing the boundaries for many organizations, so a value proposition may seem daunting. However, by putting themselves in senior management’s shoes, it can be easier for IT auditors to create and deliver value propositions. For most senior managers, an effective value proposition simply communicates that the review could help them minimize unnecessary risk, that their expenditures will remain predictable, and that the product or service’s implementation will go live on time while meeting its specifications. However, a particularly complex or lengthy (i.e., multiyear) digital transformation project may warrant a value proposition that is delivered in incremental steps, in which case an Agile approach can be used to adjust the scope and approach of the advisory pre-implementation review.
In essence, senior management must be assured that a product or service will be delivered on time and within budget. IT auditors can help senior management avoid the potential areas where most projects fail by engaging them earlier in the project life cycle.
IT auditors can help senior management avoid the potential areas where most projects fail by engaging them earlier in the project life cycle.The same argument can be made in terms of software defects and the relative cost to remediate them during the project life cycle. When an enterprise identifies software defects early in the project life cycle, it can account for them and plan the appropriate remediation prior to deployment. The relative costs to remediate defects increases as the software development life cycle (SDLC) progresses. IT auditors can use this line of thinking to make a case for advisory pre-implementation reviews, during which observations and recommendations can be shared early in the project life cycle. Attempting to remediate audit findings during production increases the cost by 2500% to 3000% compared to remediation during development, which increases cost by 500%.
Figure 1—Relative Costs to Remediate Defects in the SDLC
Timing the Review
Ideally, senior management will request the pre-implementation review when it begins to gather requirements and design the new solution. If an auditor performs the review at that stage, it can help prevent costly remediation in the future. But in most cases, the auditor is engaged when the project team transitions from development to testing. At this stage in the project, the auditor can help make course corrections. There is still a cost, but it is easier to plan remediation prior to deployment to production.
It is more challenging when senior management are preparing to deploy a solution but have an unsettling feeling that the project is not ready—and only then is the IT auditor engaged. The same applies to a situation wherein the project team has deployed the solution and received complaints that the solution is not delivering the functionality as expected. In these scenarios, everyone is firefighting to remediate the solution as quickly as possible without optimal resources and while facing extra costs.
Defining Review Objectives
To deliver the value proposition, the auditor must define objectives for the advisory pre-implementation review. Objectives should be determined carefully, as the auditor may inadvertently overpromise and underdeliver, or overwhelm the project team with too many objectives and cause them to lose sight of the value proposition.
Consider the following 2 objectives, which serve as the benefits supporting the value proposition:
- Provide management with an independent assessment of the progress, quality and attainment of project objectives at defined milestones within the project timeline.
- Provide management with an evaluation of the internal controls of proposed business processes and technologies at a point in the development cycle during which enhancements can be easily implemented.
Explaining Key Risk and Focus Areas
After explaining the objectives of the review, it is time to explain the key risk and focus areas.
The following potential risk and scope areas may apply to a project:
- Access control:
- How has user account administration been implemented in the new system, particularly roles and permissions?
- Who has been granted access to privileged accounts and how are the accounts monitored?
- Has senior management performed any access certification reviews to ensure that access levels are appropriate throughout the project?
- IT change and release management:
- Has management followed a change and release management process throughout the project life cycle? Is it monitored?
- Have separate test and production environments been established? Are the environments similar or different?
- Have backup and recovery procedures been established and tested prior to go-live with the new system?
- Application controls and business processes:
- Have the requirements and design of the new system been clearly documented, especially the processing integrity (i.e., input/output/processing/error handling) of the data throughout the new system?
- Are there any new IT-dependent manual controls which require validation?
- Are the management reports complete and accurate? Have they been validated prior to go-live?
- Testing, training and documentation:
- Has management clearly documented its testing strategy and execution plan?
- Does management use a defect tracking tool? How are defects managed?
- Does the user training align with new business processes and has training been scheduled to coincide with any user acceptance testing?
- Has management prepared any support documentation for the end users and the team that supports the system?
- Project management governance:
- Has management established a project governance framework for the oversight of the project (i.e., steering committees, escalation paths)?
- Has management established a project management model for the execution and delivery of the project (i.e., status reporting, project tracking, Risk, Issue, Assumptions, and Dependencies [RAID] logs)?
- Data management:
- What is management’s plan for data conversion and migration?
- Have all data been mapped between old and new systems?
- How is data quality protected during the migration?
This is not an exhaustive list, but a foundation on which one can base their review. Focusing on 1 or 2 key risk factors at a time can help provide quick wins to senior management.
Approaching the Project
To minimize disruption to the project team, the auditor can consider several options to gain an understanding of the project prior to planning and fieldwork. The first option is reconnaissance. The auditor can ask the project team if it has a SharePoint site or shared drive where project documentation can be reviewed and information related to the key risk and scope areas can be gathered. The auditor should consider what documentation has been reviewed and approved by senior management and ask themselves if the current state of documentation is an indicator of project success (or failure).
Another strategy is to issue a survey related to key risk and scope areas. Pre-selected response options that go beyond simple yes/no answers should be offered since they will provide the most insight. Conducting the survey using a program such as Microsoft Forms or SurveyMonkey makes it easy for audit stakeholders to complete the survey and for the audit team to collect and analyze responses.
The auditor can consider using a combination of the mentioned options. The objective is to gather as much knowledge as possible prior to planning and fieldwork to develop an efficient and effective approach to conduct the advisory pre-implementation review.
Conclusion
IT audit practitioners can serve as trusted advisors and agents of change during a time of increasing velocity of change in technology and digitization. Executing an advisory pre-implementation review can help senior management realize a higher return on investment (ROI) in the project development life cycle through minimization of unnecessary risk, assurance that project expenditures will remain predictable and ensuring that the product or service’s implementation will go live on time while meeting its specifications. Performing an advisory pre-implementation review allows IT auditors to help organizations deliver innovative products and services that benefit clients, colleagues and the community, and reinforce trust in their use.
Michael Podemski, CISA, CISM, CRISC, CDPSE
Is a senior director at Aon leading global internal audit assurance and advisory services in the areas of IT and security risk. He is primarily responsible for independent third line of defense (3LOD) oversight of all IT and security-related audit activities, including managing the day-to-day supervision and guidance of IT audit team members and coordinating, planning, executing, and reporting on audit projects. Podemski provides objective risk-centric assurances with respect to the design and/or operating effectiveness of risk management practices, governance processes, and the system of internal controls associated with IT and security-related audit activities. He also serves on the board of the ISACA® Chicago (Illinois, USA) Chapter assisting with event planning and certification review courses.