GEIT Framework at Work, Part 5: Confirming the Results

There comes a point in any long-term initiative when the issues have been identified, the mitigation plans developed, the appropriate work products defined, and those products and plans have been implemented. That is where the process described in this series of articles finds itself. In the last installment, the APO13 Manage Security process was fully implemented and stands ready to operate as business as usual. It is often at this stage in a project that those involved congratulate themselves, consider it a job well done and move on to the next initiative. But that would be leaving the project incomplete. Despite all the best efforts, things do not always go according to plan. Therefore, the final step, discussed in this article, is to verify that the process is operating effectively, as evidenced by accomplishment of the process goals and metrics.

The process goals and metrics are found at the front of each process in COBIT 5: Enabling Processes . They are not presented with a figure number but appear with each process nonetheless. The process goals and metrics for APO13 Manage Security are shown in figure 1.

Figure 1—APO13 Process Goals and Metrics

Each goal has a number of related metrics that specify artifacts that can be evidenced. Confirming each means generating each of these artifacts and then comparing these actual results against their intended design. The confirmation of results is done by testing the existence and operating effectiveness of each process goal. The personnel performing the confirmation exercises should include the project management office and, possibly, internal audit personnel with appropriate qualifications and work experience.

Process Goal Confirmation

Each process has a series of goals and related metrics that are used to determine and measure whether and how well the process is performing. Performance measurement of a process will be better enabled, or facilitated, by incorporation of these metrics when processes are initially designed. When the designers, or architects, of the governance structure incorporate these goals and metrics into the design, the performance measurement that occurs later will be facilitated and made easier to follow and the results more meaningful. Performance measurement will be performed by process owners and internal audit. The results obtained from internal measurement are used to refine process improvement and to support management assertions regarding the effectiveness of internal control.

1. Confirming that a system is in place that considers and effectively addresses enterprise information security requirements
   • Metric—Number of key security roles clearly defined
     Early in the project, responsible, accountable, consulted and informed (RACI) charts were developed and the roles described for each practice were put into the organizational structure, or at least confirmed to already exist. This is the first place to determine and confirm that appropriate security roles exist and are operational. The information security management system (ISMS) policy should also stipulate that security personnel are qualified and maintain their knowledge.
     Obtain organization charts showing the security roles and names of current employees occupying those roles. Request from human resources (HR) evidence that knowledge requirements exist that represent minimally acceptable technical skills for each role. Training plans for each role would be considered a higher level of capability and evaluated favorably.
   • Metric—Number of security-related incidents
     As soon as they are available, ask a service manager to provide copies of recent service incident logs. Ensure that incidents are recorded in a timely manner, the issue is adequately described, and related controls are detailed and have resolution plans attached to them. Every incident must have an owner who can provide resources and status on incidents owned.
2. Confirming that a security plan has been established, accepted and communicated throughout the enterprise
   • Metric—Level of stakeholder satisfaction with the security plan throughout the enterprise
     The incident resolution process should have a customer satisfaction function. Additionally, incident resolution statistics should be reported on periodic dashboards that provide transparency to all interested stakeholders.
     Customer satisfaction concepts and reporting should be reviewed and approved by the chief information security officer (CISO). Periodic reporting most likely comes from a service manager. Obtain examples of recent polls or surveys demonstrating the measurement of incident resolution satisfaction.
   • Metric—Number of security solutions deviating from the plan
     An appropriate security solution can deviate from the security plan, but when that happens there must be sufficient documentation providing a reasonable business use case and rationale. Deviations should occur infrequently and must be reviewed and approved by the relevant party, CISO, chief information officer (CIO), or another member of the enterprise C-suite or their delegates.
     Request all instances of plan deviations and confirm appropriate review and approval. Plan deviations can be provided by the approving party or by service or information security managers.
   • Metric—Number of security solutions deviating from the enterprise architecture
     As with deviations from the plan, there should be few deviations from the architecture. Anything that causes deviations in the architecture must be contemplated carefully as knock-on effects in enterprise architecture can cause unintended consequences that create material increases in risk.
     Request all instances of architecture deviations and confirm appropriate review and approval. Architecture deviations can be provided by the approving party or by service or information security managers.
3. Confirming that information security solutions are implemented and operated consistently throughout the enterprise
   • Metric—Number of services with confirmed alignment to the security plan
     Alignment with the security plan should be well documented in the implementation process of a service. An enterprise’s software delivery life cycle (SDLC) provides the mechanisms and documentation standards for this evidence. Additionally, the key stakeholders of a service will maintain up-to-date documentation related to security plan alignment.
     Request evidence of services from internal audit, process owners, service managers and information security managers. Service catalogs, or inventories, should fully describe security plan alignment.
   • Metric—Number of security incidents caused by nonadherence to the security plan
     The incident management system will apply root cause analysis (RCA) to incident investigation and resolution determination. The cause of an incident must include whether noncompliance is a contributing factor.
     Request recent records of any incidents where RCA indicated noncompliance. Information security managers can provide incident logs.

The overall assessment of the accomplishment of the process goals must be assessed in totality. All evidence collected must be taken as a whole and an opinion derived based on a comprehensive view.

Conclusion

The next installment in this series will discuss collecting evidence of the operating practices and using the results to measure performance of the governance system and to identify areas for potential improvement.

Peter C. Tessin, CISA, CRISC, CISM, CGEIT

Is a senior manager at Discover Financial Services. He leads the governance group within business technology (BT) risk. In this role, he is responsible for ensuring that policy, standards and procedures align with corporate objectives. He serves as the internal party responsible for regulatory exam management and is the internal liaison to corporate risk management. Prior to this role, Tessin was a technical research manager at ISACA where he was the project manager for COBIT 5 and led the development of other COBIT 5-related publications, white papers and articles. Tessin also played a central role in the design of COBIT Online, ISACA’s website that offers convenient access to the COBIT 5 product family and includes interactive digital tools to assist in the use of COBIT. Prior to joining ISACA, Tessin was a senior manager at an internal audit firm, where he led client engagements and was responsible for IT and financial audit teams. Previously, he worked in various industry roles including staff accountant, application developer, accounting systems consultant and trainer, business analyst, project manager, and auditor. He has worked in many countries outside of his native United States, including Australia, Canada, France, Germany, Italy, Jordan, Mexico and the United Kingdom.