• Bookmark

Critical Success Factors for Continually Monitoring, Evaluating and Assessing Management of Enterprise IT

By Zhiwei Fu, Ph.D., CISA, CRISC, CGEIT, CFE, CISSP, PMP, and Eric H. Mittnight, CISA, CGEIT, CISSP, PMP

COBIT Focus | 8 June 2015

To achieve their objectives and sustain their competitive edge, it has become increasingly necessary for contemporary businesses to bring more effective and efficient enterprise IT management capabilities to bear across their enterprises. It has also become increasingly evident that a systematic and continuous IT management process monitoring, evaluation and assessment (MEA) program is critical to continually improving IT management capabilities. Such a program is necessary to ensure that organizations maintain a focus on doing things right in an effective and efficient manner.

In many cases, organizations have implemented best practice frameworks (e.g., the COBIT Process Assessment Model (PAM): Using COBIT 5) that enable a reliable, consistent and sustainable plan, build, run and monitor (PBRM) IT management program for enterprise IT. The COBIT 5 framework, executed in alignment with the direction set by the governance entities, enables an organization to achieve its enterprise objectives.

Figure 1—COBIT 5 Governance and Management Key Areas

Source: ISACA, COBIT 5, USA, 2012

The continual support and commitment of management process owners, along with effective planning, scoping and communication, are critical factors for a successful ongoing MEA program. These critical success factors include:

  • Identifying and engaging with key stakeholders (Who)
  • Planning and communicating the in-scope processes (What)
  • Determining assessment frequency and time to execute (When)
  • Employing a risk-based assessment approach with proper prioritization (How)
  • Continually tracking, reviewing and reporting performance to management

Engage Stakeholders and Clarify Scope

For a continuous MEA program to be effective, the organization must clearly identify and communicate with the process owners and stakeholders (Who) regarding the plan, scope, schedule and activities of continual monitoring, evaluation and assessment for those in-scope processes (What). The continuous monitoring, evaluation and assessment of processes is performed at different frequencies for different controls at different time intervals. This requires effective planning and communication of when (at what frequency) and what controls (e.g., management, administrative, technical) shall be monitored, reviewed and assessed. The frequency and scope of the MEA program must be coordinated with stakeholders from the start of the project and on an ongoing basis. This includes clearly defining the processes and activities to be included, the scope of these processes and the capability level for each process—all based on the context within which the processes operate.

Determine Timing/Frequency

It is critical to carefully consider the time to execute and frequency for monitoring these processes and activities (When). The MEA program is a continual and iterative process intended to ensure that the organization is meeting business needs and stakeholders’ requirements. It is a balanced act of manual and automated process capability assessments executed at different time intervals with varying frequency that is needed to ensure complete and continual coverage.

The effective operation of controls, including management, administrative, technical and physical controls, must be performed through a combination of continuous monitoring and review, periodic testing of controls, continuous controls monitoring, management self-assessment, independent assessments, and assurance reviews.

For example, external compliance requirements need to be identified on a continuous basis and monitored for changes that must be complied with from an IT perspective. Policies, principles, standards and procedures then need to be reviewed and updated accordingly to ensure that external requirements are addressed and communicated. Careful planning and execution of these controls provides the business with the assurance of control effectiveness to meet external and internal requirements in a sustainable manner and helps the business prevent and mitigate critical control deficiencies and risk cost-effectively. The seven phases of the COBIT 5 of implementation life cycle, as illustrated in figure 2, provides the guidance for continual MEA.

Figure 2—COBIT 5 Implementation Life Cycle and Continual MEA


Program Management (Outer Ring)

Change Enablement (Middle Ring)

Continual Improvement Life Cycle

MEA Relevance and

What are the drivers? Initiate program Establish desire to change Recognize need to act System performance and conformance improvement and/or gaps to fill; Risk management, control ineffectiveness and deficiencies; Change in external requirements; Know the drivers to change and establish the desire and rationale for improvement.
Where are we now? Define problems and opportunities Form implementation team Assess current state Know the current state of system and conformance performance; Know the current capabilities and risk acceptance level; Know the minimum compliance requirements
Where do we want to be? Define road map Communicate outcome Define target state Define desired state. Incorporated into, integrated with, enterprise road map.
What needs to be done? Plan program Identify role players Build improvements Know the difference and impact between the desired and actual state
How do we get there? Execute plan Operate and use Implement improvements Develop and execute action plans to address the change/gaps
Did we get there? Realize benefits Embed new approaches Operate and measure Assess if the executed actions have achieved the goals
How do we keep the momentum going? Review effectiveness Sustain Monitor and evaluate Incorporate into, or update the enterprise performance management system, and continually monitor and evaluate

Source: Z. Fu and E. Mittnight. Reprinted with permission.

Execute a Risk-based Assessment Approach

A risk-based approach to prioritize continual MEA for business critical processes and activities can be taken. Such an approach, executed in alignment with business objectives and requirements, prioritizes recommended process improvement areas. It is generally understood that the higher the process capability, the lower the risk of the process failing to meet its intended purpose, and that the higher the capability, the more costly the process is to operate. Programs must continue to take a holistic, program-oriented, rather than task-focused, risk and security management approach to continue to manage and improve IT service management processes at the program level.

Continually Track, Review and Report Performance

Finally, it is critical to maintain timely, accurate tracking and reporting of MEA program results, with proper context, to support management of enterprise IT and business decision-making processes. For an MEA program to be successful, it is vital to:

  • Collect, validate and evaluate business, IT and process goals and metrics
  • Work with stakeholders to define, periodically review, update and approve performance and conformance targets within the performance measurement system
  • Collect process performance and conformance data
  • Monitor processes to be sure they are performing against agreed-upon performance and conformance goals and metrics and provide reporting that is systematic and timely

This provides transparency of performance and conformance and continually drives achievement of goals. Contemporary business management has already moved beyond “you cannot manage what you cannot measure.” Organizations today need to understand and manage their IT environments holistically end to end, in order to identify and analyze control and process deficiencies and their underlying root causes as well as take proper preventive, corrective and remedial actions to meet business needs and external and internal requirements. Figure 3 illustrates a sample continual process and risk management dashboard for tracking and reporting.

Figure 3—Continual Process and Risk Management Dashboard

Source: Z. Fu and E. Mittnight. Reprinted with permission.
View Large Graphic


Is the senior principal of governance, risk and compliance (GRC) and cybersecurity at IBM Global Business Services. He has an extensive background in designing, implementing and assessing governance and compliance programs and IT controls in various industries and third-party service organizations. He is a renowned researcher and practitioner in business analytics, modelling and optimization, performance measurement and process improvement, with multiple publications in international journals, book series, and conference proceedings.

Eric H. Mittnight, CISA, CGEIT, CISSP, PMP

Is a senior managing consultant with the IBM Global Business Services Cybersecurity & Privacy (CS&P) Practice (US). He has extensive experience in designing, implementing, executing and assessing governance and management of enterprise IT processes across a number of US federal government agencies. He is an experienced IT project manager and IT management consultant across a number of disciplines, including: IT governance, business transformation, enterprise IT strategy, enterprise architecture, IT portfolio management, IT release deployment management and security management.