JOnline: Framework for Measuring and Reporting Performance of Information Security Programs in Offshore Outsourcing 

 
Download Article

In today’s economy, outsourcing is an essential business strategy. Outsourcing helps enterprises save money by leveraging the economies of scale realized by the outsourcing suppliers. When effectively managed, it also helps relieve enterprises from the problems of manpower constraints, skills shortages and operating efficiencies.

In pursuit of benefits such as less expensive labor, availability of a large pool of talented resources and faster time to market, enterprises utilize offshore service providers from countries such as India and China. Many enterprises take to offshore outsourcing end-to-end, engage the service provider as a partner, and strive for achieving performance improvements, productivity gains and continuous innovations over time.

This offshore outsourcing, however, is not without risks and challenges. Some of the key challenges in this scenario are:

  • Multiple organizations and their security systems are now involved, making security system design and ongoing operations complex.
  • Since different countries are involved, different laws and regulations are applicable. Careful management is required to ensure that legal and regulatory compliance is achieved.
  • Offshore service providers usually cater to the needs of many enterprises (clients), and this results in additional complexities.
  • Priorities and standards of the service provider for business continuity and disaster recovery preparedness can be different from those of the client on many occasions, and this could result in complications.
  • Work is conducted “out of sight” at remote locations, and this results in an apparent loss of control unless suitable ways are implemented to ensure ongoing control.

While experienced companies have been outsourcing IT development and business back-office functions for many years, inexperienced organizations may be rushing to jump on the offshore outsourcing bandwagon too prematurely. The importance of proper offshore outsourcing plans and risk management is evident from a recent study published by DiamondCluster International, a US-based global management consulting firm. DiamondCluster found that the number of buyers prematurely terminating an outsourcing relationship has doubled to 51 percent, while the number of buyers satisfied with their offshore providers has decreased from 79 percent to 62 percent.1, 2

Effective information security management becomes crucial in this scenario. This article describes a framework that can be used for measuring and reporting the performance of information security in such an instance.

What Is an Information Security Program?

An information security program involves the overall combination of technical, operational and procedural measures and management structures implemented to provide the confidentiality, integrity and availability of information based on business requirements and risk analysis.3 An information security program depends heavily on the processes established as a result of information security governance efforts in the organization.

Information security governance aims to develop a system through which organizations are directed and controlled, and integrate security planning in the larger context of IT and business planning. Security governance involves the development and integration of a management structure and organization with reporting processes that encompass all aspects of a successful security program and provide assurance to businesses that risks are defined and appropriately managed.4 Information security governance is essentially the responsibility of top management.

ImageEffective information security governance and information security programs involve a wide range of activities. Some of the major categories of these activities are:

  • Issue and regularly update policies, procedures, standards and guidelines
  • Actively monitor for compliance, risks and changes to the context
  • Conduct intrusion testing with appropriate detail and regularity
  • Design appropriate security defenses and periodically enhance these defenses
  • Effectively manage security to ensure timely action and create business-relevant value

Figure 1 illustrates the activities that help ensure effective information security governance.

In every organization, information security cuts across the entire organization and requires involvement of employees at all levels. In the scenario of offshore outsourcing, information security processes span across the organizations in multiple countries (with the offshore center forming an extended part of the outsourcing organization), requiring effective orchestration of the activities across the extended organization.

Offshore Outsourcing and the Information Security Program

Enterprises typically pursue outsourcing after establishing a process and carrying out a number of activities. These activities can be grouped under:

  • Preoutsourcing activities—These are carried out by the organization before an outsourcing contract is established with the service provider. Examples include:
    • Carrying out a due diligence exercise to select a service provider
    • Assessing security risks involved
    • Assessing security competency of the service provider
    • Establishing security systems and processes
  • During outsourcing—These activities are to be carried out during the award of the contract and throughout the duration of the contract. Examples include:
    • Establishing the contractual obligations and setting up the master service agreement elaborating the service level, security and operational requirements, review and audit mechanisms, and various contractual processes
    • Assessing the continual performance of the service provider
    • Establishing the information security governance of the outsourcing, covering the structure, roles and responsibilities to manage the various aspects of the engagement, including information security. Reporting responsibilities, methods and associated aspects are also established.
    • Continually enhancing the information security programs
  • Postoutsourcing activities—These refer to the activities that the organization is to perform upon termination of the outsourcing contract.

Many service providers establish comprehensive security management systems complying with standards such as ISO 17799 (BS 7799), and they also pursue certification to enhance the information security. Many service providers pursue ITIL (BS 15000/ISO 20000) to enhance the service orientation of the IT services. Multiyear contracts are common in these outsourcing engagements, and the parties establish long-term contracts.

The enterprise and the outsourcing service provider approach the engagement as partners and establish systems and processes for the success of the relationship. In particular, the parties work to align the security systems at the offshore center in line with the security systems and requirements of the client.

Framework for Metrics and Reporting

The proposed framework aims at security performance measurement and feedback at various levels (strategic, tactical and operational) at the service provider organization and connecting the client (outsourcing its requirements) with a focus on the offshore services.

Elements of the Framework

The framework involves the following setup:

  • Metrics are worked out for the various performance aspects and are monitored on an ongoing basis using the systems and processes at the offshore service provider end. They are validated as required.
  • Performance measurement scorecards are derived and used to track performance at the following levels:
    • Strategic or business level
    • Tactical or IT/IS level
    • Operational or IT process level
  • A dashboard is maintained for the management by the service provider.
  • A dashboard is maintained for management of the client.

The reporting system is established for appropriate feedback to concerned personnel in both organizations and also for tracking these actions.

Along with the dashboards, the scorecards support the overall management structure, organization and responsibilities that are defined at both the service provider and the client, resulting in greater alignment, better cooperation and more timely action.

KGIs and KPIs

The scorecards are derived using key goal indicators (KGIs) and key performance indicators (KPIs).

According to the IT Governance Institute’s Management Guidelines, a KGI is defined as “a measure of what has to be accomplished.” A KPI is “the measure of how well the process is performing.”5

KGIs and KPIs at the various levels are interrelated. Business KGIs are dependent on the KPIs at the business level and, in turn, on KGIs and KPIs at the lower level. Figure 2 shows this hierarchy.

Image

Typical business-level KGIs at the offshore center include:

  • Average cost of security incident during a period
  • Cost incurred to deal with known threats
  • Downtime of critical operations due to security incident
  • Number of projects stopped or delayed due to security incident/issues
  • Speed of dealing with a new threat
  • Time taken to enroll an agent/employee
  • Number of reported security incidents
  • Number of incidents that are handled without resulting in a crisis
  • Time taken to implement a regulatory requirement
  • Availability of an information security management system (ISMS) that meets the requirements of the client and is suitably certified/accredited

This list is partial and suggestive only; actual indicators will depend on the scenario and priorities.

The organizations will have to decide on KGIs in the context of their priorities.

It may also be noted that business-level KPIs, in most cases, are the same as the tactical-level KGIs.

Typical tactical-level KGIs at the offshore center include:

  • Percentage of assets covered by systematic risk assessments
  • Number of vacancies in the security roles required for the ISMS
  • Time taken to grant, change and remove access privileges
  • Percentage of agents covered by an effective security awareness program
  • Number of security access violations
  • Number of emergency changes
  • Number of security incidents involving malicious code
  • Number of systems where security requirements are not met
  • Average turnaround time of incidents
  • Number of pending actions to meet response and recovery requirements
  • Number of scheduled internal audits not completed
  • Number of scheduled penetration tests not completed
  • Number of overdue actions arising from audit reports
  • Number of changes not carried out as per change control procedures

Again, this list is partial and suggestive; actual indicators will depend on the scenario and priorities of the organization. Tactical-level KPIs are, in most cases, the same as the operational-level KGIs. Typical operational-level KGIs at the offshore center are:

  • Number of violations of segregation of duties
  • Number of users who do not comply with password standards
  • Percentage of suspected and actual access violations
  • Percentage of critical assets covered by internal/external penetration tests
  • Number of computers with patches behind the service level agreement (SLA)
  • Number of outdated policies, procedures, standards and guidelines
  • Number of internal audits scheduled
  • Number of independent reviews scheduled
  • Percentage of agents/employees trained on information security policies and procedures as part of induction
  • Number of corrective/preventive actions taken based on analysis of logs
  • Number of changes not carried out as per change control procedure

This list is also partial and suggestive; actual indicators will depend on the scenario and priorities of the organization.

Typical operational-level KPIs are:

  • Number of obsolete accounts
  • Number of unauthorized IP addresses and ports
  • Number of cryptographic keys compromised and revoked
  • Number of access rights revoked, reset or changed
  • Number of security awareness training programs for agents/employees
  • Percentage of agents/employees covered by fire/evacuation drill
  • Percentage of agents/employees covered by business continuity planning (BCP) training
  • Number of security awareness training programs for thirdparty employees
  • Percentage of logs analyzed on regular basis
  • Number of dedicated security personnel at the offshore location

This list is also partial and suggestive; actual indicators will depend on the scenario and priorities of the organization.

As the organizations work together to set up KGIs/KPIs, the resulting information security program is much more focused in terms of process performance and measured outcomes that can make the relationship more successful. This enables the service provider to meet the requirements of the client. The higher the maturity of the organizations’ information security, the more readily these KGIs and KPIs can be set up and effectively used. The entire framework and ongoing interactions facilitate the efforts in this direction.

Balanced Scorecard for Strategic Performance Measurement

The management teams of the two organizations work together to set up the balanced scorecard for information security governance of the offshore partner. During the process, KGIs and KPIs are also identified for the various scorecards.

The enterprise buying the services considers its own balanced scorecard (implemented for its business) during the process. This reflects the performance requirement on the following perspectives:

  • Financial perspective
  • Customer perspective
  • Internal process perspective
  • Learning and growth perspective

The internal process perspective covers the various internal processes and the requirements of security, effectiveness, efficiency, and compliance with laws and regulations

Reporting on the balanced scorecard results in tracking the KPIs and KGIs and in overall interpretations under the four perspectives. Acceptable levels can be predefined to identify the status as green, amber or red for the purpose of reporting. The balanced scorecard provides the strategic and top-down perspective to the entire measurement and management of the partner.

Reporting on the balanced scorecard can be done on a quarterly basis to start with, and the results can be used for periodic performance reviews of the respective management teams and form part of the strategic reports. As the process settles down, the results can be tracked on a monthly basis.

Figure 3 shows the overall framework.

Figure 4 shows a typical balanced scorecard report for the program. A tactical-level scorecard is a summary report derived from the tracking of KGIs and KPIs at the tactical level. Since KPIs at the tactical level and KGIs at the operational level are the same in most cases, the performance of KPIs is derived from the performance scorecard on KGIs at the operational level.

As organizations set up ISMSs based on frameworks such as ISO 17799 and Control Objectives for Information and related Technology (COBIT),6 these KGIs can be chosen in line with these frameworks, readily summarized into the control domains and presented.

Acceptable levels of the indicators are predefined and are used to work out the report and show the overall status as green, amber or red. This can be reported on a monthly basis to start with and can be used as the report for status reporting at the tactical level. As the process settles down, it can be reported on a weekly basis.

Figure 5 depicts an example of a typical tactical-level scorecard for the program.

Performance at the operational level is tracked using the operational-level scorecard. The KPIs and KGIs for this purpose will help in tracking the operational purposes. Since substantial work at operational level is in respect of compliance tracking against implemented systems, measures are predominantly in the nature of compliance tracking.

Acceptable levels of the indicators are predefined and are used to show the overall status as green, amber or red for each group of the KGIs and KPIs.

Compliance is tracked by reports generated on a more frequent basis (usually weekly to begin with and then daily, if required). Figure 6 shows the operational-level scorecard tracking the KGIs, and figure 7 shows the operational-level scorecard for tracking the KPIs.

The service provider’s management maintains a dashboard where the balanced scorecard, tactical-level scorecard and operational-level scorecard are displayed. Actions initiated/taken to correct or prevent issues are also shown on the dashboard. This enables appropriate oversight by the service provider’s management.

The client also maintains a dashboard that is linked to the dashboard of the service provider. Using this dashboard, the client’s management tracks and takes any corrective actions required. The dashboard can even be maintained as part of the service provider’s automated system, which can provide a dashboard view relevant to the client and enable interaction by the client.

Benefits of the Reporting Framework

The reporting framework has several benefits, including:

  • The framework helps integrate the various information security mechanisms across the partners that are normally used for monitoring and managing an outsourcing engagement.
  • The framework enables an effective feedback system, integrating the reports and actions at various levels with respect to information security. This, in effect, results in a consistent and closed-loop process and engages the process owners and key personnel of both organizations.
  • This framework can facilitate the efforts toward identifying and taking up requirements in a proactive manner and with a more holistic perspective.
  • The tactical-level scorecard can be enhanced to include further tactical parameters that need to be monitored in addition to the control objectives; such additions can be periodically checked with penetration testing, audits and reviews.
  • If an enterprise is outsourcing to multiple service providers (multisourcing), this framework can be extended to cover the various relationships, enhancing the success of information security performance outsourcing.7, 8
  • If a service provider is serving multiple clients, implementing this reporting framework can support the efforts of the service provider to have an information security program appropriate to each client.
  • The framework can help in looking at performance as having more than the financial or cost dimension. A broader perspective has important ramifications and helps in achieving outcomes that lead to more satisfying relationships over a period of time. In particular, the learning and growth perspective emphasized in the balanced scorecard (and at the strategic level) leads to innovations in the overall information security program.
  • The framework can help manage a relationship with many levels of interactions, ongoing tracking and timely reviews, motivating talent and performance, and building trust. Automation of the reporting and feedback system helps set up a seamless team across the organizations.
  • The framework can enable more consistent incorporation of changes, laws and regulations, thereby helping to improve compliance.
  • While each of the organizations can implement its own internal control system according to standards and frameworks decided by business priorities, this reporting framework essentially focuses on information security performance in outsourcing and can help in meeting the information security program requirements of the client.

Implementing the Reporting Framework

This reporting framework is ideally established as the relationship is formed. If the relationship already exists, implementation can be done as a part of a major review, e.g., annual review/goal-setting exercise.

Starting with the balanced scorecard, the KPIs and KGIs are to be firmed up for the various levels, and the individual scorecards are to be finalized and implemented. Along with these, dashboards are to be set up for the management of the service provider and the client.

Effective implementation of this framework requires extensive collaboration and automation of the processes. The overall implementation results in a performance system covering all levels of the relationship and can prove to be a distinct advantage in managing the risks involved in outsourcing. These are among the aspects that justify the investment in setting up a reporting framework for the engagement. With an increasing number of software solutions available for scorecard development and dashboards, these are more readily possible than in the past.

In summary, the framework supports efforts in establishing accountability at the various levels of the organizations, facilitates continuous monitoring and joint decision making, and engages the stakeholders in the overall objectives. By using this framework for all the service providers, the enterprise also benefits from a uniform method of reporting and, thereby, ready analysis and actions. This enhances the success of information security governance and the information security program even when the work is outsourced to distant locations.

Endnotes

1 Bakalov, Rudy; Feisal Nanji; “Offshore Application Development Done Right,” Information Systems Control Journal, vol. 5, 2005

2 DiamondCluster International Inc., “2005 Global IT Outsourcing Study,” 2005

3 ISACA, CISM Review Manual 2005, USA, 2005

4Ibid.

5 Van Grembergen, Wim; Steven De Haes; “COBIT Management Guidelines Revisited: The KGI/KPI Cascade,” Information Systems Control Journal, vol. 6, 2005

6 IT Governance Institute, COBIT 4.0, USA, 2005, www.itgi.org

7 Bednarz, Ann; “Top Outsourcing Deals Focus on Business Process, Multisourcing,” Network World, 19 December 2005

8 Canfield, Bryce; “What’s Next? Top ITO Trends for 2006,” Outsourcing Journal, November 2005

Sekar Sethuraman, CISA, CISM, CISSP, CIA, CSQA, BS7799 LA
is currently head-IT security (Greater Asia) of LexisNexis. He is also currently director-programs of the ISACA Chennai Chapter. He can be reached at sekar.sethuraman@lexisnexis.com.au.


Information Systems Control Journal, formerly the IS Audit & Control Journal, is published by the ISACA. Membership in the association, a voluntary organization of persons interested in information systems (IS) auditing, control and security, entitles one to receive an annual subscription to the Information Systems Control Journal.

Opinions expressed in the Information Systems Control Journal represent the views of the authors and advertisers. They may differ from policies and official statements of the Information Systems Audit and Control Association and/or the IT Governance Institute® and their committees, and from opinions endorsed by authors' employers, or the editors of this Journal. Information Systems Control Journal does not attest to the originality of authors' content.

Instructors are permitted to photocopy isolated articles for noncommercial classroom use without fee. For other copying, reprint or republication, permission must be obtained in writing from the association. Where necessary, permission is granted by the copyright owners for those registered with the Copyright Clearance Center (CCC), 27 Congress St., Salem, Mass. 01970, to photocopy articles owned by the Information Systems Audit and Control Association Inc., for a flat fee of US $2.50 per article plus 25¢ per page. Send payment to the CCC stating the ISSN (1526-7407), date, volume, and first and last page number of each article. Copying for other than personal use or internal reference, or of articles or columns not owned by the association without express permission of the association or the copyright owner is expressly prohibited.