JOnline: Testing Your Computer Security Incident Response Plan 

Download Article

The recent passing of the state of California’s Senate Bill (SB) 24 requirement, which specifies that notices need to be sent to affected parties in the event of a data breach, serves as a reminder that organizations should dust off their computer security incident response (CSIR) plans and test them. As a matter of fact, if an organization stores or processes credit card information, it has to test its CSIR plan annually per the Payment Card Industry Data Security Standard (PCI DSS).

Analogous to regular disaster recovery (DR) testing, CSIR testing can be seen as a necessary evil; however, as Mahatma Gandhi once said, “an ounce of practice is worth more than tons of preaching.”1

This article discusses the genesis for CSIR testing, several testing methodologies and/or exercises with which an organization can assess the maturity of its CSIR plan/program, how to conduct an exercise in communicating an incident to the public, and an enumeration on several CSIR process improvement strategies.

Genesis for CSIR Testing

Figure 1The point for executing a CSIR drill/exercise is to determine the validity of an organization’s CSIR plan. Assuming that the organization has a CSIR plan and that the organization has proper training and awareness processes in place, an organization may use a CSIR test to determine the gaps between the “rubber and the road,” or, as conveyed in Deming’s Plan-Do-Check-Act (PDCA) cycle (figure 1),2 to check that the plan is valid and has been executed satisfactorily.

It is also important to remember that as former US President Dwight D. Eisenhower said, “plans are useless, but planning is indispensable.”3 As such, a CSIR plan should serve as a guide rather than a policy. This is especially important for today’s marketplace in which partners, vendors, suppliers and customers have a level of interaction that has not been seen before. With such a complex business ecosystem, it is important to include these additional parties in testing plans.

CSIR Testing Methodologies

CSIR testing should include the organization’s partners, vendors and suppliers as much as possible; however, the crawl-walk-run approach is highly advocated here, and that may suggest that the organization begin its CSIR testing with a tabletop exercise. A tabletop exercise is intended to start the assessment of the CSIR plan at a high level with the involvement of executive/senior management. In such an exercise, the multiple stakeholders that make up the organization’s CSIR team (CSIRT) and its executive/senior management would meet in a conference room to walk through several scenarios. Suggested scenarios to cover in the tabletop exercise include an internal malicious software (malware) infestation, someone from the media contacting public relations after hours about a successful computer hack, or hearing about a vendor’s breach through the media and not being able to reach that vendor. These suggested scenarios can differ to a degree by industry and jurisdiction; however, it is highly advocated that all organizations have an exercise in place for responding to the media regarding allegations of a data breach. By conducting high-level theoretical exercises, an organization should determine what knowledge and/or process gaps exist in the CSIR plan and CSIRT.

Predicated on the successful remediation of any gaps found during the tabletop exercise, an organization should conduct an internal CSIR simulation exercise. This endeavor should include the involvement of members of the CSIRT leadership and their subordinates in an exercise that would test the CSIRT’s ability to identify, respond to and remediate an internal incident. This exercise should be conducted in a nonproduction environment; however, it is crucial to mimic the production environment as much as possible. Many organizations may find setting up such a laboratory environment cost-prohibitive; in such cases, creative solutions should be identified to assist in this endeavor, namely setting up an isolated network with a colocation provider or DR vendor. If and when an organization conducts a simulation exercise, it is imperative to determine the maturity of the team, tools, plans and processes. As many areas for improvement may be found, an iterative focus on Deming’s cycle may work best for improving the focal point.

Once the internal CSIR processes are acceptable, the organization should include external parties in CSIR testing. This is extremely important, as it has been found that 20 percent of the data breaches affecting organizations in the health care industry, for example, come from an organization’s partners, vendors or suppliers.4 Vendor incident response testing can include both tabletop and simulation exercises. Additionally, the organization should review the CSIR plans of its vendors before engaging in a contract and/or service level agreement (SLA) with them. Cloud service providers (CSPs), IT outsourcing (ITO) firms, business process outsourcing (BPO) firms and/or offsite records-management storage providers are examples of some of the primary parties to reach out to for inclusion in the CSIR tabletop/simulation exercise process. It is also important to be cognizant of whom the organization’s partners, vendors or suppliers use for their products and services, as this can affect the organization as well. A solid way to determine if this downstream vendor management is governed well is to look at the organization’s primary vendors’ CSIR plans. While reviewing vendors’ CSIR plans, it may be worthwhile to determine whether these parties have a CSIRT identified. By incorporating the organization’s vendors into the CSIR process, an organization can gain further understanding of and confidence in the processes used to respond to an incident.

CSIR Communication Exercise

To help organizations that are trying to determine the exact scenario to use to begin testing, the following CSIR communication exercise is provided.

Scenario:  A reporter contacts an organization’s director of public relations on a Sunday afternoon via personal cell phone with a request for comment on the fact that the organization’s web site has been hacked by a person under the pseudonym of Anonymous.

Response:  Per Microsoft’s suggested CSIRT member assignments,5 the director of public relations should be on the CSIRT and, therefore, should be aware of the CSIR process and know his/her responsibilities.

The following highlights the next steps an organization should take in this scenario:

  • The director of public relations should articulate the organization’s planned initial response to the reporter, which should dictate that this claim is hearsay, but that the organization takes the security and privacy of its data seriously and will, therefore, investigate the allegation.
  • The director of public relations should contact the CSIRT leader to discuss the allegation and to formulate a more in-depth response strategy.
  • The CSIRT leader should delegate the incident to a CSIRT incident lead, and if, upon cursory data collection and analysis, the allegation seems credible, the incident lead should document the incident and initiate additional data collection and analysis activities.
    • If at any time the allegations are determined to be false, the CSIRT should move to the reporting step of the CSIR plan.
  • After additional data collection and analysis, if the claim is determined to be true, the CSIRT incident lead should establish a conference bridge to communicate the knowns, unknowns and action items to the CSIRT and executive/ senior management, following a need-to-know approach.
    • During the conference, a timeline for issuing a statement to the press/public should be discussed.
  • Once the affected parties are determined, and while a communication to the public is drafted for release, the CSIRT should execute the resolution and recovery steps of the organization’s CSIR plan.
  • After the incident has passed, the CSIRT should move to the reporting step of the CSIR plan under which an after- action report (AAR) should be completed to determine the strengths, weaknesses, opportunities and threats (SWOT) of the CSIR team, program, plan and processes in regard to this incident. The AAR should be used for process improvement opportunities.

CSIR Process Improvement

Process flows and decision points are a necessary part of the CSIR plan and the CSIRT. A base process for the organization and its partners, vendors or suppliers is necessary. A suggested process model is illustrated in figure 2,6 which may serve as a basis for planning.

Figure 2

Once the organization has a base process in place to be used internally, it can include and/or flesh out the process with its vendors. However, it is necessary to tweak this model as requirements, mandates and contractual agreements change. To further refine the CSIR and CSIRT processes, an organization may use proven process improvement methodologies, such as the Deming cycle/Kaizen, Capability Maturity Model Integration (CMMI) or Lean Six Sigma.

While frequently associated with manufacturing, process improvement methodologies have their place in CSIR and CSIRT efforts as they can be used to strengthen the accuracy, relevance and effectiveness of an organization’s CSIR efforts. The Deming cycle (PDCA), as alluded to previously, is analogous to the Kaizen methodology that was created and implemented in Japan for manufacturing purposes. PDCA is an iterative, open methodology that encourages all facets of the organization to make process improvements.

CMMI was created by the Software Engineering Institute (SEI) at Carnegie Mellon University (USA) to establish a process improvement model for software development. Now used for purposes beyond software development, CMMI includes five levels an organization may strive to achieve with the maturity of its processes. These are level 1, initial; level 2, managed; level 3, defined; level 4, quantitatively managed; and level 5, optimizing.7

Beyond CMMI, there is also Lean Six Sigma, which uses the principles from lean manufacturing and Six Sigma to create a further defined cycle than Deming’s PDCA. Lean manufacturing focuses on executing a process with the minimal amount of resources necessary to produce a quality product, while Six Sigma leverages statistical analysis to reduce defects to a very small margin of error. There are two flavors of Lean Six Sigma: DMAIC and DMADV. DMAIC stands for define, measure, analyze, improve and control, while DMADV stand for define, measure, analyze, design and verify.8 In the interest of CSIR planning and testing, DMADV is more relevant, as it is more focused on communication and work flows.

Regardless of what methodologies are used, once an organization’s processes have been updated and improved upon, the organization may refine the processes, procedures and requirements as necessary.


An organization in today’s climate needs to not only have a CSIR, it needs to test the CSIR as well. And, once the test has flushed out any gaps between the plan and the process, the organization needs to incorporate process improvement methods to close those gaps. It is important to test the plan and team as often as possible with different scenarios; different exercises; and different partners, vendors or suppliers.

An organization should start with internal tabletop testing, including a communication exercise, and then move to simulation testing. Once the organization has completed several internal exercises, it can incorporate partners, vendors or suppliers as necessary. However, it is up to the organization to determine its CSIR testing strategy as well as its CSIR process improvement strategy. In the end, an organization needs to test its CSIR plan just as it must test its DR plan.


2 Walton, Mary; The Deming Management Method, Perigee Books, 1988
4 Jones, Ed; 200 Breaches Impacting Almost 5.9 Million Individuals, With Theft and Loss of Laptops and PEDs Major, 2 December 2010,
5 Microsoft, Responding to IT Security Incidents,
6 Freiling, Felix C.; Bastian Schwittay; A Common Process Model for Incident Response and Computer Forensics, 2007,
7 Carnegie Mellon University, McGraw, Shane; Blash, Deen; CMMI on the Web:  Remastered, 2009,
8 iSixSigma, DMAIC Versus DMADV, 2010,

Steve Markey is the principal of nControl, a consulting firm based in Philadelphia, Pennsylvania, USA. He is also an adjunct professor and the current president of the Delaware Valley (Greater Philadelphia) chapter of the Cloud Security Alliance (CSA). Markey holds multiple certifications and degrees, and has more than 11 years of experience in the technology sector. He frequently presents on information security, information privacy, cloud computing, project management, e-discovery and information governance.

Enjoying this article? To read the most current ISACA Journal articles, become a member or subscribe to the Journal.

The ISACA Journal is published by ISACA. Membership in the association, a voluntary organization serving IT governance professionals, entitles one to receive an annual subscription to the ISACA Journal.

Opinions expressed in the ISACA Journal represent the views of the authors and advertisers. They may differ from policies and official statements of ISACA and/or the IT Governance Institute and their committees, and from opinions endorsed by authors’ employers, or the editors of this Journal. ISACA Journal does not attest to the originality of authors’ content.

© 2012 ISACA. All rights reserved.

Instructors are permitted to photocopy isolated articles for noncommercial classroom use without fee. For other copying, reprint or republication, permission must be obtained in writing from the association. Where necessary, permission is granted by the copyright owners for those registered with the Copyright Clearance Center (CCC), 27 Congress St., Salem, MA 01970, to photocopy articles owned by ISACA, for a flat fee of US $2.50 per article plus 25¢ per page. Send payment to the CCC stating the ISSN (1526-7407), date, volume, and first and last page number of each article. Copying for other than personal use or internal reference, or of articles or columns not owned by the association without express permission of the association or the copyright owner is expressly prohibited.