Measure and Monitor Application Security 

Download Article Article in Digital Form

In the increasingly digitally connected world, information is the most valuable asset. Web applications are the gateway to access information, and they are no longer confined to a simple browser interface invoked from a laptop or desktop. With smartphones and smart TVs (televisions with a Wi-Fi or network connections) and appliances, applications are available everywhere.

This thrusts a lot of responsibilities on the security community to safeguard the interests of stakeholders and the information that is exchanged via web applications. The security community1 has published best practices, guidelines and checklists to embed security into web applications. For example, one of the best practices is to call out specifically how to handle the SQL injection or to handle the cross-site scripting in the design document itself so that when the developers implement the design, the security is inherent in the code. Securing the system development life cycle (SDLC) is no longer a separate activity.

How does one ensure the effectiveness of application security? How are the security initiatives that minimize the risks and threats from hackers measured? This article attempts to define metrics that measure the effectiveness of application security in an organization.

Define the Metrics

Metrics are the prime indicators of management initiatives in any organization. Organizations witness a slow, but steady, increase in need for information security within. In many organizations, information security has attained a fairly considerable level of maturity. Web application security, as a part of the overall information security program, plays a major role in protecting valuable information. Therefore, the best time to define a metric is at the start of the application security program. The best possible approach is to:

  • Identify the metrics.
  • Identify the data-collection techniques.
  • Obtain agreement from key stakeholders.
  • Report the metrics to key stakeholders at the agreed-upon time interval.

The identified metrics should be useful to measure the effectiveness of the security program as well as to identify the gaps for future improvement.

There are two broad categories of metrics that can be captured for application security. The first set of metrics is for incidents and vulnerabilities (figure 1), and the second set is for the application security program itself (figure 2).

Figure 1

Figure 2

Collect the Data

The data for the metrics should be collected on an ongoing basis (e.g., weekly, per event). The data collection template needs to be defined and kept in a centralized repository. As soon as the reviews are done, the review metrics can be updated in the data-collection template.

The incident’s data can be collected from the incident database. The vulnerabilities and effort can be collected from the project team. The reviews and comments should be updated in a shared repository, and data are to be reported from the repository.

Report the Metrics

At the end of the month, or as per the agreed-upon cutoff date with the project team, the data can be collated and presented. A few presentation examples are illustrated in figures 3 and 4

Figure 3

Figure 4

Case Study

The following section offers a brief case study related to application security.

Problem Statement
The problem statement for the case example is: Manage the application security for the e-commerce applications of a pharmaceutical company by measuring the key metrics associated with application security.

The key metrics agreed upon for the measurement include:

  • The number of vulnerabilities reported
  • The number of vulnerabilities resolved
  • Total security effort
  • The number of security awareness sessions
  • The design review ratio (number of design reviews/total number of designs delivered)
  • The code review ratio (number of code reviews/total number of modules)

After the key metrics were agreed upon by the senior management team, the kickoff meeting was scheduled with the project leadership team. The metrics were explained to the team, and a monthly measurement window was chosen. The data collection techniques were explained, and the schedule was given to the project leads.

Figure 5At the end of the first measurement cycle, the data were collected and collated as shown in figure 5.

Business Benefit
The metrics were presented in the project-review meeting, and the project manager approved the data. Figure 5 indicates that the secure code review process needs to be institutionalized; this would enable the project team to take corrective action and reduce further vulnerabilities. However, metrics should be collected for subsequent months to understand long-term trends.


With more stringent security controls in place for the infrastructure and networks of organizations, hackers are turning their attention to web applications. Through the vulnerabilities of web applications, the network and infrastructure can be compromised. Along with the increased adoption of cloud computing, there is more attention given to application security. The metrics discussed in this article should help organizations measure their application security postures. In this age of information, an organization needs to safeguard its information to have a competitive edge and to win the trust of key stakeholders.


1 By “security community,” the author is referring to groups such as the Open Web Application Security Project (OWASP), the Cloud Security Alliance and social media outlets.

Sivarama Subramanian, CISM
is senior architect of technology at Cognizant Technology Solutions, where he is currently overseeing the security initiatives for retail and retail e-commerce engagements. Subramanian is a member of the ISACA Chennai, India, Chapter and can be reached at

Enjoying this article? To read the most current ISACA Journal articles, become a member or subscribe to the Journal.

The ISACA Journal is published by ISACA. Membership in the association, a voluntary organization serving IT governance professionals, entitles one to receive an annual subscription to the ISACA Journal.

Opinions expressed in the ISACA Journal represent the views of the authors and advertisers. They may differ from policies and official statements of ISACA and/or the IT Governance Institute® and their committees, and from opinions endorsed by authors’ employers, or the editors of this Journal. ISACA Journal does not attest to the originality of authors’ content.

© 2011 ISACA. All rights reserved.

Instructors are permitted to photocopy isolated articles for noncommercial classroom use without fee. For other copying, reprint or republication, permission must be obtained in writing from the association. Where necessary, permission is granted by the copyright owners for those registered with the Copyright Clearance Center (CCC), 27 Congress St., Salem, MA 01970, to photocopy articles owned by ISACA, for a flat fee of US $2.50 per article plus 25¢ per page. Send payment to the CCC stating the ISSN (1526-7407), date, volume, and first and last page number of each article. Copying for other than personal use or internal reference, or of articles or columns not owned by the association without express permission of the association or the copyright owner is expressly prohibited.