JOnline: What’s in a Word? Measuring the Language of Information Security 

 
Download Article

Cybersecurity professionals are critically dependent on language—regardless of their role or specialization. The code that runs an organization’s software is language. So too are the laws, regulations and policies that define what information security means and how it is accomplished. Whether a chief information security officer (CISO), an auditor or a hacker, the characteristics and use of language are central to one’s daily responsibilities.

Considering the importance of language to this discipline, security professionals often miss opportunities to assess and improve their linguistic behaviors. Industry experts emphasize the requirement to assess and measure security program performance and effectiveness, but language is not usually viewed as a core security component. Analysis of language is typically reserved for pattern matching and anomaly detection, rather than higher-order communication and organizational behavior. Some security professionals may express skepticism that such measures and analyses are useful or even possible—unaware of successful application of such techniques in other fields and industries.

Language challenges are often what makes security challenging. Consider the following examples:

  • Definitions—Ask a security organization’s members the question, “What is the biggest risk?” and there will probably be many responses. Very often the organization fails to properly define security terms like “risk” or “threat” before making decisions about how to address or manage them.
  • Code—Computer security exists because huge collections of words (in the form of software) are vulnerable to mistakes in grammar and form. Imagine trying to find a dangling participle in one sentence within War and Peace, and then imagine that failing to discover it could allow an adversary to seriously damage or steal parts of the book.
  • Laws and policy—Laws are similar to the software of society.1 Laws and regulations are the code driving legal systems, including control over security and privacy in technology, which in turn influences how organizations implement security policies and programs.
  • Metaphors and stories—From firewalls to viruses to spear phishing, information security professionals depend on metaphors to describe their activities and to create narratives about protecting digital assets.2 Of particular concern today is the need to make these stories matter to professionals outside of security, such as business executives, who may use a different business language.
  • Culture and geography—If internal language challenges were not enough, cybersecurity professionals must also consider the differences and nuances of language among the various cultures and governments that comprise the international digital society.

Measuring Language

As linguistic researchers, public policy analysts, advertisers and a host of other language-intensive practitioners can attest, measuring the use and effectiveness of words and discourse is both possible and necessary for success. Cybersecurity is no different; security professionals owe it to themselves and to everyone who depends on the protection of information. This means not only recognizing the important role that vocabulary, conversation and discourse play in the daily accomplishment of security goals, but also making a commitment to measure and improve the understanding of language within the field.

Consider the following three examples of areas where the empirical measurement of language can directly support and improve an information security program:

  1. Security policy analysis—Most experts agree on the importance of having a security policy in place; adequate policies are typically seen as a necessary prerequisite for any security program to succeed. In practice, however, security policies are often treated as boilerplate artifacts that are infrequently read once in place, much less analyzed for the effectiveness of language.
  2. Executive messaging and outreach—The ability to persuasively market security program efforts to senior management is increasingly seen as a requirement for CISO success. As with any product or service, marketing drives adoption and growth. CISOs who struggle with or neglect empirical measures of how well they are communicating with their ultimate customers increase their likelihood of failure.
  3. Audit and compliance—Many organizations experience what can only be described as translation problems between IT security staff and auditors. Regulations and frameworks must be read, interpreted and applied within an organizational context, and then the results articulated back to an auditor who may or may not speak or understand the “language.” Measuring success in these efforts only by passing or failing an audit can be a frustrating and confusing process.

Measuring Success

Fortunately for the field, there is no need to invent (or reinvent) anything to apply measurement and analysis techniques to the language of cybersecurity. Security professionals must simply open their minds to some different approaches and be willing to learn a few new skills and tools.

In some cases, security teams are already heavily engaged in linguistic analysis. They perform code reviews on software and implement secure development guidelines to help authors of programs get better at creating and crafting their language. In the process of creating language patterns, information security specialists create signatures for antivirus, intrusion prevention and data loss prevention systems. The industry’s language skills and proficiency are greatest in the areas where the methods are algorithmic and quantitative.3 However, the farther away analysis gets from things that can be easily automated and counted, the more the idea of measuring language can seem strange and questionable to an information security professional who is more comfortable with practical issues of technology and automation.

One way to begin promoting language measurement in security is by staying close to the industry’s traditional comfort zones and then expanding the analytical horizons as people get more comfortable with incorporating qualitative methods and approaches. The term qualitative has a precise meaning in research, different from that of the security industry, in which “qualitative” tends to mean data that are, for a variety of reasons, subjective and unreliable. In fact, the term qualitative implies in research only that data may be observed empirically, but are not “numerical in nature.”4

A comprehensive review of the many disciplines and methods for analyzing language is beyond the scope of this article. What is possible is to provide a more practical guide to analyzing and measuring language of use to the security industry. This requires a bit of simplification (with apologies in advance to any specialists who may be offended by perceived oversimplification). The proposed framework for measuring language in security contains three types of analysis that begin with the tools and techniques familiar to the profession (figure 1). (Resources for further study are included in the references.)

Figure 1

Expecting security professionals to begin conducting sophisticated frame and discourse analysis to improve the effectiveness of cybersecurity language is unrealistic. Being frustrated by a technology vendor’s advertising claims, struggling with understanding the nuances of a security or privacy regulation, or competing for resources with someone who has inferior ideas but more power and influence to make his/her case are all examples of how language is a strategic differentiator in how well one can do his/her job.

Understanding the nature of qualitative research methods can be particularly valuable to security practitioners, even if they never end up using tools such as TAMS Analyzer or ATLAS.ti. These tools require a researcher to code in a different way—assigning themes and metadata to words, concepts and language structures—and then to analyze those codes for patterns. The tools enable researchers to build novel interpretive structures around information in order to discover new insights into what the data may mean. In many cases, these codes are then subjected to quantitative analysis.5

Information security can benefit from exploring these analyses and methods as ways to understand how the field uses language internally and in relation to other disciplines and industries. The challenge is not simply academic, and measuring language is not just a theoretical exercise. The role of linguistic analysis is growing in technology, from sentiment analysis in social networking6 to the ways that natural language processing can generate unforeseen and undesirable results.

Case Study: Measuring Security Policies

In this case study, the measurement and analysis of language was used to recommend improvements to the security program of an organization. Many organizations communicate on the importance of security policies but, in practice, build inadequate policy architectures on which to base their information protection activities. Understanding the limits and flaws in these policies can help a security organization directly improve its efforts.

This case involves a consultive review for a company experiencing regular violations of its security policies. The organization deployed a comprehensive security policy system with several dozen distinct policy documents that were all posted to a centrally accessible repository. Users were required to read and acknowledge the policies regularly. Yet violations, sometimes serious, were ongoing. The manager in charge of the policies complained during an interview with the review team, “You know, it is like people do not even read these things.”

One of the analyses performed during the engagement measured the readability of the language used in the policy documents. Literacy and readability standards make up an entire field of study, with impacts on everything from instructions for taking prescription drugs to the manuals for operating complex machinery. While studies vary, the average reading ability among adults in the US is typically cited at between 7th and 9th grade levels, for example.7 Thus, the average reader may not fully comprehend a policy document written in a more complex dialect.

Figure 2 shows the results of a Flesch test8 conducted on security policy documents indicating that they were written in a language that was very difficult to comprehend. Many of the policies were written in a manner similar to legal contracts—containing large, compound words and complex grammatical structures. By measuring language readability, the documents were shown to be ineffective for many users, not because employees were ignoring the policies, but because they often could not understand what they had read and, thus, could not take appropriate action in their security-related activities. The analysis included recommendations for simplifying the policy language in order to make the security policy documents more reader-friendly for the organization’s employees.

Figure 2

Exploring the use and effectiveness of language in cybersecurity is a challenge worthy of a growing and evolving profession, particularly when the words and texts are so central to the mission of protecting information and digital assets. By building on existing skills and techniques for analyzing and measuring language and embracing new techniques for content, frame and discourse analysis, information security professionals can improve the maturity and effectiveness of programs and activities.

Conclusion

Language is a critical and undervalued component of cybersecurity. While the security field analyzes language and linguistic behaviors in some cases, there is significant room for growth and improvement. By exploring qualitative techniques used in other fields, such as content, frame and discourse analysis, security professionals can greatly improve the efficiency and effectiveness of security programs.

References

Endnotes

1 Koester, Eric; “Four Things Lawyers and Hackers Have in Common,” Xconomy, 4 October 2011, www.xconomy.com/seattle/2011/10/04/four-things-lawyers-and-hackers-have-in-common/
2 Metaphors further one’s understanding by referring to something as something else—often mapping one conceptual domain to another domain. See: Lakoff, G.; M. Johnson; Metaphors We Live By, University of Chicago Press, USA, 1980.
3 For an excellent overview of content analysis in security, see: Nisbet, Jim; “The Security Role for Content Analysis,” presentation at BayLISA (Bay Area Large Installation System Administrators), 17 November 2004, http://static.usenix.org/events/lisa04/tech/talks/nisbet.pdf.
4 Trochim, William M. K.; “Qualitative Data,” Web Center for Social Research Methods, 20 October 2006, www.socialresearchmethods.net/kb/qualdata.php
5 A very accessible explanation of qualitative analysis, including coding qualitative data in a business context, is available at http://answers.mheducation.com/marketing/marketing-research/analyzing-and-reporting-qualitative-market-research.
6 See https://developers.google.com/prediction/docs/sentiment_analysis for more information on sentiment analysis.
7 See the National Assessment of Adult Literacy provided by the National Center for Educational Statistics at http://nces.ed.gov/NAAL/ for more information on reading levels.
8 The Flesch formulas for readability are among the most widely used tests for measuring the complexity of language in a document.

Lance Hayden, Ph.D., CISM, CRISC, CISSP, is responsible for Cisco Systems’ IT governance, risk and compliance (GRC) consulting services. Hayden is the author of IT Security Metrics from McGraw-Hill and consults on security strategy, measurement and improvement for Cisco’s global customers. He also teaches at the University of Texas at Austin School of Information (USA).


Enjoying this article? To read the most current ISACA Journal articles, become a member or subscribe to the Journal.

The ISACA Journal is published by ISACA. Membership in the association, a voluntary organization serving IT governance professionals, entitles one to receive an annual subscription to the ISACA Journal.

Opinions expressed in the ISACA Journal represent the views of the authors and advertisers. They may differ from policies and official statements of ISACA and/or the IT Governance Institute and their committees, and from opinions endorsed by authors’ employers, or the editors of this Journal. ISACA Journal does not attest to the originality of authors’ content.

© 2013 ISACA. All rights reserved.

Instructors are permitted to photocopy isolated articles for noncommercial classroom use without fee. For other copying, reprint or republication, permission must be obtained in writing from the association. Where necessary, permission is granted by the copyright owners for those registered with the Copyright Clearance Center (CCC), 27 Congress St., Salem, MA 01970, to photocopy articles owned by ISACA, for a flat fee of US $2.50 per article plus 25¢ per page. Send payment to the CCC stating the ISSN (1526-7407), date, volume, and first and last page number of each article. Copying for other than personal use or internal reference, or of articles or columns not owned by the association without express permission of the association or the copyright owner is expressly prohibited.