Measurement can be a tricky thing. Many practitioners want more ways to measure the effectiveness of cybersecurity programs to determine whether they are doing the right things to protect their organizations. Executives want measurement so that they can help provide oversight in terms of whether their organization is doing enough to shield stakeholders from harm. A colloquial saying I have heard in the cybersecurity sector is that it is not like other industries, such as insurance, where there exist reams of data to make better decisions. Some who have been conducting pioneering work in cyberrisk quantification (CRQ) have utilized statistical methods to overcome this barrier. Somewhat contradictory is that the use of statistical methods is necessitated by limited data.
Many professionals in the actuarial, economic and financial disciplines are conversant in techniques that allow one to make assumptions and forecasts about costs, activities and policy changes. So, too, have those that have applied such techniques to cybersecurity begun to see benefits from learning how to apply quantitative values to subject matter opinion, or expert elicitation, to enable such forecasts. This has given rise to a professional practice of CRQ in many organizations to answer business questions about how much risk an organization has and what the best opportunity is to improve that risk posture.
The downside of this marked improvement and maturation in cyberrisk measurement has been the application of strict criteria to determine whether one is appropriately applying CRQ. Typically, this has meant the denigration of those doing yeoman’s work measuring control compliance and maturity and an almost complete ignoring of the work of those building and measuring better security cultures. Any measure that involves the use of an ordinal scale—a measure that allows for rank order but does not allow for a degree of difference between them (common examples of this are maturity scales of 1 to 5)—is an often-unwitting target of those working on quantifying cyberrisk.
However, control compliance and maturity measures are valuable as long as they are presented as answers to the right questions. While an ordinal measure of maturity is not a valid measure of cyber loss exposure, it does say something about risk. So, too, does a penetration test convey information about one’s propensity to experience a cyberincident whereby a certain amount of cyber loss exposure will be possible. The sum of these various security assessments from control compliance to cultural maturity to red team testing to CRQ provides the corpus of information from which those with a fiduciary obligation to an organization’s well-being need to consider when discharging their duties.
The credit rating business has long taken complex financial statistics and boiled them down to an ordinal rating (e.g., AAA or AA1) to facilitate decision-making among stakeholders. Furthermore, there has been a move to provide the environmental, social and governance (ESG) ratings of organizations to help facilitate those that are interested in socially conscious investing. ESG ratings take especially qualitative measures and transform them into a graded, ordinal scale to facilitate decision-making. The time is ripe for an equivalent cyberrisk rating that considers a myriad of inputs, both qualitative and quantitative, to give organizations and the public a better sense of the cyberposture of organizations. Such a scale should consider the impact of threat activity, control posture, governance of the security program and culture, and an assessment of an organization's cyberrisk exposure using CRQ methods. But there is no need to eschew the decades of good work in building assessments based on controls and maturity. Rather, now is the time to blend these assessments with CRQ and mature them into a measure that can facilitate decision-making among stakeholders.
Jack Freund, Ph.D., CISA, CISM, CRISC, CGEIT, CDPSE, is head of cyberrisk methodology for VisibleRisk, coauthor of Measuring and Managing Information Risk, 2016 inductee into the Cybersecurity Canon, ISSA Distinguished Fellow, FAIR Institute Fellow, IAPP Fellow of Information Privacy, and ISACA’s 2018 John W. Lainhart IV Common Body of Knowledge Award recipient.