HelpSource Q&A 

Download Article Article in Digital Form

We invite you to send your information systems audit, control and security questions to:

HelpSource Q&A
ISACA Journal
3701 Algonquin Road, Suite 1010
Rolling Meadows, IL 60008 USA

Q My organisation got certified for BS 7799 and then later got the certificate upgraded to ISO 27001:2005. One area in which we feel the organisation is weak relates to security compliance. We believe and have been told by our auditors that we do not have appropriate measurement systems in place to measure the effectiveness of our levels of security compliance.

What is your advice? Should we implement any tools? How do we achieve our objectives with minimal costs? Ours is a small to medium-sized organisation and we cannot afford a huge expenditure to make this happen.

A A famous quote of management’s:  ‘What you cannot measure, you cannot manage’. I strongly agree with the quote. If we need to manage something effectively, we should be able to lay our hands on metrics relating to the same.

Having dealt with topics on security metrics in the past, let us discuss more in terms of a generic framework and the need for a structured methodology to determine the metrics. Not all metrics or measures are essential or useful to all organisations.

My formula on effectiveness metrics is very simple and straightforward. We should aim for the measurement of the following metrics:

  • Exceptions to policies and standards
  • Incidents
  • Status of compliance with key control parameters
  • Tracker on issues identified and reported in audits

Policies and standards dictate the baseline controls expected to be in place across the organisation. Unless specifically stated in polices and standards, the tenets of the policy will, or rather must, apply to everyone in the organisation. At the same time, it is understood that there will be compelling business needs where exceptions to the policies and standards have to be granted.

However, it is essential to have is a foolproof exceptions-approval process in place. The requests for all such exceptions must be tracked using a repository of any form; the risks that get opened due to the granting of exceptions must be clearly documented in the same repository, against each of the requests. The compensating controls to mitigate any potential risks and outstanding residual risks must also be documented in the repository. Depending on the quantum of residual risks and exposure, there should be a multi-staged approval process to give a green signal in terms of exceptions.

Exceptions granted must be for set or predefined periods only, after which they must expire automatically. The repository system must have inbuilt features, ideally, to notify the users in advance that their exceptions requests will be expiring soon and asking them to reapply for the same.

If such a system were to exist and function, the information on exceptions in the repository would be of value to measure the effectiveness of security and it would form part of the framework.

Tracking of incidents is the next most important part of the framework. An incident-free regime is next to impossible. No number of security controls can ever lead to or guarantee zero incidents. If people tell you that they have nil security incidents in their organisations, it means that they are kidding you or themselves. Something may be fundamentally wrong in their definition of incidents and/or their incident management policies.

It is essential to develop and roll out a process to report and track incidents. The size of the potential or actual impact caused should determine the various escalation levels of reporting. Needless to say, all incidents reported must be tracked centrally, through a repository system.

It is essential to determine and understand whether any patterns or trends emerge out of the reported incidents. Trend analysis of incidents renders better information than incidents looked at in isolation. I am not ruling out the need to do a post-incident review in terms of lessons learnt individually, but my point is that taking a holistic view can render better information whilst we undertake any trend analysis.

The third component of this framework relates to the compliance status of the various control parameters chosen in terms of security. Whilst a number of controls may get implemented and rolled out as enunciated by the various policies and standards, it is important to classify the controls and segregate the most important ones amongst the various others. Not all controls may be measurable and not all of them may be subjected to measurement. Once the key controls are identified, it may be easier to seek the status of compliance for those key controls. Let us take the simple example of desktops or workstations. Whilst there can be a multitude of parameters against which standards can be set, it may work out to be a costly proposition in terms of time and efforts to measure the status of all parameters. So, key parameters such as domain membership (assuming the organisation is operating in a typical Windows environment) and antivirus precautions may alone be measured. Given that various controls are pushed through the central domain policy, a domain membership can, in general, mean that a lot of controls are in place, eliminating the need for doing any individual checks.

The last component relates to the tracking of issues identified during the course of various internal and external audits. A culture in which people fear audits and feel that they may get penalised for adverse findings reported in audit reports is not going to help the organisation in the long run. Appearance and reporting of issues in any audits should never be construed as something linked to an individual’s performance in that area. Audits are there to identify and unearth issues given their independent perspective, and it is essential to track them to closure. People can be penalised for not closing the issues effectively or if the same issues recur again and again in audit reports.

Given the size and complexity of your small to medium-sized organisation, I do not envisage any other special requirements. Of course, you know your organisation better than anyone else and can best determine the right requirements.

Enjoying this article? To read the most current ISACA® Journal articles, become a member or subscribe to the Journal.

The ISACA Journal is published by ISACA. Membership in the association, a voluntary organization serving IT governance professionals, entitles one to receive an annual subscription to the ISACA Journal.

Opinions expressed in the ISACA Journal represent the views of the authors and advertisers. They may differ from policies and official statements of ISACA and/or the IT Governance Institute® and their committees, and from opinions endorsed by authors’ employers, or the editors of this Journal. ISACA Journal does not attest to the originality of authors’ content.

© 2010 ISACA. All rights reserved.

Instructors are permitted to photocopy isolated articles for noncommercial classroom use without fee. For other copying, reprint or republication, permission must be obtained in writing from the association. Where necessary, permission is granted by the copyright owners for those registered with the Copyright Clearance Center (CCC), 27 Congress St., Salem, MA 01970, to photocopy articles owned by ISACA, for a flat fee of US $2.50 per article plus 25¢ per page. Send payment to the CCC stating the ISSN (1526-7407), date, volume, and first and last page number of each article. Copying for other than personal use or internal reference, or of articles or columns not owned by the association without express permission of the association or the copyright owner is expressly prohibited.