IS Auditing Procedure: IS Risk Assessment Measurement
Document P1

Download this document in PDF format (237K)

1 BACKGROUND

1.1 Linkage to Standards/Guidelines

1.1.1 Standard S5 Planning states, “The IS auditor should plan the information systems audit coverage to address the audit objectives and to comply with applicable laws and professional auditing standards.”
1.1.2 Standard S6 Performance of Audit Work states, “During the course of the audit, the IS auditor should obtain sufficient, reliable and relevant evidence to achieve the audit objectives. The audit findings and conclusions are to be supported by the appropriate analysis and interpretation of this evidence.”
1.1.3 Guideline G13 Use of Risk Assessment in Audit Planning provides guidance.

1.2 Need for Procedure

1.2.1 This procedure is designed to provide:

• A definition of IS audit risk assessment
• Guidance on the use of a IS audit risk assessment methodology for use by internal audit functions
• Guidance on the selection of risk ranking criteria and the use of weightings

2. IS RISK

2.1 Risk is the possibility of an act or event occurring that would have an adverse effect on the organisation and its information systems. Risk can also be the potential that a given threat will exploit vulnerabilities of an asset or group of assets to cause loss of, or damage to, the assets. It is ordinarily measured by a combination of effect and likelihood of occurrence.
2.2 Inherent risk refers to the risk associated with an event in the absence of specific controls.
2.3 Residual risk refers to the risk associated with an event when the controls in place to reduce the effect or likelihood of that event are taken into account.

3. IS RISK ASSESSMENT MEASUREMENT

3.1 Risk assessment measurement is a process used to identify and evaluate risks and their potential effect.

4. IS AUDIT RISK ASSESSMENT MEASUREMENT METHODOLOGY

4.1. IS audit risk assessment measurement is a methodology to produce a risk model to optimise the assignment of IS audit resources through a comprehensive understanding of the organisation’s IS environment and the risks associated with each auditable unit. See Section 9 for details of auditable units.
4.2. The objective of a risk model is to optimise the assignment of IS audit resources through a comprehensive understanding of the IS audit universe and risks associated with each universe item.

5. RISK-BASED IS AUDIT APPROACH

5.1. More and more organisations are moving to a risk-based audit approach that can be adapted to develop and improve the continuous audit process. This approach is used to assess risk and to assist an IS auditor’s decision to do either compliance testing or substantive testing. In a risk based audit approach, IS auditors are not just relying on risk. They are also relying on internal and operational controls as well as knowledge of the organisation. This type of risk assessment decision can help relate the cost/benefit analysis of the control to the known risk, allowing practical choices.
5.2. By understanding the nature of the business, IS auditors can identify and categorise the types of risks that will better determine the risk model or approach used in conducting the review. The risk assessment model can be as simple as creating weights for the types of risks associated with the business and identifying the risk in an equation. On the other hand, risk assessment can be a scheme where risks have been given elaborate weights based on the nature of the business or the significance of the risk.
5.3 The IS auditor is interested in uncontrolled risks and in critical controls. Thus in a risk-based audit approach the IS auditor will be interested in technology-based systems which provide controls for business functions where there is a high inherent risk and in technology-based functions where there is a higher than acceptable residual risk.
5.4 Defining the IS audit universe is the first prerequisite to risk ranking. The determination of the audit universe will be based on knowledge of the organisation’s IT strategic plan and organisation operations, a review of organisation charts and function and responsibility statements of all organisation affiliates, and discussions with responsible management personnel.
5.5 Audit planning cycles are ordinarily aligned with business planning cycles. Often, an annual audit planning cycle is selected—either a calendar year or another twelve-month period. Some organisations have planning cycles other than for twelve month periods such as six or eighteen months. Rather than have a fixed planning cycle, some organisations have rolling planning cycles that keep rolling forward a set period. For consistency, this procedure will assume an annual audit planning cycle.
5.6 Selection of audit projects to be included in the IS audit plan is one of the most important problems confronting IS audit management. The audit planning process presents the opportunity to quantify and justify the amount of IS audit resources needed to complete the annual IS audit plan. Failure to select appropriate projects results in unexploited opportunities to enhance control and operational efficiency.
5.7 The assumption underlying the IS audit plan is that an evaluation of prospective audit reviews/projects will be more effective if a formal process is followed for gathering the information necessary to make review/project selection decisions. The approaches described herein are basically a framework in which to apply common sense and professional judgment. 
5.8 The methodology presented is relatively simple. However, in a great majority of cases, it should suffice to reach reasonable, prudent and defensible IS audit review/project selection decisions. A framework to use in performing a risk exposure analysis and establishing an audit review/project priority schedule is detailed in this procedure.
5.9 As used here, risk assessment is a technique used to examine auditable units and choose reviews/projects that have the greatest risk exposure. A risk assessment approach to audit review/project selection is important in that it affords a means of providing reasonable assurance that IS audit resources are deployed in an optimal manner, i.e., the IS audit plan allocates IS audit resources in a manner likely to achieve maximum benefits. To this end, the risk assessment approach provides explicit criteria for systematically selecting audit projects. The IS audit plan is often attached with the financial and operational audit plan to detail the complete planned IS audit coverage.

6. IS RISK ASSESSMENT MEASUREMENT TECHNIQUES

6.1 When determining which functional areas should be audited, the IS auditor could face a large variety of audit subjects. If possible all IS areas of the organisation should be included in the risk assessment exercise. Some organisations only rate IS projects. Others rate every IS auditable area/system. Each of these may represent different types of audit risks. The IS auditor should evaluate these various risk candidates to determine which are the high-risk areas and therefore should be audited. The purpose of this process is to:

• Identify areas where the residual risk is unacceptably high
• Identify critical control systems that address high inherent risks
• Assess the uncertainty that exists in relation to the critical control systems

6.2 Using risk assessment to determine IS areas to be audited:

• Enables management to effectively allocate limited IS audit resources
• Provides reasonable assurance that relevant information has been obtained from all levels of management, including the board of directors and functional area management. Generally, the information includes areas that will assist management in effectively discharging their responsibilities and provides reasonable assurance that the IS audit activities are directed to high business risk areas and will add value to management.
• Establishes a basis for effectively managing the IS audit function
• Provides a summary of how the individual review subject is related to the overall organisation as well as to the business plans

7. IS RISK ASSESSMENT MEASUREMENT METHODS

7.1 Several methods are currently employed to perform IS risk assessments. One such risk assessment approach is a scoring system that is useful in prioritising IS audits based on an evaluation of risk factors that consider variables such as technical complexity, extent of system and process change and materiality. These variables may or may not be weighted. These risk values are then compared to each other and ordinarily an annual IS audit plan is prepared. Often the IS audit plan is approved by the audit committee and or the chief executive officer. Reviews are then scheduled according to the IS audit plan. Another form of IS risk assessment is judgmental. This entails making an independent decision based upon executive management directives, historical perspectives and business climate.

8. COLLECTION OF DATA

8.1 Information describing all aspects of the organisation’s operation will be used to define the various auditable units and to model the IS risks inherent in the unit’s operations. Sources of this data include:

• Interviews conducted with senior management for the purpose of gathering data for the development of the IS risk model
• Returns of structured questionnaires sent to management to facilitate the gathering of IS risk model data
• Recent review reports
• The IT strategic plan
• The budgetary process may be a useful source of information
• Issues raised by the external auditors
• IS audit knowledge and awareness of significant issues gathered from any other sources
• The specific methods used to collect the data, whether they will be sufficient considering the time and resources available for the task

9. IS AUDITABLE UNITS

9.1 The model is meant to include and provide a risk rating for every IS auditable unit in the organisation (the IS audit universe). An auditable unit can be defined as the discrete segments of every organisation and its systems. There are no specific rules for determine or differentiate an individual auditable unit. However, the following are guidelines for use in this audit risk model for each unit/topic/function:

• Auditable in a reasonable timeframe
• A system, i.e., have recognisable inputs, processes, outputs, outcome
• Separable, i.e., able to be audited with minimal reference to other systems (This may be difficult if an application system under review has many interfaced systems.)

10. EXAMPLES

10.1 There are many different methods of performing IS risk assessment measurements. Sections 11 through 14 contain several types of IS risk assessments.

11. EXAMPLE I

11.1 Example I shows an IS risk assessment measurement evaluation with eight key variables. Each unit/area in the IS audit universe will be rated on these eight key variables using a numeric descriptive value ranking of 1 (low) to 5 (high). The results of these ranking judgments are then multiplied by significance weighting factors that range from 1 (low) to 10 (high) to give an extended value. Arbitrary examples of significance weighting factors are included in example I. These extended values are added together to give a total. Once the totals for each auditable unit/area have been obtained, the auditable units/areas are ranked by risk. The framework of the annual IS audit plan is then built from these rankings. The eight key variables are listed in sections 11.1.1 to 11.1.3 with a brief explanation of each.

11.1.1 Measures of Effect

• Character of activity—The criticality of the activity and the part of the organisation that utilises the activity. Infrequent or unusual activities or projects are more likely to result in error or inefficiency and are of greater audit interest.
• Fall back arrangements—This factor relates to the measures that have been put in place to continue operations if the new system has problems. Factors to consider include business continuity plans, disaster recovery plans, manual procedures, and the old system.

Generally speaking, if the above issues have been addressed, are achievable or are cost beneficial, then the risk is lowest.

• Sensitivity of the function to executive management—This factor relates to how important the unit, function or area is viewed by executive management.
• Materiality—A concept regarding the importance of an item of information with regard to the effect on the functioning of the organisation. An expression of the relative significance or importance of a particular matter in the context of the organisation as a whole. 

11.1.2 Measures of Likelihood

• Extent of system or process change—A dynamic environment in terms of system or process change increases the probability of errors and consequently increases audit interest. A considerable amount of process re-engineering may have taken place. System or process change ordinarily occurs to effect improvement in the long term but often has short-term offsets that require increased audit coverage.
• Complexity—This risk factor reflects the potential for errors or misappropriation to go undetected because of a complicated environment. The rating for complexity will depend on many factors. Extent of automation, complex calculations, interrelated and interdependent activities, number of products or services, the time spans of estimates, dependency on third parties, customer demands, processing times, applicable laws and regulations and many other factors, some not recognised, affect judgments about the complexity of a particular audit.
• Project management—Consideration should be given to the following when ranking project management:
  - In-house or outside developers
  - Project structure
  - Personnel skills
  - Project timeframes

Generally speaking, the risk is shared if the project is outsourced.

11.1.3 Measures of Uncertainty about the Controls

• Period since last review—As the time since the last review coverage lengthens, the value of a new review is likely to increase. The beneficial effects of a review are greatest immediately before or after system implementation.
  

EXAMPLE I—IS RISK ASSESSMENT MEASUREMENT EVALUATION

* Uses arbitrary Significance Weighting Example

12. EXAMPLE II

12.1 Example II extends the IS risk assessment measurement evaluation used in example I by incorporating business risks as well as the eight IS audit key variables used in example I. The IS audit risk ranking factor (from example I) is multiplied by business risk in this example. The business risk factors (financial, strategic, operational, and legal compliance) are considered regarding their relevance to each auditable unit/area.
12.2 Each unit/area in the IS audit universe will be rated on these eight key variables using a numeric rating of 1 (low) to 5 (high). The results of these rating judgments are then multiplied by a significance weighting factor, which ranges from 1 (low) to 10 (high) as in example I. These extended values are added together to give a total (using the arbitrary significance weightings used in example I). This total is the IS audit risk ranking factor.
12.3 The four business risk factors are defined below:

• Financial risk—As most systems potentially have some effect on the organisation’s financial performance, the level and likelihood of such an effect needs to be considered. If the anticipated effect is indirect and relatively minor in comparison with other effects and purposes of the system and/or in comparison with other auditable areas/systems then we would probably score 0 rather then 1 for the financial risk factor.
• Strategic risk—Systems may have direct strategic effect on the organisation. Some that would be expected to score 1 on the risk factor are those identified by executive management.
• Operational risk—Operational risk will probably be rated 1 more commonly than any of the other business risk factors since most systems are designed to affect the manner in which, and the effectiveness with which, the organisation conducts its day-to-day business.
• Legal compliance—Systems can have a direct effect on how the organisation complies with statutory obligations.

12.4 Insert a score of 1 (relevant) or 0 (not relevant) for each business risk factor. Then multiply each score by the respective weighting and add, to give the total business risk ranking factor for each audit topic.

12.5 In assigning scores consider the following three issues:

• What are the anticipated purpose and objectives of the system being audited?
• What are the anticipated scope and objectives of the audit?
• Does the system directly effect the organisation’s financial/strategic/operational/compliance performance? For example, if the system does not operate as intended, is it probable that the organisation will suffer financial loss, experience strategic disadvantage, have operational problems or contravene relevant legal requirements?

12.6 The final step in this example is to multiply the audit risk ranking factor by the business risk ranking factor, to give the total risk ranking. See the example in the table below. Once the total risk rankings for each auditable unit/area have been obtained the auditable units/areas are ranked by risk. The framework of the annual IS audit plan is then built from these rankings.

EXAMPLE II—IS RISK ASSESSMENT MEASUREMENT EVALUATION INCORPORATING BUSINESS RISK FACTORS 

For Example-Treasury System: 158 *(5*1+4*1+3*1+2*0)=158*(5+4+3)=158*12=1896

13. EXAMPLE III

13.1 Some IS auditors prefer to just rank IS projects and not the whole IS auditable universe. Example III provides a methodology to rank IS projects. Each IS project in the IS audit universe will be rated on these eight key variables using a numeric risk value ranking of 1 (low) to 5 (high). The results of these ranking judgments are then multiplied by a Weighting factor that ranges from 1 (low) to 10 (high) to give an extended value. These extended values are added together to give a total. Once the totals for each project have been obtained, the projects are ranked by risk. The framework of the annual IS audit project coverage is then built from these rankings. The categories used in Example III are listed in 13.2 and 13.3.

13.2 Measures of Effect

• Project budget—The total budget of an IS project is an important factor to consider. As a guide, some organisations rank project budgets over US$500,000 as a risk level of 4 or 5. These organisations rank budgets between US $100,000 to US$ 500,000 as a risk ranking of 2 or 3 and budgets under US $100,000 as a risk level of 1.
• Transaction volume—The total volume of transactions that are estimated to be processed by the system in a given period.
• Character of activity—The criticality of the activity and the part of the organisation that utilises the activity. Infrequent or unusual activities or projects are more likely to result in error or inefficiency and are of greater audit interest. 
• Executive management interest—This factor relates to how important the unit, function or area is viewed by executive management.
• Fall back arrangements—This factor relates to the measures that have been put in place to continue operations if the new system has problems. Factors to consider include:
  - Business continuity plans
  - Disaster recovery plans
  - Manual procedures
  - Old system

Generally speaking, if the above issues have been addressed, are achievable or are cost beneficial then the risk is lowest.

13.3 Measures of Likelihood

• Changes in procedures—The extent of procedural change or reengineering accompanying the system implementation.
• Complexity of system—Factors such as number of users, number of system modules, mainframe versus a client-server environment (centralised versus a decentralised environment), and the number of interfaces are considered.
• Project management—Consideration should be given to the following when ranking project management:
  - In-house or outside developers
  - Project structure
  - Personnel skills
  - Project timeframes

Generally, speaking the risk is shared if the project is outsourced.

EXAMPLE III—IT PROJECT RISK RANKING

14. EXAMPLE IV—IS Risk Assessment of Auditable Units

14.1 Example IV ranks various categories of auditable units in the IS auditable universe after they have been identified. The categories are listed based on the nature of risk that these units are exposed to. Relevant information, such as, financial exposure, effect on business, and scope is collected. The categories are as follows:
i. Data centre operations
ii. Application systems (production)
iii. Application systems (development)
iv. IS procurement (manpower and material)
v. Software package acquisition
vi. Other IS functions

14.2 Under each category, major risk components are enumerated. Depending on the type of risk a weight is assigned to each risk element. Each risk element is then further subdivided and a score attached to it. This risk score of a particular risk element is the product of the score and its weight. The total risk score of the function is the sum of the scores of all its risk elements. For ease of comparison, the risk score is measured on a scale of 100. Separate risk assessment sheets can been prepared for each of the auditable unit. Finally the scores obtained for each of the auditable units are consolidated and audits prioritised. 

EXAMPLE IV—RISK ASSESSMENT—IS AUDIT
i. DATA CENTRE OPERATIONS

EXAMPLE IV—RISK ASSESSMENT—IS AUDIT
ii. APPLICATION  SYSTEMS (PRODUCTION)

EXAMPLE IV—RISK ASSESSMENT—IS AUDIT
iii. APPLICATION  SYSTEMS (DEVELOPMENT

 EXAMPLE IV—RISK ASSESSMENT—IS AUDIT
iv. IS PROCUREMENT (MANPOWER AND MATERIAL)

EXAMPLE IV—RISK ASSESSMENT—IS AUDIT
v. SOFTWARE PACKAGE ACQUISITION

EXAMPLE IV—RISK ASSESSMENT—IS AUDIT
vi. OTHER IS FUNCTIONS

15. EFFECTIVE DATE

15.1 This procedure is effective for all information systems audits beginning on or after 1 July 2002.

APPENDIX-GLOSSARY

Inherent risk—The susceptibility of an audit area to error which could be material, individually or in combination with other errors, assuming that there were no related internal controls.

Residual risk—The risk associated with an event when the controls in place to reduce the effect or likelihood of that event are taken into account.

Risk—The possibility of an act or event occurring that would have an adverse effect on the organisation and its information systems.

Risk assessment—A process used to identify and evaluate risks and their potential effect.

©Copyright 2002
ISACA®
3701 Algonquin Road, Suite 1010, Rolling Meadows, IL 60008 USA
Telephone: +1.847.253.1545 Fax: +1.847.253.1443
E-mail:  research@isaca.org
Web site: www.isaca.org