JOnline: Enterprise Architecture Metrics in the Balanced Scorecard for IT 

 
Download Article

The increasing concern about the value of IT drives numerous efforts worldwide toward better alignment between business and IT. IT governance and enterprise architecture (EA) are two areas found to be rewarding by both practitioners and researchers. However, most of the approaches provide a holistic system of one aspect and a fragmented solution in others. Based on models for strategic IT management supported by the balanced scorecard (BSC) method, this article provides further information about measurement of interrelated IT goals, particularly those related to enterprise architecture.

IT is generally recognised as failing to meet business expectations.1 The reasons are found mainly in the lack of IT-business alignment,2, 3 problems with IT strategy execution,4 and inadequate performance measurement and control. The lack of alignment is a common problem not only between, but also within, business and IT domains of many organisations. It is often missing between strategy definition and execution in business5 and IT,6 as well as between business and IT strategies where the latter are not consistent with the former and the strategic goals are not interlinked with causal relations. Another common misalignment is between strategy and organisation.7 The efforts of researchers and practitioners toward achievement of these alignments have led to the introduction, worldwide adoption of and increasing interest in methods and disciplines such as BSC, IT governance and EA.

One attempt to suggest a coherent model for IT strategies is presented in A Coherent View on IT Strategies,8 which is briefly described later in this article. However, such an approach can be successful only if applied uniformly in all management phases—strategy definition, planning, design, implementation and control. An integrated approach that could serve planning, design and implementation is presented in ‘Integration of IT Strategy and Enterprise Architecture Models’,9 which proposes a way to maintain relations amongst BSC, EA and project management objects in a single modelling repository, thus providing an integrated environment for methods, processes, models and objects and enabling holistic strategy, project and architecture management.

This article supplements these frameworks by discussing the applicable metrics. There is a lot of experience with indicators pertinent to commonly used strategy perspectives, such as internal business processes and clients. However, measuring the value of EA (the new perspective in the BSC for IT) is a very challenging task and only a few researchers have attempted to provide an adequate set of metrics. For that reason, EA metrics are given special attention in this article.

The second section of this article presents the new BSC model for IT, based on the coherence10 view achieved by applying the service-based concept. The third section elaborates on the metrics for EA, and the fourth presents some aspects and results of an implementation case in a large government agency.

Balanced Scorecard for IT

The alignment of business and IT has been a major concern during the last 20 years. It has driven much effort and investment in creation of unified notations and languages for better communication of business and IT aspects and their relations, improvement of IT governance, e.g., by better definition of control objectives,11 standardisation of IT processes and services, 12, 13 and extending the information systems (IS) architecture to provide a holistic view that includes the business tier and environment. 14, 15 The starting point of such an alignment should be sought in the rationale behind business investment in IT.

What the business actually ‘buys’ is the value created by IT. The ‘value’ is a universal indicator for motivation at all levels. Business exists to add value in the value chain that is created internally in business processes, and uses the value provided by software applications and the value of the information and communication technology (ICT) infrastructure. The value is created within different domains of an enterprise and is exchanged between domains. The service concept is well suited to represent value exchange between domains. A service is defined as a unit of functionality that an entity makes available to its environment, and that has some value for other entities in the environment.16 As shown in the left diagram in figure 1, the ICT infrastructure layer provides infrastructure services (e.g., processing, storage, communication) to the application layer, which provides application services to the business layer, which in turn provides business services to its clients.17 Business clients use application and IT services as part of business services, but more often they use application services by directly interacting with businesses through the Internet and other media. Likewise, they can use IT services provided by IT personnel, such us help desks for e-services.

Figure 1

The business makes use of IT services to achieve its objectives. Those IT services are basically two types:

  1. Organisational IT services realised by IT processes, some directly (e.g., maintenance), some indirectly (e.g., application development). The latter are consumed in the information system’s life cycle and used by the business in the form of application services (left diagram in figure 1).
  2. Application services realised by software applications using ICT infrastructure services (e.g., storage) realised by the ICT infrastructure18

The described coherence model could be used for IT strategy management along with other models and techniques to help solve aforementioned problems with alignment, execution and performance control. One such application is the definition of perspectives when a BSC method is used for IT strategy management. The balanced scorecard19 can be applied to the IT function and its processes, as has been described in various models.20, 21, 22, 23, 24 Although these models have been proven successful in a number of organisations, the 2004 research of the IT Governance Institute (ITGI) revealed that only 48 percent of IT professionals evaluated the BSC as effective.25 This justifies the proven need for BSC systems to increase their adaptability and applicability. Based on the cohesion concept explained earlier, another set of perspectives, as shown on the right side of figure 1, might be considered as an option to existing models.26, 27, 28 The perspectives are those of IT personnel and partners, IT process owners, enterprise architecture, internal and external users of information systems, and business owners. Going from the bottom up, changes in IT personnel—planned and measured by the pertinent indicators—cause changes in IT processes, which then enable changes in the IT architecture. The enterprise architecture perspective sustains the holistic view. On top of that are the perspectives of internal and external users and business owners (figure 1). Thus, the cause-and-effect relationships between IT goals naturally flow in parallel with the value chain expressed by the services in the coherence model.

A key factor in the success of any performance-based management system is the definition of adequate metrics. The perspective of EA represents a special challenge in this respect. The measurement of the value returned by the EA is increasingly under the consideration of researchers and practitioners.29, 30, 31 The framework proposed by a group of researchers from the University of St. Gallen adopts a BSC approach of the measurement of EA itself, detailing perspectives of services, processes, assets and finance within IT architecture, IT, enterprise and cross-company frames.32 A. Vasconcelos has developed a detailed framework of evaluation metrics for information system architecture (ISA) called the CEO Framework for ISA Evaluation Metrics.33

The IT scorecard (the strategic IT goal) is assigned to the responsible organisation unit, realised through (a set of) initiatives, influenced by success factors (SF) and other strategic goals, and measured by different indicators. A strategic IT goal related to EA is normally connected to the change of one or a set of quality characteristics, such as systems usability, security, portability and technical interoperability (figure 2). Any applicable EA metric will indicate a change in the state of one or more of these characteristics. Based on the CEO Framework for ISA Evaluation Metrics with the addition of and adaptation to the discussed IT BSC model, 11 metrics for EA are shown in figure 2 and the computation of four of them is shown in figure 3.

Figure 2

Figure 3

Application

The IT BSC model described in this paper has been applied in a large government agency, referred to in this article as GAX. The IT strategy of GAX has four themes called ‘strategies’. Three of them are related to the three main business domains using IT services—core business processes, support business processes, and relations with clients and external parties. The fourth theme was related to improvement of the IT capacity.

About 40 strategic goals were defined and allocated into six perspectives. The map showing cause-and-effect relations between strategic goals in the third strategy, e-government, is shown in figure 4 together with example indicators. (The relations to and from strategic goals belonging to the other three strategies as well as those to and from success factors are filtered out.) One example is that changing the technology of a particular e-service so that it could be accessed by more than one browser, measured by PCAF and POSF (figures 3 and 4), and combined with the automation of additional steps of a client process that leads to end-to-end online transactions, measured by PASF (figures 3 and 4), leads to an increased e-service penetration rate, which was a goal in the upper perspective of GAX external users and was further supported by higher business goals (figure 3).

Figure 4

Each indicator had a different weight in calculating the value of the particular strategic goal measured by it. Thus, GAX was able to keep a coherent view on the execution of its IT strategy, controlling planned and actual values of its performance indicators; the results of the decisions and initiatives; and the causal relations between strategic goals and their contribution to the business goals. The BSC models and their variants were stored in a single object repository together with models for EA and included organisational, business and IT processes; data; applications; and technical infrastructure. As models related to GAX business were kept in the same repository, GAX was able to maintain consistency amongst domains and over time.

Conclusion

Including the EA perspective in the IT BSC together with a system of appropriate metrics can support the execution of IT strategies and their alignment with business strategies. Storing objects pertinent to the BSC and EA in a single repository can help IT governance professionals and management in planning, communication, impact evaluation and control. Further research could provide reference models for evaluating the causal relations between performance indicators within and between IT BSC perspectives.

Endnotes

1 Van Grembergen, W.; S. De Haes; Lighthouse Global; Measuring and Demonstrating the Value of IT, IT Governance Institute, USA, 2005
2 Brown, R. M.; A. W. Gatian; ‘Strategic Information Systems and Financial Performance’, Journal of Management, Information Systems, (11:4), 1995, p. 215-248
3 McDavid, D.; The Business-IT Gap: A Key Challenge
4 Charan, R.; G. Colvin; ‘Why CEOs Fail’, Fortune, 21 June 1998
5 Kaplan, R.; D. Norton; The Strategy-Focused Organization, 2001
6 Op cit, Charan and Colvin
7 Op cit, Kaplan and Norton
8 Velitchkov, I.; A Coherent View on IT Strategies, 2007
9 Velitchkov, I.; ‘Integration of IT Strategy and Enterprise Architecture Models’, CompSysTech 08, 2008
10 ‘Coherence’ is a term used by the ArchiMate founders and other EA authors. A coherence view on EA is a crosslayer view; coherence defines the view object and is not a characteristic of the view itself.
11 IT Governance Institute, COBIT 4.1, USA, 2007
12 Office of Government Commerce (OGC), IT Infrastructure Library (ITIL), Service Management, Version 3, UK
13 Van Grembergen, W.; D. Timmerman; ‘Monitoring the IT Process Through the Balanced Scorecard’, Proceedings of the Ninth Information Resources Management (IRMA) International Conference, Boston, Massachusetts, USA
14 Zachman, J.; ‘A Framework for Information Systems Architecture’, IBM Systems Journal, vol. 26, no. 3, 1987
15 Sowa, J. F.; J. A. Zachman; ‘Extending and Formalizing the Framework for Information Systems Architecture’, IBM Systems Journal, vol 1992
16 Lankhorst, M.; ArchiMate Language Primer, Telematica Institute, 2004
17 Ibid.
18 Lankhorst, M.; Viewpoints Functionality and Examples, Telematica Institute, 2004
19 Kaplan, R.; D. Norton; Translating Strategy Into Action—The Balanced Scorecard, Harvard Business School Press, 1996
20 Gold, C.; ‘Total Quality Management in Information Services—IS Measures: A Balancing Act’, Research Note, Ernst & Young Center for Information Technology and Strategy, USA, 1992
21 Willcocks, L.; Information Management: The Evaluation of Information Systems Investments, Chapman & Hall, UK, 1995
22 Van Grembergen, W.; S. De Haes; ‘Measuring and Improving IT Governance Through the Balanced Scorecard’, Information Systems Control Journal, vol. 5, 2005
23 Van Grembergen, W.; ‘Balanced Scorecard and IT Governance’, Information Systems Control Journal, vol. 2, 2000
24 Op cit, Van Grembergen and Timmerman
25 Williams, Paul A.; Lighthouse Global; Optimising Value Creation From IT Investments, IT Governance Institute, USA, 2004
26 Reo, D. A.; The Balanced IT Scorecard—Quality of Strategy vs. Strategy Execution, European Software Institute, 2002
27 Op cit, Van Grembergen 2000
28 Op cit, Van Grembergen and De Haes, Information Systems Control Journal, 2005
29 Rico, D. F.; ‘A Framework for Measuring the ROI of Enterprise Architecture’, www.igi-global.com/files/prefaces/joeuc%20preface%2018(2).pdf
30 Schelp, J.; M. Stutz; ‘A Balanced Scorecard Approach to Measure the Value of Enterprise Architecture’, Via Nova Architectura, 2007
31 Vasconcelos, A.; CEO Framework Information System Architecture Evaluation Metrics, Instituto Superior Tecnico, 2007
32 Op cit, Schelp and Stutz
33 Op cit, Vasconcelos

I. Velitchkov
is an IT entrepreneur, enterprise architecture consultant and university professor. Currently, he works as a freelance consultant and instructor. He is participating in several research projects related to enterprise architecture and is developing a framework for IT strategy management.


ISACA Journal, formerly Information Systems Control Journal, is published by ISACA, a nonprofit organization created for the public in 1969. Membership in the association, a voluntary organization serving IT governance professionals, entitles one to receive an annual subscription to the ISACA Journal.

Opinions expressed in the ISACA Journal represent the views of the authors and advertisers. They may differ from policies and official statements of ISACA and/or the IT Governance Institute and their committees, and from opinions endorsed by authors, employers or the editors of this Journal. ISACA Journal does not attest to the originality of authors’ content.

Instructors are permitted to photocopy isolated articles for noncommercial classroom use without fee. For other copying, reprint or republication, permission must be obtained in writing from the association. Where necessary, permission is granted by the copyright owners for those registered with the Copyright Clearance Center (CCC), 27 Congress St., Salem, Mass. 01970, to photocopy articles owned by ISACA, for a flat fee of US $2.50 per article plus 25¢ per page. Send payment to the CCC stating the ISSN (1526-7407), date, volume, and first and last page number of each article. Copying for other than personal use or internal reference, or of articles or columns not owned by the association without express permission of the association or the copyright owner is expressly prohibited.

Subscription Rates:
US: one year (6 issues) $75.00
All international orders: one year (6 issues) $90.00
Remittance must be made in US funds.