Professional information systems (IS) auditors are faced with several difficult decision-making situations while maintaining their professional integrity and independence, especially if the client happens to be heavily involved in business-to-business (B2B) e-commerce. Therefore, the B2B e-commerce audit engagement planners are faced with the daunting task of planning critically needed human resources for auditing extremely complex technologyrich audit environments. Mostly, these audit planners intuitively decide on the choice of such dissimilar and difficult-to-obtain resources based on their own cost-benefit analysis, using highly subjective judgment and experiential learning. The top 10 critical success factors (CSFs) for achieving success in B2B e-commerce audit engagement resource planning programs are presented in this article. These CSFs are identified through a globally administered web-based survey of IS audit professionals1 experienced in various levels of the audit organization.
In general, the issue of e-commerce remains not well understood by top management. The issue of e-commerce audit is often the most pervasive and wide-ranging across the organizational components of an entity. Business organizations face the challenges of establishing management expertise and control mechanisms that are necessary for the successful implementation of newer e-commerce networks, e.g., B2B and business-to-consumer (B2C), that can accommodate the onslaught of information technology (IT) in the years to come. Today’s e-commerce is dependent on various technologies, such as JAVA, ASP, client-server computing, Pearl, CGI-Bin, Secure Sockets Layer (SSL), Transmission Control Protocol/Internet Protocol (TCP/IP), Asynchronous Transmission Mode (ATM), Common Object Request Broker Architecture (CORBA), Distributed Component Object Model (DCOM), enterprise resource planning (ERP), intelligent agents and so on. The Internet, extranets and intranets are designed and devised on various platforms with different layers of security. The overall impact of these complex and higher-plane technologies is now visible in modern-day auditing processes. The pervasive nature of IT, the favorable economic and functional versatility of modern computing technology, and the globally open and competitive market forces that drive the rate of technological evolution all favor an era of profound change in the marketplace and work culture. Most of the trends underway are expected to change the ways in which organizations conduct their business and auditors conduct their audits.2 Modern IT auditing processes require an auditor to have a relatively higher level of understanding of these complex technologies to succeed in the job.3 Robert Half4 shows that professional accountants’ interests and competencies are increasingly expected to include a blend of accounting and IT knowledge.
This article is an effort in this direction and provides scientifically ascertained CSFs to the professional IS auditors involved at every level of the audit engagement. The present study is based upon survey data obtained through a web-based instrument from 203 IS/IT-auditing-trained e-commerce auditor respondents with a minimum of two years of B2B audit experience in a global setting (see figures 1, 2 and 3 for the level of responsibility, the level of credentials and the country of operations of the respondents). More than 85 percent of the respondents were professional accountants trained as Certified Public Accountants (CPAs) or Chartered Accountants (CAs) in their own jurisdictions.
The results suggest that the globally identified CSFs were not statistically different from those of North American e-commerce auditors, reflecting the homogenized perceptions of criticality. This is the first survey-based empirical study of IS auditing to identify CSFs in audit resource planning in B2B e-commerce scenarios.
Framework, Results and Discussion
In brief, the principles of e-commerce auditing deal with the general theories of auditing and IS auditing, such as those relating to business threats or managerial and operational assurances and controls, and with audit techniques, namely reviewing controls, analyzing a program and testing integrity.5
These principles also deal with the content of audits by identifying several candidate targets (such as IT facility management, computing and networking operations, or application developers), system targets (such as key applications, enterprise systems and databases during preand/ or postimplementation phases), and functional targets (such as system and network change management; public key infrastructure; or intrusion prevention, detection and management). These principles, in general, are applicable across the e-commerce categories and technologies. There is considerable room for professional judgment, as IS auditing in centralized or distributed computing is very different from auditing a complex, inter-networked B2B e-commerce environment.6 Therefore, in specific situations, the professional judgment of the auditor is needed to assess the scenario that controls the content and to determine the effectiveness of a given audit. The theoretical framework is based on a literature review7 and prior studies in the closely related areas of e-commerce published in various mainstream computer science and informatics, business, accounting, and auditing publications with scholarly and professional focuses.
The study attempts to answer the question: what are the CSFs for audit resource planning in B2B e-commerce audit engagements globally?
Thirty-eight potential success factors (as shown in figure 4) were suggested in the survey instrument. These were derived from discussions with professionals in the field of e-commerce audit and previously published literature, including the framework published in the International Journal of Auditing.8 Content validity, discriminant validity and internal/external validity tests were completed for reliability of the survey items in pilot/final analysis. Each item in the instrument was a potential success factor presented in the questionnaire; each was followed by two five-point Likert scales anchored at the extreme ends. The first scale sought to know whether respondents agree/disagree that the item is a success factor. The second Likert scale sought to know how much criticality is assigned to the item by the respondents. Though this second scale was a five-point Likert scale, it was methodologically transformed into a dichotomous scale or binary9 during the data analysis. The study followed analytical procedures10 for computing the CSFs.
Top 10 Critical Success Factors
The top 10 CSFs are identified in order of criticality. Audit managers should pay due attention to them while managing audit resources in a B2B engagement.
CSF No. 1: Knowledge of Organizational Security Vulnerabilities
E-commerce auditors—required for B2B audit engagements—should be knowledgeable about organizational IS security vulnerabilities. Such vulnerabilities may be internal or external to the organization, and/or logical or physical in nature. Most organizations in business and otherwise are highly vulnerable to security lapses and breaches; therefore, security audit specialists should be on the audit engagement team of a B2B e-commerce entity.
CSF No. 2: Expertise in System and Network Change Management
Auditors chosen for a B2B audit engagement should have acquired a minimum level of expertise in system and network change management, including management of various software patches sourced from an in-house team or vendors. Professional auditors with limited knowledge of IT should utilize the expertise of IT auditors to identify the vulnerabilities introduced by various software patches in networks and systems.
CSF No. 3: Security Specialist on Audit Team
The B2B audit team should have members with expertise in intrusion detection, prevention and recovery management. This expertise may be outsourced, if not available within the firm. This is a complex IT expertise, not easily available among IT auditors in general. B2B organizations are susceptible to this vulnerability due to dependence on the Internet.
CSF No. 4: Knowledge/Training in Auditing of Financials in Integrated Systems Environment
B2B audit team members should have knowledge of/training in auditing of financials in integrated accounting environments to appreciate various aspects of materiality planning and the quality of audit evidence obtained. In general, most of the professional financial auditors are expected to hold this expertise for conventional client audits in computerized scenarios.
CSF No. 5: Experience in Web Site Review/Audit
B2B audit team members should possess a minimum level of experience in web site review/audit. Since e-commerce entities are interfaced with the users through various layers of web sites and portals, web site audit experience among accountants becomes equally important. Those with experience in WebTrust are useful in B2B audits.
CSF No. 6: Training in the Technical Aspects of Web Site Review and Audit
The B2B audit team should have training in the technical aspects of web site review and audit. This is a corollary to CSF no. 5, as work experience in web site review and audit alone is not adequate for auditors. They must have technical training in several vital aspects of WebTrust/SysTrust.
CSF No. 7: Knowledge of B2B Partner Agreements
B2B audit team members should have full knowledge of various operational agreements of the relationship between B2B partner organizations. This CSF is special to the B2B environment.
CSF No. 8: Basic Training in Various OS Programming Tasks
The B2B audit team should have at least one member with basic training in various operating system (OS) programming tasks. This is achieved by providing training to technologyoriented individuals in generic/open source OS and in enterprise systems. These individuals then extend support to the team in forming opinions on the integrity of information provided by such systems.
CSF No. 9: Establishment of Audit Objectives by Audit Team
Every member of the B2B audit team should establish audit objectives in each audit situation before beginning the audit task. This is similar to the conventional audit, except that the environment in which this audit is performed changes. Here, audit objectives acquire importance from technical complexities of B2B systems. Since audit tasks are extensive and complex in an e-commerce scenario, the objectives should be set accordingly.
CSF No. 10: Minimum Experience of Auditing/Reviewing the Outsourced Software
B2B audit team members should have, at least, minimum experience in auditing/reviewing the outsourced software from third parties. This millennium has seen an increase in outsourcing by businesses. Previously, most outsourcing was for software, but now various critical business processes are also outsourced to third-party providers. There should be a trained member on the team capable of auditing the outsourced application systems and business processes in greater detail.
The CSFs described here are those found to be the most critical across the entire spectrum of B2B e-commerce enterprises. B2B clients are prone to some recurring issues in the area of systems and network change management, intrusion prevention, detection, and protection management knowledge. Such issues are important in an engagement of an e-commerce client, and an audit planner needs to give special attention to these CSFs. To be successful in B2B e-commerce audit engagement resource planning, professional auditors must pay attention to these 10 CSFs in order of criticality. To analyze the results of this study, success factor theory was employed with separate assignment of criticality to each success factor.11 In this method, success factors and critical success factors were used interchangeably in analysis of cases.
This study attempts to extend the CSF theory and provides weights to the individual assertions of success by their perceived level of criticality. Auditors involved in B2B e-commerce audit as planners should be cognizant of these CSFs to achieve efficiency in the audit resource planning stage of an audit engagement. This study also suggests that identified CSFs are equally useful for internal and external IS auditors.
1 This study could not have been successful without the professional support from ISACA, the American Institute of Certified Public Accountants (AICPA), the Canadian Institute of Chartered Accountants (CICA), and the Institute of Chartered Accountants of Australia (ICAA), all of which encouraged their membership to participate in this study during October-December 2005.
2 AICPA/CICA, “WebTrust Program—Security Principles and Criteria—Version 3.0,” 2001, www.aicpa.org and www.cica.org
3 CICA, EDP Auditing Guidelines, EDP 6, “EDP Environments—Database Systems,” CICA Virtual Professional Library, 2002
4 Half, R.; The Next Generation Accountant, Robert Half International, USA, 2001
5 Pathak, J., “Information Technology Auditing: An Evolving Agenda,” Springer, Berlin, 2005
7 Pathak, J.; “A Model of Audit Engagement Planning in E-commerce,” International Journal of Auditing, July 2003, p. 121. This article provides complete scanning of literature to identify the CSFs in e-commerce auditing. The author makes use of the same framework as suggested in the cited review paper with some modifications.
9 The responses on the fourth or fifth point were considered as 1 (meaning critical) and first, second or third were considered as 0 (noncritical).
10 Pathak, J.; Abdulkadir Hussein; S. Ejaz Ahmad; “On Quantification of the Criticality of Success Variables and Processes in Accounting and Organization Information Systems: Some Methodological Procedures and Empirical Validation,” April 2006, http://ssrn.com/abstract=987548
11 Rockart, J. F.; “A Primer on Critical Success Factors,” published in, The Rise of Managerial Computing: The Best of the Center for Information Systems Research, edited by Christine V. Bullen; Dow Jones-Irwin, USA, 1986
Jagdish Pathak, Ph.D.
is associate professor of accounting and systems at the Odette School of Business, University of Windsor. He can be contacted at firstname.lastname@example.org.
Abdulkadir A. Hussein, Ph.D.
is assistant professor of statistics at the University of Windsor.
S. Ejaz Ahmad, Ph.D.
is professor of statistics and head of the department of mathematics and statistics at the University of Windsor, Ontario, Canada.
The authors are thankful for the support, in various capacities, of ISACA, AICPA, CHL Global Associates, Security Benchmark, Institute of Chartered Accountants in England and Wales (ICAEW), ICAA, and CICA. Thanks for extensive support of the study are also due to senior partners and directors Sri Ramamurthy, Ernst and Young (now with Grant Thornton, Chicago), USA; Toni Panicea, PricewaterhouseCoopers Canada; Pat Goggins, KPMG Canada; and Jim Pryce of Deloitte Canada. We are also grateful to the University of Windsor for providing funding to carry this study under USSHRC and RTIF research grants.
Information Systems Control Journal, formerly the IS Audit & Control Journal, is published by the ISACA. Membership in the association, a voluntary organization of persons interested in information systems (IS) auditing, control and security, entitles one to receive an annual subscription to the Information Systems Control Journal.
Opinions expressed in the Information Systems Control Journal represent the views of the authors and advertisers. They may differ from policies and official statements of the Information Systems Audit and Control Association and/or the IT Governance Institute® and their committees, and from opinions endorsed by authors' employers, or the editors of this Journal. Information Systems Control Journal does not attest to the originality of authors' content.
Instructors are permitted to photocopy isolated articles for noncommercial classroom use without fee. For other copying, reprint or republication, permission must be obtained in writing from the association. Where necessary, permission is granted by the copyright owners for those registered with the Copyright Clearance Center (CCC), 27 Congress St., Salem, Mass. 01970, to photocopy articles owned by the Information Systems Audit and Control Association Inc., for a flat fee of US $2.50 per article plus 25¢ per page. Send payment to the CCC stating the ISSN (1526-7407), date, volume, and first and last page number of each article. Copying for other than personal use or internal reference, or of articles or columns not owned by the association without express permission of the association or the copyright owner is expressly prohibited.