Enterprises rely on systems, and systems sometimes fail. A system can either break down during one of its component processes or initiate processes that run smoothly but fail to produce the desired result/output. When this happens, it raises the question of whether organizations and individuals rely too heavily on systems. In an environment filled with constant background noise about how much should or should not be automated and how to avoid the potential negative effects of artificial intelligence (AI), it is understandable for workers, particularly in large, modern organizations, to feel like they are drowning in a sea of systems. The IT audit function must help organizations examine their systems and mitigate the risk of improperly managed systems, which could otherwise hinder productivity and innovation.
Simply put, systems make sure that an organization does not consist of hundreds or thousands of employees who are making processes up as they go along.The Need for Systems
Do organizations really rely too heavily on systems? Probably not. After all, large organizations are too complex to function without the support of systems. This discussion extends beyond information systems and technology. At the fundamental level, when a task exceeds the point where it can be done by any single person, with no specialized knowledge,
on an as-needed basis, then that task needs some sort of system to expedite its associated outcome. Systems ensure that processes are conducted in a manner that is repeatable, reliable, and transferable (i.e., teachable). Simply put, systems make sure that an organization does not consist of hundreds or thousands of employees who are making processes up as they go along.
Nevertheless, system failure (or subpar system performance) does occur, and the resulting worker dissatisfaction can create real issues for an organization. System failures can cause workers to take shortcuts and workarounds to get their jobs done, become dissatisfied with their roles, or even question the mission of the entire enterprise. When these failures and frustrations occur, the problem is often due to the failure to select, implement, manage, and change systems effectively, rather than an overreliance on the systems themselves.
System Development
Some organizations feel that they must take a top-down approach to systems development, which prescribes a set of procedures and outlines the methodologies the organization must follow. In some cases, this approach is necessary, particularly as it relates to tone-setting matters such as organizational culture and core values. However, systems development often works better when it begins with the people on the front lines (e.g., engineers, product managers, team leaders), followed by individuals with a broader perspective (e.g., auditors, senior leadership), who can make informed adjustments and leverage insights that may elude those involved with daily operations.
Proceed from Need
“Unless sounds are held by the memory of man, they perish, because they cannot be written down,” said the scholar Isidore of Seville.1 This statement is very old but highlights the enduring need for systems. In 650 AD, Isidore developed a new system of writing music, using a notation called neumes. Vocal chants would be written on parchment with the text, above which neumes would be notated, indicating the contour of the melody. This was a big step forward in musical notation. However, this system of notation only indicated whether one note should be higher or lower than the note preceding it. As time passed, incremental innovations such as the 5-line stave, key signatures, time signatures, and cues for rests, accents, and dynamics coalesced into the system of musical notation we have today, a system that has remained virtually unchanged for centuries and transcends language and time.
Ideally, good business systems begin with a well-understood need and are honed over time by trial, error, and incremental improvement until they become well-oiled machines. In reality, however, system implementation can be driven by a variety of factors. Sometimes, systems are implemented to send signals to the marketplace. Organizations aiming to cultivate a certain image may adopt systems or operational approaches that convey their status to the marketplace as leaders in the industry. This is not always a bad thing, however, the risk of redundancy, and ultimately system failure, increases when too much emphasis is placed on the desire to adopt the system itself as opposed to solving a well-understood problem.
Problems can also occur when grafting systems from outside sources onto one’s own organization, particularly if not enough care is given to scrutinizing the system for its ability to handle unique circumstances. In some cases, such as with laws, regulations, and certain industry standards, it is advisable to implement a system that meets the specifications of an outside party. This is because the ultimate aim is to comply with the law or uphold certification within a given industry. Then, of course, there are customer expectations and requirements that must be met as a condition of doing business. Organizations may have to develop, adopt, or modify their systems to accommodate prospective customers, and often they will determine that the revenue generated from pleased customers is worth the cost and effort of doing so.
Ultimately, compromises must be made to meet the needs of various stakeholder groups. However, when organizations unnecessarily impose external systems on their workers—forcing them to conform to someone else’s approach—it can lead to counterproductive outcomes, wasted effort, and diminished morale.
System Implementation
While long-term, grassroots system development has great benefits, circumstances often dictate that systems must be put into place relatively quickly.
This could be in response to a security incident, a change in regulatory requirements, or a change in customer demand for a product or service. Ensuring a well-thought-out implementation plan reduces the risk that a particular system will be burdensome to workers downstream.
As it relates to IT systems, system implementation may prompt decisions and actions that can have lasting effects post-project completion, including:2
- An inability to implement certain technologies due to incompatibility with the system
- Financial losses due to the cost of maintaining the system
- Legal and security implications from the project
- Impacted business continuity due to lacking disaster recovery plans and procedures at the time of implementation
Themes such as compatibility and continuity are essential to IT system planning. In other words, auditors need to be looking for synergy between systems. Generally, leaders in individual business areas are highly capable of selecting (or designing) a good system for what they want to accomplish. It is much more difficult to ensure that there are no disconnects with other processes in the organization. What is needed are people in the middle, such as IT systems auditors, who can see the perspective of the subject matter experts while assessing compatibility with high-level concerns such as regulation and pending regulation, long-term business strategy, and other systems. These concepts should be central to the overall approach of the IT audit team.
System Management
Regardless of how a system is utilized, attentive system management is critical. As organizations rely more on automated systems, system management is taking over the duties of personnel both inside and outside the IT department.
Nowhere is this more evident than in the proliferation of AI systems and the emerging governance frameworks attached to them. In its 2024 AI business predictions, PwC reports that nearly three-quarters of US companies have already adopted AI in at least some areas of their business.3 AI/machine learning (ML) applications are enabling organizations to accomplish tasks at a scale that was, in some cases, previously unimaginable, particularly as it relates to advanced data analytics. But, as established in AI governance frameworks such as the EU AI Act,4 the National Institute of Standards and Technology (NIST) AI Risk Management Framework,5 International Organization for Standardization (ISO)/International Electrotechnical Commission (IEC) standard ISO/IEC 42001,6 and others, the use of AI systems places a new burden of responsibility on organizations to ensure that the systems are trained on the right data, based on empirically validated models, and regularly monitored for the accuracy and fairness of output. “Besides learning how to use AI responsibly,” PwC notes, “Middle managers will need skills to oversee and assess teams in which AI agents do much of the work.”7
As this reality is playing out in business management, auditors will need to respond accordingly. In an article on IS audit, a crucial observation stood out: “Modern [IT] solutions do not merely assist in the process but are often the process itself…The information system auditor's approach to what to audit will need to change. An isolated audit of one application solution or an infrastructure setup may serve a limited purpose. The interconnections and integration between solutions also need to be considered during these audits.”8
Increasingly, system management needs to be about system optimization as much as it is about verifying that the system is working as intended. It may be working as intended, but that does not mean it is meeting the current needs of the organization. For example, systems that are unnecessarily complex, prescriptive, or restrictive need to be revised, but it is difficult for teams to devote energy and resources to sharpening the axe, so to speak, when they are absorbed in day-to-day duties. Auditors are adept at identifying anomalies and assessing whether processes align with their intended functionality. A highly skilled auditor will analyze root causes and proactively recommend changes to optimize the system whenever the opportunity arises.
System optimization provides a clear and quantifiable way to communicate savings in real dollars. This is a great asset for the audit team and is more likely to result in recommendations for system changes to be accepted and acted upon. System optimization has rewards that go beyond cost savings, including improved employee engagement and morale.
System Change
Systems can be hard to change, but if they are not monitored and scrutinized, they can stifle innovation and create inefficiencies. Making changes to systems incurs upfront costs and ongoing expenses for maintenance and employee training. However, personalities and egos can get in the way of these efforts. Moreover, a desire for control or a lack of trust can obscure an objective view of the existing system. An audit cannot necessarily correct all of these issues, but it can provide an independent, unbiased assessment of system suitability and efficiency, which makes the audit function critical to knowing when it is time to pivot from one way of doing things to another. As noted in a recent Harvard Business Review article, “Every organization should consider developing a gatekeeping process…to vet prospective IT systems before committing employees’ time—and company dollars—to implementing them.”9
Conclusion
As emerging technologies foster the growth of what practitioners can accomplish, they will need to rely on systems in order to do those things in a reliable, repeatable way. Thus, one’s job, whatever the job may be, will become increasingly about systems management. This reality represents an opportunity for the IT audit function, which is uniquely well equipped with independence and subject matter expertise, to help stakeholders throughout the organization scrutinize their systems and understand system interconnectivity, ultimately resulting in more streamlined, user-friendly, and effective systems.
Endnotes
1 Classicfm,“How Did Music Notation Actually Begin?,” 17 March 2018, https://www.classicfm.com/discover-music/how-music-notation-began/#:~:text=Around%201250%2C%20Franco%20of%20Cologne,for%20minims%2C%20crotchets%20and%20semiquavers
2 Seedat, H.; “Plan for Successful System Implementations,” ISACA® Journal, vol. 2, 2020, https://www.isaca.org/archives
3 PwC, “2024 AI Business Predictions,” https://www.pwc.com/us/en/tech-effect/ai-analytics/ai-predictions.html
4 Future of Life Institute, “The EU Artificial Intelligence Act,” https://artificialintelligenceact.eu/
5 National Institute of Standards and Technology, “AI Risk Management Framework,” USA, 2023, https://www.nist.gov/itl/ai-risk-management-framework
6 International Organization for Standardization/International Electrotechnical Commission, ISO/IEC 42001:2023—Information technology artificial intelligence management system, 2023
7 Op cit PwC
8 Sayana, A.; “The Evolution of Information Systems Audit,” ISACA Journal, vol. 1, 2022, https://www.isaca.org/archives
9 Campion, M.; Campion, E.; “Research: When New IT Systems Shift the Burden onto Employees,” Harvard Business Review, 15 February 2021, https://hbr.org/2021/02/research-when-new-it-systems-shift-the-burden-onto-employees
KEVIN M. ALVERO | CISA, CDPSE, CFE
Is chief compliance officer at Integral Ad Science. He leads the company’s regulatory and industry standards compliance initiatives, spanning its global ad verification products and services.