In a world of cloud computing, software as a service (SaaS), remote work, and application programming interface (API)-intensive systems, it is dangerous to assume that everything on an organization’s network can be considered “safe.” According to the National Institute of Standards and Technology (NIST), zero trust is about moving away from static perimeter controls to providing continual security for users, assets, and resources.1
The concept of continual security has become even more essential with the emergence of artificial intelligence (AI) technologies as large language models (LLMs), copilots, and autonomous agents sit in the intersection between users, data, applications, and business processes. As a result, zero trust must consider not only employees and devices, but also machine identities, access, and automated decision making. These systems expand the attack surface and increase the potential impact of compromised identities and automated actions. As a result, organizations must proactively fortify sensitive systems and zero trust architectures against this rapidly evolving threat.
A New IAM Landscape
The eternal zero trust rule still applies—Trust should never be unconditionally granted. Therefore, organizations should still require access to be authenticated, authorized, and limited.
The IAM landscape of today is becoming increasingly difficult to manage due to the proliferation of the types and number of identities with which professionals are tasked to work with. For example, many AI systems are using service accounts, application programming interface (API) keys, OAuth tokens, workload identities, certificates, and automation secrets. NIST has released a set of zero-trust based cloud-native guidance that states that trust is not only based on location, ownership, or affiliation, but also that access to users, services, and devices must include explicit access controls.2 This evolution represents a fundamental shift from managing primarily human identities to securing a complex ecosystem of human and nonhuman identities. As organizations adopt AI and automation at scale, unmanaged or overprivileged machine identities can significantly expand the attack surface, create new paths for lateral movement, and enable attackers to automate malicious actions at machine speed. As a result, organizations must extend identity governance and zero-trust principles beyond employees and devices to encompass the full life cycle of machine identities and AI-driven workloads.
The Expansion of AI-Driven Identity
The expansion of identity has created a significant risk regarding security. A human employee may only require access to a few systems to perform their job. However, an internal AI assistant could be authorized to access multiple customer records, HR files, financial documents, legal files, and engineering repositories simultaneously. If the credentials of the AI assistant were compromised through a leak or if that assistant’s access is too broad, the AI assistant could become an extremely effective attack vector. As a result, identity has become the new perimeter. When organizations create an inventory of AI-related identities, they should first:
- Classify the privileges associated with those identities.
- Rotate credentials regularly.
- Consider using only short-lived tokens whenever possible.
- Remove any standing authorization to access that is not clearly justifiable.3
Risk and Opportunities
LLMs also present the kinds of risk that many zero trust programs are unequipped to combat. OWASP provides several threats associated with the use of LLM applications, including:4
- Prompt injection
- Sensitive information leakage
- Supply chain vulnerabilities
- Model and data poisoning
- Insufficient output handling
- Excessive agency
- Embedding and vector issues
- Misinformation
- Unbounded consumption
These issues are of great consequence because malicious instructions can be embedded within Emails, web pages, documents, and even support tickets. If an AI system processes the content that contains malicious instructions, it may potentially produce, expose, or misuse the data contained therein or otherwise retrieved by those instructions. Prompts, documents retrieved due to those prompts, outputs, and calls made to tools should all be logged as security-relevant events.5
As AI systems perform functions and respond to queries, the risk associated with their operations increases dramatically, as independent entities may now be able to perform potentially damaging actions that were not possible before. An autonomous chatbot can take actions such as updating a database record, generating software code, sending Emails, and/or calling out to other services or APIs. OWASP defines the term excessive agency as, , “an LLM-enabled system that has excessive functionality, permission, and autonomy, and therefore has the ability to perform potentially damaging actions.”6 OWASP recommends a zero trust architecture to alleviate this risk by separating the action of making a recommendation from executing that recommendation, requiring approvals for all sensitive actions, restricting the tools that autonomous agents can access to those required to complete their assigned duties, and tracking the actions taken by autonomous agents.
AI can also enhance elements of a zero trust architecture when there are proper governing procedures in place. Security teams can utilize an AI application to identify anomalous behavior patterns, summarize access patterns, determine overprivileged accounts, classify sensitive data, and assist with expedited investigations of security incidents. At the same time, it is important that AI not become an unknown entity within the security stack; this is why it is so important to establish a governance framework for managing the risk associated with AI throughout its life cycle according to NIST’s AI Risk Management Framework (AI RMF).7 AI can provide additional insight into risk scoring and/or alert prioritization; however, high impact access decisions must still be made based upon documented logic, must be subject to audit, and must have a human element of supervision.
Zero trust must consider not only employees and devices, but also machine identities, access to LLMs, and automated decision making.Next Steps for Practitioners
Organizations must maintain a real-time inventory of their AI applications, model APIs, agents, plug-ins, datasets, data connections, and machine identities to ensure a high level of visibility. After organizing this inventory, organizations should place access levels on the appropriate resources, specifically retrieval systems and service accounts. CISA depicts zero trust as an evolving modernization program through identity, devices, networks, applications, workloads, and data, rather than merely a new technology project.8 This maturation logic aligns with the proliferation of AI as many organizations will have to enhance governance and access policies in lockstep with AI development.
Attention must also be given to third party and supply chain risk. Many AI deployments rely on third-party model suppliers, SaaS copilots, open-source libraries, plug-ins, embedding models, datasets, and integration frameworks. OWASP includes supply chain vulnerabilities as a top risk to applications using LLMs. Supply chain vulnerabilities will affect the integrity of the system and increase the risk of data exposure and unsafe behavior because of compromised components, services, or datasets.9 Therefore, zero trust must be expanded to cover third-party vendors and the use of third-party AI services; this can be accomplished by providing limited access to the vendor or service, monitoring access, performing regular access reviews,10 and limiting data usage to the minimum necessary.
Conclusion
There is no way for AI to make zero trust obsolete; instead, professionals must adopt AI while following the principles of zero trust. LLMs and autonomous systems create more identities (in relation to users), expands the paths to access those identities, and allows for software to support decision making at machine speed based on the outcome of a set of activators (e.g., inputs to the AI). Therefore, the response from security professionals should not be to prevent the use of AI but rather to develop AI that is governable.
A modern zero trust program should have the capability to continuously verify user identity, devices, workloads, models, agents, data flows, and automated actions. Financially strong identity management, a least privilege approach, AI-specific testing, and well-defined governance capabilities will allow organizations to utilize AI without creating an uncontrolled trust boundary.
Endnotes
1 National Institute of Standards and Technology (NIST), Special Publication 800-207, Zero Trust Architecture, USA, August 2020
2 NIST, Special Publication 800-207A, A Zero Trust Architecture Model for Access Control in Cloud-Native Applications in Multi-Location Environments, USA, September 2023
3 NIST, Special Publication 800-207A
4 Open Worldwide Application Security Project (OWASP), “OWASP Top 10 for Large Language Model Applications”
5 OWASP, “OWASP Top 10”
6 OWASP, “OWASP Top 10”
7 National Institute of Standards and Technology (NIST), Artificial Intelligence Risk Management Framework (AI RMF 1.0), USA, January 2023
8 US Cybersecurity and Infrastructure Security Agency (CISA), “Zero Trust Maturity Model Version 2.0,” USA, 11 April 2023
9 OWASP, “OWASP Top 10”
10 CISA, “Zero Trust Maturity Model”
Adedayo Ojo, CCOA, CISSP, ASEA, Security+
Is an information security research analyst whose expertise spans emerging technologies, cybersecurity, and AI. He currently serves as Principal, Emerging Technologies and Professional Practices at ISACA®, where he translates complex topics into practical insights that help organizations navigate today's rapidly evolving technology landscape.