What Security Means for AI

In the earlier posts in this series, we talked about treating AI adoption as a transformation program, not just another technology rollout. We also talked about the foundations organisations need to move from strategy to implementation: data, security, governance, platforms, skills, and use case management. 

In this post, I want to focus on security. 

Security is often considered too late in AI adoption. A team finds a promising tool, builds a proof of concept, connects it to data, demonstrates the value, and only then asks whether the solution is secure enough to use. By that point, the organisation may already have created risk through the data it has used, the assumptions it has made, or the way the AI system has been connected to business processes. 

That is not the right way to approach AI. 

Security for AI should not be treated as a final approval step. It should be part of how AI is selected, designed, tested, deployed, monitored, and improved. The objective is not to slow adoption down. The objective is to make sure AI can be adopted safely, repeatedly, and with confidence. 

This matters because AI is no longer only a standalone tool that answers questions. In many organisations, AI is being connected to documents, emails, source code, customer records, workflows, enterprise applications, and automated actions. That changes what security needs to cover. 

 

Security is not only about protecting the AI tool 

When people first think about AI security, they often think about protecting the AI platform itself. That is important, but it is only part of the picture. 

AI security is really about protecting the whole environment around AI. That includes the data the AI uses, the prompts people enter, the model or service that produces the output, the applications it connects to, the users who rely on it, and the business decisions it supports. 

Traditional cybersecurity controls still apply. Organisations still need identity and access management, least privilege, encryption, logging, monitoring, secure development, vulnerability management, vendor assurance, incident response, and data protection. 

But those controls are not enough on their own. Organisations also need to ask whether the AI can be manipulated, whether it can leak sensitive information, whether its outputs can be trusted, and whether people understand when human judgement is still required. 

 

AI security is really workflow security 

Many AI risks do not come from the model alone. They come from the way the model is connected to the organisation. 

A large language model can answer a question. But an enterprise AI assistant may also search documents, summarise emails, analyse files, query data, generate code, call an application programming interface, or trigger a workflow. An AI agent may go further by planning steps, choosing tools, using memory, retrieving information, and acting across a process. 

That means the risk is not just “will the AI give a bad answer?” The risk is also “what could happen if the AI has access to the wrong data, follows the wrong instruction, calls the wrong tool, or takes the wrong action?” 

This is why AI security needs to be treated as workflow security. 

The model still matters, but the surrounding architecture often matters more. The organisation needs to understand the data sources, permissions, connectors, plugins, tools, identities, prompts, outputs, logs, and business processes that sit around the AI system. 

If those parts are not controlled, the organisation may secure the AI platform but still leave the real risk exposed. 

 

 

 

The first security question is: what data is involved? 

Most AI risk starts with data. 

AI tools are useful because they can work with large amounts of information, summarise it, classify it, search it, transform it, and generate new outputs from it. But the more useful the AI becomes, the more likely it is to touch sensitive data. 

That may include customer information, employee records, financial data, intellectual property, legal material, source code, credentials, commercially sensitive information, or client data. In some industries, it may also include health information, safety information, regulated data, or critical infrastructure information. 

This is why organisations need clear rules about what data can be used with which AI tools. 

For example, using an approved enterprise AI platform to summarise an internal policy may be acceptable. Pasting confidential client data into an unapproved public AI tool may not be. Using AI to draft a generic marketing email is very different from using AI to analyse personal information or make recommendations that affect a customer outcome. 

If the organisation does not provide approved tools and practical guidance, people will often find their own way. That creates shadow AI: unapproved tools, unknown data flows, weak logging, unclear vendor terms, and no reliable way to know what information has been exposed. 

The goal is not to block people from using AI. The goal is to make safe use easy and unsafe use obvious. 

 

AI creates new ways for systems to be attacked 

AI introduces risks that many organisations have not had to manage before. 

One example is prompt injection. This occurs when an attacker, or even a piece of untrusted content, gives instructions to the AI system that override what the organisation intended it to do. This becomes especially important when AI systems read emails, documents, websites, tickets, source code, or knowledge bases. The AI may treat malicious instructions hidden in that content as if they are legitimate. 

Another example is insecure output handling. If an AI system generates code, commands, queries, configuration, or instructions, those outputs need to be validated before they are used. An unsafe AI-generated output can become a traditional security weakness if it is passed into another system without proper checks. 

Data poisoning is another risk. If the data used to train, tune or improve a model is manipulated, the AI system may learn the wrong pattern or retrieve untrusted information as if it were authoritative. 

There are also risks around model evasion, model inversion, membership inference, and supply-chain compromise through third-party models, libraries, datasets, application programming interfaces, plugins, cloud platforms, and open-source components. 

These risks do not mean organisations should avoid AI. They mean AI needs to be treated as a new class of system with its own threat model. 

A useful question for security teams is: how could this AI system be misused, manipulated, or made to act outside its intended purpose? 

 

Agentic AI raises the stakes 

AI assistants and AI agents need to be treated differently. 

An AI assistant helps a person produce an answer. An AI agent can plan steps, choose tools, call systems, use memory, retrieve information, and act across a workflow. This can create significant business value, but it also changes the security model. 

With agents, the key question is no longer only “what can the user access?” It also becomes “what can the agent access, decide, and do on the user’s behalf?” 

That means organisations need to think carefully about agent identity, permissions, tool access, approval steps, logging, and accountability. If an agent uses the same broad permissions as a user, it may access more information than it needs. If it can call multiple tools, one compromised input may influence a sequence of actions. If it can use memory, that memory may become a place where sensitive information, incorrect assumptions, or malicious instructions persist. 

This is where least privilege becomes critical. AI agents should only have the access they need to perform their approved purpose. High-impact actions should require human approval. Sensitive actions should be logged. Tool use should be constrained. 

In simple terms, the more an AI system can do, the more carefully it needs to be controlled. 

 

Connectors and suppliers are part of the security boundary 

AI tools become more useful when they connect to other systems. 

Many AI solutions use connectors, plugins, retrieval systems, and emerging standards such as the Model Context Protocol to connect AI models to tools and data sources. These integrations allow AI systems to access business context and take useful actions. They also expand the trust boundary. 

A connector may expose documents. A plugin may perform actions. A retrieval system may bring untrusted content into the AI context. A third-party integration may introduce vulnerabilities or unclear data handling. 

This is why AI supply-chain security matters. 

Organisations should know which models, tools, plugins, frameworks, data sources, and vendors are involved in each AI use case. They should understand where data is processed, whether prompts or outputs are retained, whether customer data is used for training, how access is controlled, how changes are managed, and what evidence is available for assurance. 

As AI systems become more connected, vendor assessment and technical architecture review become part of AI security, not separate procurement activities. 

 

Security also means protecting trust in the output 

One of the most important security risks in AI adoption is overconfidence. 

AI outputs can look polished, structured, and authoritative. That can make people trust them more than they should. In a business context, this can lead to poor decisions, incorrect reports, weak security recommendations, misleading customer advice, or actions based on information that has not been validated. 

This does not mean AI outputs are not useful. They can be extremely useful. But they should be treated as decision support, not unquestionable truth. 

Organisations need to decide where human review is required. For low-risk tasks, a lightweight review may be enough. For higher-risk tasks, such as legal, financial, security, health, employment, safety, or customer-impacting decisions, human oversight should be explicit. 

Security teams also need to think about monitoring. AI systems can drift over time. Data changes, business processes change, user behaviour changes, and threat patterns change. A model or workflow that worked well at launch may become less reliable later. 

That means AI security does not end at deployment. Organisations need ongoing monitoring, feedback, testing, and review. 

 

Security should be proportionate to risk 

Not every AI use case needs the same level of security review. 

A low-risk internal productivity use case should not be treated the same way as an AI system that makes recommendations affecting customers, employees, safety, regulated decisions, or critical operations. 

If the security process is too heavy for every use case, people will avoid it or work around it. If the process is too light, the organisation will not have enough control over high-risk AI adoption. 

The answer is proportionality. 

Low-risk use cases should have clear baseline rules: use approved tools, do not enter sensitive data unless permitted, check outputs, and register the use case where required. Medium-risk use cases may need additional review around data, access, vendor terms, and monitoring. High-risk use cases should require deeper assessment, testing, formal ownership, human oversight, auditability, adversarial testing, and ongoing assurance. 

This is where security connects closely with governance and use case management. Security provides the risk lens. Governance defines the decision rights. Use case management ensures ideas are captured, assessed, prioritised, and monitored. These foundations need to work together. 

 

What good AI security looks like in practice 

A practical AI security foundation should answer a few simple questions. 

  • Which AI tools are approved for use? 
  • What data can be used with those tools? 
  • Who owns each AI system, workflow, or agent? 
  • What access does the AI system have? 
  • What actions is it allowed to perform? 
  • Which tools, plugins, connectors, and data sources can it use? 
  • Where is human review required? 
  • How are prompts, outputs, logs, files, and memory protected? 
  • How are vendors and third-party models assessed? 
  • How are AI systems tested before release? 
  • How are AI systems monitored after release? 
  • How are incidents, data leakage, unsafe outputs, or unexpected actions reported? 

If an organisation cannot answer these questions, it does not yet have a secure AI adoption model. It may still have AI activity, but it does not have controlled AI adoption. 

 

Final Thought 

Security in AI adoption is not about saying no to innovation. It is about making sure innovation can survive contact with real business risk. 

AI can improve productivity, reduce cost, support better decisions, and create new ways of working. But if it is adopted without security, it can also expose sensitive data, create unreliable decisions, expand the attack surface, and reduce trust. 

AI security is not just about protecting a model or writing acceptable use rules. It is about securing an ecosystem of data, users, models, prompts, agents, tools, connectors, suppliers, and business processes. 

The organisations that succeed with AI will not be the ones that ignore these risks, or the ones that prevent people from using AI altogether. They will be the ones that make secure use practical. 

In the end, secure AI adoption is not just about protecting technology. It is about protecting trust: trust in the data, trust in the systems, trust in the decisions, and trust that the organisation can use AI responsibly while still moving forward. 

 

Author: Damian Wernert 

Share