AI Security: Challenges and Opportunities for IT Managers in 2026

In late 2025, a financial services firm discovered that its AI-powered customer support agent had been exfiltrating summarized conversation data to an external endpoint for eleven weeks. The attack wasn't a zero-day exploit. Someone had injected a malicious instruction into a document the agent was asked to process, and the agent — faithful to its task — followed it. No firewall caught it. No SIEM fired.

That's the new threat model. AI security isn't just about protecting the model; it's about controlling what the model does when it has access to your systems, your data, and your users.

If you manage infrastructure or own security posture for an org deploying AI agents, this post maps the real attack surface and where you can actually push back.

The Attack Surface Is the Agent's Permissions

Traditional security drew a perimeter. An AI agent dissolves that perimeter by design. It reads files, calls APIs, sends emails, queries databases, and sometimes spawns sub-agents — all in a single workflow run.

The attack surface isn't the model weights. It's the tool permissions you've granted the agent and the data those tools can reach. A customer service agent that can search your knowledge base and send emails is one prompt injection away from exfiltrating your product roadmap.

Start your threat model there: list every tool your agents can call, then ask what a malicious actor could accomplish by controlling the agent's reasoning for 10 seconds.

Prompt Injection: The Attack You're Probably Underestimating

Prompt injection remains the most practical attack vector against deployed agents in 2026. It doesn't require access to the model or your infrastructure — just the ability to place adversarial text somewhere the agent will read it.

That includes: emails the agent processes, web pages it scrapes, PDFs it summarizes, calendar invites it parses, and customer input fields it acts on. Any untrusted data the agent ingests is a potential injection surface.

Indirect prompt injection — where the malicious instruction is embedded in data the agent fetches, not in the user's direct input — is harder to catch and increasingly common. Your input validation layer needs to treat all external content as untrusted, not just user messages.

Tool-Call Auditing: You Need a Log You Can Actually Read

Most agent frameworks emit logs. Most of those logs are noisy JSON blobs that tell you what happened but not why it was suspicious. What you need is structured, queryable tool-call telemetry.

At minimum, log: tool name, input parameters, output summary, calling agent ID, timestamp, and user or session context. If your agent made a send_email call at 2 AM to an external address, you want to surface that in under 30 seconds.

Frameworks like LangGraph and AutoGen support custom callback hooks where you can intercept every tool call before and after execution. Use them. Wire them to your existing SIEM if you have one, or drop events into a structured log file you can grep.

# LangGraph tool-call audit hook (simplified)
def audit_hook(run_id, tool_name, inputs, outputs):
    log_event({
        "run_id": run_id,
        "tool": tool_name,
        "inputs": sanitize(inputs),
        "outputs": summarize(outputs),
        "ts": utc_now()
    })

For more on building watchdog tooling around agent behavior, see how to turn logs into alerts with a devops watchdog agent.

Credential Handling: Stop Putting API Keys in Context

This one is embarrassingly common. An agent needs to call a third-party API, so the developer puts the API key in the system prompt or in a config file the agent reads at runtime. Now every prompt injection that reaches that agent potentially has access to your credentials.

The fix: never pass raw credentials into an agent's context window. Use environment variables resolved at the process level, secret managers (AWS Secrets Manager, HashiCorp Vault, Doppler), or a credential broker that issues short-lived tokens per tool call.

Your agent should receive a capability ("you can call the Stripe API"), not a credential ("here is the Stripe secret key"). The runtime enforces the capability; the model never sees the key. See how to stop giving your agent raw API credentials for a concrete implementation pattern.

Multi-Agent Trust Boundaries

When agents call other agents — orchestrators spawning sub-agents, or agents communicating via message queues — you have a new category of trust problem. Can agent B trust a message from agent A? Should it?

In most current deployments, the answer is: no verification happens at all. Agent B receives a message claiming to be from agent A and acts on it. That's a spoofable channel.

The practical minimum: sign inter-agent messages with a shared secret, verify sender identity at the receiving agent's entry point, and treat messages from unknown agents as untrusted user input. If your multi-agent system runs on a message bus like Redis or RabbitMQ, apply the same access controls you'd apply to any internal service-to-service communication.

For a deeper look at memory safety in multi-agent architectures, memory safety in multi-agent systems covers the state-persistence risks that compound this problem.

Data Exfiltration via the Output Layer

Agents don't just read data — they write it. They send emails, post to Slack, write files, call webhooks, update records. Every output channel is a potential exfiltration path.

An agent that can email users can, under adversarial control, email anyone. An agent that writes to an S3 bucket can, under adversarial control, write to a bucket you didn't intend.

Output allowlists work here. Define the exact domains an agent can email, the exact S3 prefixes it can write to, the exact Slack channels it can post to. Enforce these at the tool implementation level, not in the prompt. Prompts can be overridden; code cannot (assuming your tool code isn't itself injectable).

Governance, Audit Trails, and Compliance

AI governance in 2026 isn't optional for most regulated industries. If you're in financial services, healthcare, or any sector subject to data residency requirements, your AI agent deployments need to produce auditable records of what data was processed and what actions were taken.

File-based agent configurations (think SOUL.md, AGENTS.md, or equivalent behavioral spec files committed to version control) give you a durable, reviewable record of what an agent was authorized to do. Combine that with structured tool-call logs and you have a paper trail that satisfies most audit requirements.

For the financial services angle specifically, AI agent governance in financial services covers the regulatory touchpoints worth planning for.

Where the Opportunities Are

The security challenges are real, but the opportunity side of this equation is underrated. AI agents are also good at security tasks when they're properly scoped.

Agents running static analysis on code diffs, triaging CVE feeds, correlating anomalous access patterns in log data, drafting incident reports — these are workflows where the agent's pattern-recognition and text-processing capabilities genuinely reduce manual work. The constraint is the same as everywhere else: scope the agent tightly, give it read-only access where write isn't needed, and log everything it touches.

Orgs that get AI security right aren't the ones that lock down all AI tooling. They're the ones that deploy narrow-scoped agents with strong audit trails and iterate from there.

Building a Security-First Deployment Checklist

If you're deploying or inheriting AI agent infrastructure, here's a working checklist:

None of these require buying a new product. Most can be implemented in the frameworks you're already using — LangGraph, AutoGen, CrewAI, or OpenClaw all expose the hooks you need.

What to Do Right Now

If you've read this far and you're looking at an agent deployment that doesn't have most of these controls in place, the priority order is: credential isolation first, tool-call logging second, output allowlists third. Those three address the most common actual incidents.

AI security is a moving target — new attack patterns surface as agent deployments become more complex and more connected. But the fundamentals — least privilege, audit trails, input validation, and behavioral specifications you can point to — hold regardless of what the attack looks like.

For a broader look at how these controls fit into an enterprise security posture, navigating AI security risks covers the organizational side of the problem in more detail.