Human-Speed Response Is Not a Control Against Machine-Speed Intrusion

Human-speed response is not a control against machine-speed intrusion.

Spain's data protection authority reported its first breach notification involving an attack allegedly carried out through an AI agent. The investigation is still early, and there is a lot we do not know. The regulator has not identified the organization, attacker, or model provider, and we should be careful about making claims the evidence does not support.

What caught my attention was the sequence. According to the organization that reported the breach, the agent used valid credentials, searched the environment for weaknesses, continued probing after it got inside, modified personal data, and accessed invoices. Whether every step was truly autonomous is still being investigated. The important part for enterprise leaders is that this kind of system can move through a chain of actions much faster than the people responsible for responding to it.

Human in the loop is not the same as reaction time

We talk constantly about keeping humans in the loop with AI. I believe in that principle, but I think we sometimes use it as shorthand for a control that does not actually exist. A human can be accountable for a system and still have no realistic chance of reacting before that system has completed several additional steps.

That matters because most incident response was built around human speed. An attacker gets access. They explore the environment. They try another system. They elevate privileges. They touch data. Somewhere along the way, security tools produce enough evidence for someone to connect the dots and respond.

Agents compress that timeline. By the time an analyst understands the first alert, the system may already have moved through several more actions. Requiring a person to approve every defensive response sounds safe until the approval process is slower than the thing causing the problem.

Removing people from the decision is the wrong fix. Separate human accountability from human reaction time.

Containment has to move at the speed of the agent

Security systems already work this way. Firewalls block traffic without waiting for the CISO. Endpoint tools quarantine malware before an analyst has completed the investigation. Banks stop suspicious transactions and ask questions afterward. We do this because some forms of containment lose most of their value if they happen five minutes too late.

Agentic systems make that principle more important because the thing being contained can adapt. It can call another tool, try another path, create an account, change a permission, or continue operating somewhere else. If the environment detects behavior outside the boundaries of the task, it needs the ability to narrow that authority immediately.

That might mean suspending a token, terminating a session, blocking outbound access, isolating the workload, or removing access to a sensitive system. Those actions should happen under policies defined by people, but the response itself may need to happen automatically.

The human role begins where judgment becomes important. Was this actually malicious activity? Was the containment appropriate? Should access be restored? Does the incident require customer notification? What needs to change before the system runs again? Those are decisions I want people making.

Containment cannot destroy the evidence

It is easy to imagine an organization responding to a misbehaving agent by killing the session, revoking the credentials, wiping the environment, and congratulating itself for stopping the incident. Then the investigation starts and nobody can explain what the system actually did.

If we are going to give agents meaningful authority, we need to preserve the record of how they used it. That means task state, tool calls, permissions exercised, identity changes, network activity, data accessed, generated artifacts, approvals, and interactions with other agents or systems. The goal is not perfect observability. The goal is enough evidence to reconstruct the event and improve the system afterward.

Policy has to become the operating environment

This is where a lot of enterprise AI governance is still too dependent on documents. We write policies saying that agents require oversight. We define approval processes. We classify use cases by risk. All of that is useful, but none of it stops a live system that has already started doing the wrong thing.

At some point the policy has to become part of the operating environment. The approval has to map to an actual permission. The boundary has to be enforceable. The kill switch has to work. The evidence has to survive.

Humans stay important. Put them where they add the most value. Machines can detect and contain a sequence that is moving too quickly for a person to follow. People remain accountable for deciding what the sequence means, whether the response was appropriate, and what happens next.

That is a much more useful definition of human oversight than asking someone to watch an agent move faster than they can.