TL;DR
On October 16, NIST, NSA and CISA issued voluntary guidance for training and evaluating highly capable agents. One concern is an agent taking a significant action nobody instructed it to take and crossing a control boundary.
For ordinary business agents, I’d add a response path for consequential control failures, even before investigators know the cause.
Stopping the agent is only containment. Preserve the evidence, investigate the action and name who can approve a restart.
Imagine a procurement agent that can compare suppliers, request quotes and prepare purchase orders. Its instructions say that a person must approve any new supplier before an order goes out.
Now suppose the agent adds a new supplier and places an order anyway. The credentials are legitimate. The purchasing application accepted the request. Nobody has found a stolen password or an attacker-controlled prompt.
Those facts don’t tell you whether the purchase was authorized. They tell you the software let it happen.
Procurement has to work out what the order commits the company to. Engineering needs to find out why the approval step failed. Security has to check whether someone manipulated the agent or bypassed a control.
All three have a job. I’d want the company to decide in advance who can declare an incident and stop the agent while those teams work out what happened.
What the new federal guidance actually says
On October 16, NIST, the NSA and CISA published ten voluntary practices for incident readiness and response while training and evaluating what they call super intelligence agents. The document is aimed primarily at organizations running very capable agents on long tasks with limited human supervision. It isn’t a new compliance requirement for every company deploying an AI assistant.
The joint document defines a super intelligence (SI) control incident. In plain English, the agent takes a significant action that its operator didn’t instruct it to take, and the action compromises a boundary set to limit the task or maintain human oversight. The agencies also recognize near-misses, where the risk was reasonably foreseeable or imminent but didn’t become an incident.
There’s an unusual distinction in the definition. An action directed by an adversary, including through prompt injection, isn’t uninstructed under this specific category. The agencies treat that as a different cause. A prompt-injection attack can still be a security incident. And an uninstructed agent action can also create a traditional cybersecurity incident if it damages a system or exposes information.
So I wouldn’t copy the federal label into a corporate runbook and treat it as a universal classification rule. I’d borrow the part that changes the investigation: the absence of an attacker doesn’t make a consequential control failure harmless.
Traditional incident response already covers violations of security policy, compromised systems and threats to information. The new guidance builds on that foundation. I’d give agent-control failures a clear path through the existing response process. Whether a particular failure is a cybersecurity incident still depends on the action and the boundary it crossed.
Classify the action before arguing about its cause
Return to the procurement example. The first report says, ‘The agent sent an unapproved order.’ At that point, the team may not know whether someone altered its instructions, the approval check was broken or the agent found another way to use an authorized tool.
It doesn’t need to solve that question before deciding whether the agent should keep placing orders.
I’d use two questions in the first response:
What did the agent actually do, and what could it still do? Capture the transaction, the amount, the recipient and the systems the agent can reach. Check whether the same credentials or workflow can repeat the action. If other agents share the same tool or policy, they may need to be paused too.
Which boundary was supposed to stop it? Find the business rule, the approval step and the technical enforcement point. If nobody can identify the rule, the problem may be an authority gap rather than a bypass. I wrote about that earlier in Your AI Agent Can Commit the Company. This case starts after the action has already happened.
The incident commander should own the response process. The business owner should assess what the action committed the company to and whether it can be reversed. Engineering should reconstruct the agent and tool behavior. Security should investigate whether access or controls failed and whether an adversary was involved. Legal and privacy join when the facts call for them.
Those roles can overlap in a small company. The ownership of the decision can’t disappear between them.
I wouldn’t classify every odd answer or harmless tool retry as an incident. Severity should follow the real or reasonably imminent consequence, the authority crossed and the ability to contain the behavior. A blocked attempt with no exposure may be a near-miss. A completed external payment, irreversible data change or unauthorized disclosure deserves a different response. The classification can change as the investigation develops, and the team should record why.
Your chat transcript won’t be enough
The joint guidance spends real attention on evidence. It calls for preserving configurations, environment state, reliable timestamps, tamper-resistant logs, tool calls that succeeded and those that were rejected, and enough information to reconstruct an agent’s activity.
This matters. If the only record is a final answer saying the task is complete, investigators may never see the call the agent attempted, the approval it skipped or the retry that eventually succeeded.
For an enterprise agent, I’d want a reconstructable chain: who assigned the task, the instruction and policy versions, the agent identity, each tool request and response, the approval decision, the resulting external transaction and the time each step occurred. Capture the surrounding application logs too. A model’s self-explanation isn’t a substitute for the tool and system records.
Preserve that evidence somewhere the affected agent can’t change it. If the agent used credentials that might still permit harmful actions, restrict or revoke them as the incident warrants. Don’t let an enthusiastic cleanup script erase the run state the investigators need.
There’s a practical constraint here. Some models and hosted services don’t expose every internal reasoning trace. You can’t preserve data you never had. That makes independent records of tool calls, decisions and effects even more important. It also makes observability a purchasing and design question, not something the incident team can retrofit after the fact.
The goal isn’t to reproduce every token. It’s to establish what the agent was authorized to do, what it tried to do and what the connected systems actually did.
Who gets to turn it back on?
Stopping an agent is the easy part to put on a diagram. Recovery is where the organization has to make a harder decision.
The federal guidance says to verify the repaired controls before restarting. It recommends rebuilding potentially compromised environments from trusted versions, checking that logging and restrictions work, replaying incident actions against updated protections and documenting the remaining risk. It also calls for leadership to be satisfied with the risks before a high-risk training or evaluation run starts or resumes.
For ordinary business agents, I’d make the restart decision smaller and more concrete. Before returning the procurement agent to service, the team should be able to show that the unapproved order path now fails. The approval gate should reject the same action when replayed in a controlled test. The agent’s credentials should permit only the tasks it still needs. Monitoring should catch a repeat attempt. Procurement should confirm how any outstanding order was handled.
Then name the person accepting the residual business risk. Security can verify the technical controls. Engineering can attest to the fix. Neither should silently decide that an unauthorized financial commitment is acceptable to procurement.
A green test run isn’t proof an agent will never make another mistake. It’s evidence that the specific failure path was addressed, alongside whatever broader testing the severity calls for.
The distinction also matters if the agent caused damage without breaking a technical permission. A workflow might allow the action because someone granted authority too broadly. In that case, restarting with the same policy and a new prompt is not much of a recovery plan.
Run the exercise while nothing is broken
Pick one agent with permission to send something outside the company or change a consequential record. Use an isolated test environment with no production access or real external recipients.
Create a harmless test case in which the agent completes one action outside its approved scope. Capture the actual tool and approval records, then give the response team the event without identifying its cause. Ask someone to assign severity using the information available at that moment. Have engineering reconstruct the action from the records. Have the business owner identify the consequence that action would have had in production. Then ask who can pause the agent, who can reverse the action and who signs off on restarting it.
If the drill stops at ‘we can disable the agent,’ keep going. You haven’t tested the investigation or the recovery decision.
Before the exercise ends, have the person approving the restart write down what has to change and what evidence they’d need before turning the agent back on.
Resources
NIST, NSA and CISA, Key Practices for Incident Readiness and Response in Super Intelligence Training and Evaluation, October 16, 2026. Voluntary joint guidance, especially definitions and practices 2, 5, 6 and 7.
NIST announcement of the joint guidance, October 16, 2026. Describes audience, scope and five-stage lifecycle.
NIST SP 800-61 Rev. 3, Incident Response Recommendations and Considerations for Cybersecurity Risk Management, April 2025. The existing incident-response framework on which the joint guidance builds.
Source and scope note: The hypothetical procurement incident, the proposed enterprise response roles and the restart checklist are my operational interpretation. They’re not incidents documented in the federal guidance or obligations imposed on all enterprises.


