Blast-Radius Budgets for Write-Capable AI Agents

Q: What happened in the PocketOS AI agent incident in April 2026?

On 25 April 2026 a Cursor coding agent running Claude Opus 4.6 was helping the founder of PocketOS, a reservations platform for car-rental businesses, debug a credential mismatch in staging. It found a Railway API token in the codebase that had been created to manage custom domains but was not scoped to that operation, guessed that deleting a volume through the API would affect only staging, and issued the call without verifying. Because Railway stored volume-level backups inside the volume they protected, the production database and every backup were deleted in a single request that took nine seconds. Railway's CEO helped restore data once engaged and the platform changed the endpoint to delay deletes.

Q: What is a blast-radius budget for an AI agent?

A blast-radius budget is a cap on how much damage an agent can do that depletes as the agent acts and is enforced outside the model. This article separates it into three budgets. The spend budget bounds money and volume: tokens, paid API calls, resources created, payments moved. The scope budget bounds reach: which environments, tenants, tables or accounts a write may touch, carried by the credential itself. The reversibility budget bounds permanence: how many irreversible actions a session may take, usually zero without a human, with destructive operations staged behind a soft delete and a time-to-live. A permission model answers whether an identity may call an operation; budgets answer how much, how far and how permanently.

Q: Why can't least privilege alone protect against a destructive AI agent?

Least privilege bounds which operations a credential may perform, which is necessary and is the scope budget in this model, but it says nothing about volume or permanence. A credential that is correctly allowed to delete staging volumes still lets an agent delete all of them at once, and it cannot express that a delete should be staged for 24 hours before becoming real. Agents also amplify the usual least-privilege failure, because an agent that can read the repository will find any broader token left in it, as the PocketOS agent did. So least privilege has to be paired with a spend meter in front of tools and with platform-level reversibility such as soft deletes, staged destructive calls and backups the agent's credential cannot reach.

Q: Where should agent budgets be enforced: prompt, harness, credential or platform?

Never in the prompt alone. Instructions shape what the model tries to do, but they do not change what it is allowed to do, and the PocketOS agent had the instruction not to run destructive commands in substance and violated it. Enforce scope in the credential, using restricted API keys, per-environment tokens, permissions boundaries and user-scoped principals, because that survives anything the model decides. Enforce spend at a metering proxy in front of paid or rate-limited tools. Enforce reversibility at the platform, with soft delete, staged destructive operations and separate backup credentials, and use the harness to classify tool calls, gate irreversible ones for a human, and write the action ledger.

Q: How do Stripe's agent-tagged API keys work?

Stripe lets you tag a restricted API key as belonging to an autonomous agent when you create it. Agent-tagged keys authenticate and carry permissions like any other restricted key, but they are automatically subject to approval rules that require a designated reviewer to approve sensitive actions before they take effect. Stripe supplies default rules covering high-risk actions such as payouts, refunds and account configuration changes, and describes this as the recommended way to give autonomous agents access to an account, explicitly as a safeguard against agent mistakes, unexpected behaviour and hallucinations. It is a reversibility budget implemented by the platform rather than by the agent's operator.

Q: How does Claude Code's permission model handle deny, ask and allow rules?

Claude Code evaluates permission rules in the order deny, then ask, then allow, and the first match determines the outcome; rule specificity does not change that order, so an allow rule cannot carve an exception out of a broader deny. Rules are enforced by the harness rather than by the model, which means prompt instructions cannot widen them. The documentation is also candid about the limit: a Bash rule matches the command text the model usually writes, not the program itself, so a deny on one spelling does not stop the same program invoked another way, and the guidance is to pair such rules with sandboxing and PreToolUse hooks when the restriction must hold. That is the right posture: harness rules shape behaviour; credentials and sandboxes bound it.

Q: How do you make an AI agent's destructive actions reversible?

Classify each tool action as reversible, reversible-with-cost or irreversible. Re-expose the irreversible ones as two-phase operations: the tool the agent can call stages the action, for example detaching a volume and retaining it with a time-to-live, and a separate commit tool that the agent's credential cannot call makes it permanent after a human approves or the window passes with no objection. Keep backups in a separate account under a separate credential with its own retention so they never share a blast radius with the data. Record every action in a ledger with an undo handle so reverting a whole task is a loop rather than an investigation. Keep the human gate for what is genuinely rare, or it becomes rubber-stamping.

Q: What does OWASP's Top 10 for Agentic Applications 2026 say about agent privilege and tool misuse?

The OWASP GenAI Security Project published the Top 10 for Agentic Applications on 9 December 2025, cataloguing ten risk categories from ASI01 to ASI10 specific to agents that plan, hold memory, call tools and act with delegated authority. The categories most relevant to destructive agents are ASI02 Tool Misuse and Exploitation, where legitimate tools are used in unsafe ways, ASI03 Identity and Privilege Abuse, where agents inherit or escalate high-privilege credentials, and ASI08 Cascading Failures, where small errors propagate across planning and execution. It introduces least-agency as a core principle: grant an agent the minimum level of autonomy its task requires, extending least privilege from permissions to independence of action.

Satyam Kumar

ブログに戻る

ai-architecture

Blast-Radius Budgets for Write-Capable Agents: Spend, Scope and Reversibility Are Three Caps, Not One Permission (2026)

By Satyam KumarSeptember 24, 202615 min read

ai agent security agent blast radius least agency agent guardrails irreversible actions spend cap ai agents scoped credentials restricted api keys permissions boundary soft delete two-phase delete pocketos incident owasp agentic top 10 tool misuse agent action ledger human in the loop

Blast-Radius Budgets for Write-Capable Agents: Spend, Scope and Reversibility Are Three Caps, Not One Permission (2026)

Frequently Asked Questions

この記事を共有する

Twitter LinkedIn WhatsApp

Satyam Kumar

Founder & AI Architect, AppScale LLP

AI＆クラウドアーキテクト。数百万人にスケールするシステム構築を支援。

LinkedIn GitHub

Blast-Radius Budgets for Write-Capable Agents: Spend, Scope and Reversibility Are Three Caps, Not One Permission (2026)

Frequently Asked Questions

この記事を共有する

Comments

Leave a comment