Dry-run endpoints that keep AI coding agents honest

AI coding agents should not discover whether a change is valid by making the change. That is a lazy API design, and autonomy makes the cost visible fast. An agent can retry, branch, and move to the next task faster than an operator can reconstruct an accidental permission edit, partial migration, or badly scoped deletion.

A preview action earns its place only when it predicts a specific execution with enough fidelity that a person or agent can decide whether to proceed. A response that says "valid" is not a plan. A diff that omits a cascading update is worse than no diff because it manufactures confidence.

The useful design target is simple: submit the proposed write, evaluate it against current state and normal business rules, return the planned effects and failures, then make execution refuse stale or altered plans. That takes more care than adding dryRun=true. It also gives an agent a way to repair an invalid request before it asks a person for approval.

A preview must describe the exact write

Dry-run endpoints must accept the same meaningful intent as execution and compute the effects of that exact intent. If POST /memberships can grant a role, send an invitation, add the member to a billing group, and write an audit entry, the preview needs to report each effect that execution would create.

Teams often ship a "validate" endpoint that checks JSON shape and required fields. That endpoint has a place, but it does not preview a write. It cannot tell the caller that the requested role conflicts with an existing role, that the target account is suspended, or that the invitation will consume a limited seat. Call it validation if that is all it does.

The distinction matters because agents treat successful calls as evidence. A validation-only response followed by execution leaves the agent blind to the state-dependent parts of the decision. A full preview evaluates both the request and the current world.

For every writable operation, write down the execution contract in one sentence before you design the preview:

Given this input and this observed target revision, execution will create, update, delete, or trigger these named effects.

That sentence exposes vague behavior. "Update project settings" is too broad. "Change retention_days from 30 to 14, recalculate expiry for 18 active items, and reject items under legal hold" gives the preview something testable to return.

A good preview preserves the operation's semantics. Do not turn a bulk delete into a vague count because the real list feels inconvenient. Do not use "may affect" when your service can determine the actual resources. If the resource set is too large to return inline, return a total, a bounded sample, and a cursor or report reference that lets the caller inspect the full set before execution.

The plan needs identity, scope, and consequences

A person cannot judge "12 records will change" without knowing which records and how they change. The planned action should identify its inputs, its target scope, and its consequences in forms that both a program and a human can inspect.

For an update to a single resource, a field-level diff often works well. For a deployment, the plan may need images, environments, configuration revisions, restart behavior, and health checks. For a billing change, it may need the old charge, new charge, effective date, and whether a customer receives a notification. Match the output to the domain instead of forcing every operation into a JSON Patch array.

At minimum, expose these parts:

• An operation name and an explicit preview status.
• A stable identifier for every affected resource, plus its revision when your service supports revisions.
• The prior and proposed values for each meaningful change.
• Secondary effects, such as jobs, notifications, access changes, or calculated charges.
• Warnings, execution blockers, and assumptions that could change the result.

"Meaningful" needs judgment. A raw database timestamp rarely helps an approver. A newly assigned owner, an expanded group membership, or a planned deletion absolutely does. Show the semantic result first and offer lower-level detail when a caller needs it.

A preview also needs to distinguish direct effects from derived effects. Suppose an agent lowers a team's storage quota. The direct change is one quota field. The derived result might freeze uploads for three existing projects. Burying that under a generic warning makes the operation look safer than it is. Put it in a separate effects array and name the cause.

Be equally precise about uncertainty. A preview can state that execution will query an external tax service or schedule work for later. It should not claim a final tax amount if the service has not resolved one. Use an assumption record that names the dependency and whether execution can proceed without it.

Validation must separate blockers from warnings

A preview should tell an agent exactly what prevents execution, what deserves review, and what merely provides context. Mixing those categories guarantees bad retries and approval fatigue.

A blocker means the service will refuse execution under the evaluated conditions. An agent should repair the input, obtain missing authority, or stop. A warning means execution can proceed, but a reasonable operator may want to inspect the consequence. Context gives information without implying danger.

Return structured errors, not prose that an agent has to parse. This shape is deliberately ordinary:

{
  "mode": "preview",
  "executable": false,
  "validation": [
    {
      "severity": "error",
      "code": "version_conflict",
      "path": "/if_match",
      "message": "Project prj_184 is at revision 73, not revision 71.",
      "blocks_execution": true,
      "repair": "Fetch the current project and create a new preview."
    },
    {
      "severity": "warning",
      "code": "member_count_change",
      "message": "The group will gain 42 members through nested groups.",
      "blocks_execution": false
    }
  ]
}

Stable codes let an agent choose a response. It can fetch a current revision after version_conflict; it cannot responsibly invent a fix after legal_hold_active. The message exists for the person reviewing the action. Keep both.

Do not label every surprising condition as a warning. A warning that always requires someone to change the request should be an error. Conversely, do not block execution because the API finds an unusual but permitted condition. Teams turn every warning into a blocker because they fear missing something, then agents submit previews that cannot ever complete without manual cleanup. The interface becomes performative.

The useful test is plain: if execution receives the same input against the same state, would it run? If yes, report a warning or context. If no, report an error. Keep authorization failures separate from domain validation. They explain different problems and need different remediation.

A dry run cannot write behind the caller's back

A preview must avoid durable external effects, including the ones developers dismiss as housekeeping. Creating a "temporary" row, reserving inventory, incrementing a sequence that users see, queuing a webhook, sending an email, or updating a last-access timestamp all violate the expectation that the request was safe to inspect.

This bug appears in mature services because the execution code grew around convenience. A create handler may allocate an identifier at the beginning, write a pending record before validation, and call an event publisher before the transaction commits. Someone later wraps it in if preview around the final insert. The preview looks harmless in a local test and still consumes identifiers, produces event traffic, or leaves debris in production.

Treat preview execution as a separate mode in the application service, not a condition in the controller alone. The mode can call shared parsing, authorization, policy, and planning functions. It must route writes and external sends through interfaces that either produce a proposed effect or fail the request.

A useful implementation boundary looks like this:

parse request
  -> authorize caller
  -> load consistent current state
  -> validate business rules
  -> build plan
  -> preview: return plan
  -> execute: apply plan in a transaction, then publish committed effects

The order matters. If your database supports transactions, build the plan from the same reads that guide execution. If a dependency cannot participate in the transaction, report its pending interaction as an explicit effect and design a compensating action for failures. Pretending that an external call is transactional does not make it so.

Audit records deserve a decision, too. You may want to record that a caller requested a preview. That is reasonable, but write that event to a clearly separate audit path and make sure it cannot trigger workflows built for completed changes. Do not put "previewed" beside "permission granted" and expect downstream consumers to infer the difference.

Test absence, not only output. Before and after a preview request, assert that relevant tables, outgoing queues, object storage, email test sinks, and downstream webhook receivers have not changed. Unit tests rarely catch this. An integration test around a disposable environment will.

HTTP semantics need an explicit contract

HTTP has no universal dry-run method, and pretending otherwise causes interoperability problems. RFC 9110 defines GET, HEAD, OPTIONS, and TRACE as safe methods in the sense that a client does not request state change. It does not say that a POST with a query parameter is safe, nor does it define dryRun as a standard request control.

That means an endpoint designer must make the mode visible in both request and response. A POST is often still appropriate because planning complex writes needs a request body and can require costly evaluation. The important part is that clients, logs, and humans can tell a preview from execution without guessing.

For a simple operation, an explicit body field is easy to read and difficult to lose:

POST /v1/projects/prj_184/memberships/plan
Content-Type: application/json

{
  "subject_id": "usr_92",
  "role": "admin",
  "if_match": "73"
}

A dedicated /plan endpoint works when planning has its own output, lifecycle, or permissions. It also avoids a recurring failure with query flags: a generated client omits the flag, a proxy ignores it in cache configuration, or a caller copies the URL incorrectly and performs the write. If you choose one endpoint with a mode field, reject missing or unknown values on operations where accidental execution would hurt.

Return a response type that cannot be mistaken for the executed resource. 201 Created with a resource-shaped body is a bad preview response even if you attach a preview: true field. Use 200 OK for an immediate plan, or 202 Accepted only when planning itself runs asynchronously. Include mode: "preview" in the response body, and set an explicit content type if your API uses typed media types.

Avoid caching previews unless you understand every input that affects them, including caller identity and authorization. The safest default is Cache-Control: no-store. A stale plan is not merely an old page. It can direct an agent toward a write that now affects a different set of resources.

Do not misuse OPTIONS for this job. RFC 9110 uses it to describe communication options, not to simulate a write with an arbitrary body. A service that overloads it will confuse libraries, security controls, and anyone who expects ordinary HTTP behavior.

The execution must prove that the plan is still current

A preview can become wrong in the interval before execution. Another user can edit the record, a scheduled job can run, an entitlement can expire, or the agent can alter the request after reading the response. This is a time-of-check to time-of-use problem, and a reassuring preview does not remove it.

Bind a plan to the evaluated request, the resource revisions it read, the caller identity, and a short expiry. The server can return a signed opaque plan_token, or it can retain the plan and return an identifier. Opaque tokens keep a client from treating the plan as editable authority. Stored plans make it easier to inspect large effects and revoke an approval. Either approach works if execution rechecks the right conditions.

A response might contain:

{
  "mode": "preview",
  "plan_id": "plan_7f4c",
  "expires_at": "2025-06-18T14:05:00Z",
  "request_digest": "sha256:...",
  "read_revisions": [
    {"resource": "projects/prj_184", "revision": "73"}
  ],
  "executable": true
}

At execution, the service must verify the caller, digest, expiry, and revisions. Then it must either apply the already approved plan atomically or regenerate the plan inside the write transaction and compare it to the approved one. If it cannot guarantee equivalence, it should reject the request with plan_stale and ask for a new preview.

Do not allow an agent to preview a request for one subject and execute the plan ID with a different subject in the body. Better yet, make execution accept only the plan ID and an expected revision, so there is no second mutable copy of the request for the server to reconcile.

Some changes cannot receive a meaningful guarantee. A plan to send a message may become inappropriate because the recipient's address changes a moment later. A plan to call a third-party service may depend on a price that moves before the call. Say so in the output, revalidate immediately before the irreversible action, and require a fresh decision when the difference matters.

Agent workflows need a deliberate stop before execution

An agent should treat a preview as evidence for a decision, not as permission to run the write automatically. The agent needs rules for when it can execute, when it should repair the request, and when it must present the plan to a person.

The most dependable workflow has four actions:

1. Submit the intended write in preview mode with an idempotency reference and expected resource revisions.
2. Stop if the response has blockers, then repair only the fields identified by the response or ask a person for missing intent.
3. Present the planned effects and warnings when the operation crosses the team's approval boundary.
4. Execute only the returned plan while it remains current, then record the execution result separately from the preview.

Approval should focus on consequence, not a raw JSON dump. A person deciding whether to grant access wants to see the principal, the role, the resources reached through group expansion, and the duration. They should not have to infer that impact from a request body full of IDs.

Do not make the agent preview every harmless action and ask for approval on every warning. That produces a queue of cards nobody reads. Define meaningful boundaries in the application: irreversible operations, changes to access, money, external communication, broad resource sets, and actions whose effects the service marks uncertain. The agent can execute small, well-understood changes within the authority you give it.

Sallyport can require a human decision for an agent's actual HTTP or SSH call, while the API's preview gives that decision concrete substance. The two controls solve different problems: one governs whether a process may act, and the other explains what the target service will do.

A failed bulk change shows why summaries are insufficient

Consider an agent asked to remove contractors from a production support group. It finds a filter that matches 37 accounts and submits a preview. The service returns count: 37, valid: true, and a generic note that inherited memberships may change. An operator approves because the requested outcome sounds routine.

Execution removes direct membership for those 37 accounts. Four of them retain access through nested groups. Six others lose a separate on-call permission because the service also removes a linked entitlement. A notification job tells all 37 people their access changed. The operator now has to determine which effects were intended, which were hidden, and whether the notification described the actual access state.

The preview was technically truthful in the narrowest sense. It did not promise that the filter identified only contractors. It was still a poor interface because it returned a count where the user needed a membership graph and an effect list.

A better response groups the result by consequence:

{
  "mode": "preview",
  "operation": "remove_group_members",
  "selected": 37,
  "effects": [
    {"type": "direct_membership_removed", "count": 37},
    {"type": "access_retained_via_nested_group", "subjects": ["usr_8", "usr_19", "usr_31", "usr_44"]},
    {"type": "on_call_entitlement_removed", "subjects": ["usr_2", "usr_7", "usr_11", "usr_24", "usr_29", "usr_35"]},
    {"type": "notification_queued", "count": 37}
  ],
  "validation": [
    {
      "severity": "warning",
      "code": "access_outcome_varies",
      "message": "Four selected subjects retain group-derived access."
    }
  ]
}

The right response may include a downloadable report or paginated detail for larger batches. The point is not to force a person to read thousands of rows. The point is to make exceptional and irreversible outcomes visible before the write.

This example also exposes a common bad recommendation: "use dry runs only for destructive actions." Teams repeat it because deletes feel dangerous and previews cost engineering time. But a permission grant, a configuration edit, or a notification can have a larger blast radius than a deletion. Choose preview support by consequence and reversibility, not by HTTP verb or database operation.

Tests must compare previewed effects with executed effects

A preview endpoint rots when tests only prove that it returns a 200 response. Its central promise is equivalence: when the state and request match, the reported effects must match execution.

Build paired tests. Set up a fixture, issue the preview, capture the normalized plan, reset the fixture, execute the same intent, and compare the execution journal with the predicted effect set. Ignore fields that cannot reasonably match, such as server timestamps or generated correlation IDs. Do not ignore created resources, changed values, published events, notifications, or outbound calls.

Property tests help with filters and bulk operations. Generate a collection of resources with mixed states, ask for a preview against a predicate, execute it in a fresh copy, and assert that the selected set and final state agree. These tests find the nasty cases where a planning query joins one table but the write query joins another.

Keep a test specifically for preview side effects. Use fake adapters for mail, webhooks, queues, and payment providers that fail the test if preview mode calls them. Then run at least one integration test against the real persistence layer, because an ORM flush or trigger can write even when your application code appears clean.

Finally, test staleness on purpose. Preview a change, mutate a resource through another request, then execute the old plan. The service should reject it. A system that applies the old plan because the diff "still looks close enough" will eventually overwrite somebody else's work.

Preview is an API capability, not an excuse to skip controls

Preview endpoints reduce surprise. They do not replace authorization, concurrency checks, transaction design, idempotency, audit trails, or review for operations that deserve review. A caller who lacks authority should not gain a detailed map of protected resources by probing previews. A caller who repeats an execution request should not create the same side effect twice because a plan token was valid.

Start with the write that has hurt your team most in rehearsal or production. List every direct and indirect effect, implement a plan that reports them, and make execution reject stale plans. Then write the paired test that proves the preview and execution agree. If you cannot state what a write will do before it runs, the agent is not the risky part of the system. The API is.