Agent API access testing before a production rollout

An agent earns production API access by behaving correctly when the request is denied, malformed, slow, or dangerous, not by producing one successful response in a sandbox. A friendly demo hides the failures that matter: an approval aimed at the wrong process, a token copied into an agent transcript, a retry loop that pounds a rate limit, or a journal that cannot explain who approved a destructive call.

Test the whole action path against a nonproduction destination before you hand an agent a production capability. That path includes the agent's request, authorization, credential injection, the remote API response, the agent's interpretation of failure, and an account of each call that survives the session. If one part is missing, you have tested an API client, not an autonomous actor.

A nonproduction endpoint must be separate in the ways that hurt

A useful test endpoint has separate credentials, separate data, and a damage boundary that you can explain in one sentence. Calling a production hostname with a query parameter that supposedly selects test mode does not meet that bar when the same token still reads customer records or can spend money.

Use a vendor sandbox when it gives you an isolated account and test credentials. Use a dedicated tenant when the service has no sandbox. If neither exists, put a small service under your control behind a different hostname and back it with disposable data. The point is not a label such as staging. The point is that a mistaken request cannot affect production users, production balances, or production secrets.

Make the endpoint prove that it is nonproduction. Return an obvious environment field in every successful response, and make destructive routes write only to a test ledger. A response that looks like this prevents an operator from mistaking a green test for a production change:

{
  "environment": "test",
  "request_id": "req_7f1a",
  "status": "accepted",
  "resource_id": "demo-order-184"
}

Keep test data realistic enough to exercise pagination, missing fields, permissions, and conflicts. A single perfect record teaches an agent bad habits. Seed a few records it may read, one record it may update, and one it must never access. You do not need a giant fixture set. You need enough variation to catch code that assumes every response is clean and complete.

Do not reuse a production bearer token in a test environment for convenience. I have seen teams call that a temporary shortcut, then leave it in place because every follow-up task seemed more urgent. A test identity should have a name, an owner, an expiry date, and permissions that match the specific test cases. If someone cannot say which test requires a permission, remove it.

The first run should test the authorization boundary, not the API

Before you validate business behavior, prove that an unrecognized agent process cannot act silently. Start a fresh agent process and make it attempt one harmless read against the test endpoint. The expected result is an authorization event before the API request goes out.

This catches a distinction that teams blur: user approval is not process approval. An operator may trust their own terminal while distrusting a plugin, a copied script, or an agent launched by a different application. The approval surface should tell the operator what executable authority is asking to act. A button labeled Allow without that context asks people to bless a mystery process.

For this test, capture four observations:

• The new process receives an approval request before the external call.
• The approval identifies the process in a way an operator can recognize.
• Approval lasts only for the intended run, not for every future process.
• Ending the process removes the session authorization.

Reject the request once before approving it. The rejection must leave the agent with a usable failure signal rather than an invented success. Good agent instructions say what to do after refusal: stop the operation, report that approval was denied, and do not hunt for a second route to the same endpoint.

Then restart the agent and repeat the harmless read. If the second process inherits the first process's permission, find out why. Cached authorization often looks efficient in a demo and becomes a quiet permission grant when an agent restarts after an update or when a different launcher invokes the same command.

Sallyport's session approval is on by default and presents the process code-signing authority on the first call from a new agent process. That is the right point to test human judgment, before a credentialed request reaches the outside service.

Credential injection needs a proof that the agent never held the secret

Credential injection passes only when the agent can request an action and cannot retrieve the credential used for it. Masking a token in console output is not protection. A token that entered an environment variable, a tool response, a prompt, a shell history entry, or a local file was available to the agent even if nobody happened to print it.

Set up a test credential that the remote service can identify without exposing its value. Many APIs provide a token label, client identifier, or audit field. If yours does not, create a test route that returns the credential identity it saw, not the credential itself. The result should prove which test identity authenticated the call.

For a bearer-token API, the wire-level request usually has this shape:

GET /v1/test/projects/demo HTTP/1.1
Host: api.test.example
Authorization: Bearer [injected outside the agent]
Accept: application/json

The bracketed text is documentation, not a value for the agent to fill in. An agent should supply the method, destination, and permitted request arguments. The credential handler adds the authorization header only after the approval decision. Basic authentication and custom header schemes need the same test, because they fail in different places when configuration is wrong.

Inspect the agent transcript, tool request history, shell environment exposed to the agent, and any files created during the run. You are looking for both literal token material and indirect leaks such as a request object with an authorization header. Redaction after the fact does not repair a design that gave the secret to the process.

Then rotate the test credential and run the same request. A successful second run proves the action path reads the current stored credential rather than a stale value embedded in an agent configuration. A failed run can also be useful if the error names authentication failure without printing the rejected secret.

Do not test with a token that can do more than the scenario requires. Read-only credentials expose enough to prove injection. Add a narrowly reversible write permission later for mutation tests. The person reviewing the test should never need access to the raw token to decide whether the test passed.

Approval friction should match the damage of the call

A session approval and approval for every use solve different problems. Session approval establishes that a particular agent run may use a bounded capability. Approval for every use forces a person to review each request made with a sensitive credential. Treating them as substitutes produces either a useless prompt storm or an unattended path to expensive mistakes.

Use a low-risk credential to test the session boundary. Put a per-use approval requirement on a credential that can create, delete, transfer, publish, or change access. Ask the agent to make two distinct test calls with that credential. You should see two decisions, and the second request should not ride on approval granted for the first.

The test has to include rejection. Approve the first test action and reject the second. Confirm these facts in the agent's report and the action record:

1. The first action reached the test service and returned its request identifier.
2. The rejected action never reached the test service.
3. The agent did not claim that it made the change.
4. The session remained usable for operations that did not require the rejected credential.

That fourth point catches an ugly failure mode. Some integrations treat a declined sensitive request as a reason to terminate every later operation. Others ignore the refusal and retry until a person approves by accident. Both behaviors make human control harder than it should be.

Approval fatigue is a design failure, but removing approval is not the usual cure. Reduce it by grouping work into a short-lived session, reducing the number of sensitive calls, or giving the agent a safer bulk operation. Do not solve noisy prompts by granting a broad permanent token to a process whose plan can change halfway through a task.

HTTP status codes should drive agent behavior

An agent needs explicit behavior for failure classes because HTTP success and task success are not the same thing. RFC 9110 defines the semantics of HTTP response status codes, including that a 401 response indicates missing or invalid authentication credentials and a 403 response means the server understood the request but refuses to fulfill it. Treat those responses differently. Retrying either one with the same request normally adds noise, not progress.

Build a failure table before the rollout and exercise each row against the test endpoint. Keep the required actions narrow enough that a reviewer can tell whether the agent followed them.

A 400 response deserves more attention than it gets. It often exposes a mismatch between an agent's tool schema and the remote API's actual contract. Have the test server return field-level validation errors, then check that the agent reports the bad argument without inventing a replacement value. An agent that guesses at fields can turn a harmless validation error into a request against the wrong account.

Test transport failure separately from an HTTP 503. Disconnect the test service or point a controlled test request at an unreachable address. The agent should distinguish no response from a server response. That distinction matters when the operation may have reached the service but the response was lost. Retrying a create operation after an ambiguous timeout can create duplicates.

Use idempotency identifiers where the API supports them. If it does not, make the agent read for an existing result before repeating a potentially mutating call. Saying retry three times is not a recovery plan when each retry may charge a card, create a user, or send a message.

A failed rollout often starts with a harmless retry loop

A common failure begins with an agent asked to create a test resource, then verify it. The create request succeeds at the service, but a network interruption hides the response. The agent sees an error, retries the create call, and receives a second successful result. It then fetches one resource by a guessed name and reports success. The operator now has duplicate changes and no clear account of which request caused which one.

You can reproduce this without risking production. Make a test route accept a create request, store the test object, then deliberately close the connection before returning the response. Run the agent with a fixed request identifier. The expected safe behavior is to query the test service for that identifier before repeating the create. If the agent cannot do that, it should stop and report an ambiguous outcome.

A minimal test service contract can make the check concrete:

POST /v1/test/jobs
{
  "request_id": "rollout-042",
  "name": "reconcile-demo"
}

GET /v1/test/jobs?request_id=rollout-042
{
  "items": [
    {"id": "job_128", "request_id": "rollout-042", "state": "queued"}
  ]
}

This is also where you find prompts that ask the agent to keep trying until it works. That instruction sounds sensible to someone watching a human operator. It is unsafe for an actor that can make calls faster than anyone notices. Replace it with a bounded retry rule, a duplicate check, and a condition that requires human review.

OWASP's API Security Top 10 calls out unrestricted resource consumption and broken object level authorization. Both show up in agent rollouts as ordinary behavior mistakes: a loop that ignores a limit, and an agent that substitutes a nearby object identifier after the requested one fails. Your test needs a forbidden object and a rate-limited route because a happy-path fixture will never expose either habit.

Records must explain the decision and the external effect

A call record must let someone reconstruct what happened without reconstructing the agent's entire private reasoning. Save the facts that establish authority and effect: which session acted, which process asked, what destination and method it used, which credential reference applied, whether a person approved it, when it happened, and what result came back.

Do not put raw request bodies into every record by reflex. Some payloads contain customer data, tokens from third parties, or content the agent was instructed to process. Store a safe summary or selected fields when that meets the investigation need. The remote service's request identifier is especially useful because it joins your local record to the service's own audit trail.

Separate the run record from the call record. A run record answers whether a particular agent process was allowed to operate and whether a person later revoked it. A call record answers what happened on each external action. Folding both into a chat transcript loses the structure you need when one run makes many requests.

Sallyport projects session and activity journals from one encrypted, hash-chained audit log. Its sp audit verify command can check that chain offline over ciphertext without a vault key, which makes it practical to test record integrity separately from access to secrets.

Run verification after a normal test, then copy the encrypted audit file to a test location and alter bytes in the copy. The command should report a failure for the altered copy while the original verifies. Do this with a disposable copy only. The exercise teaches the review team what a valid report looks like before they need it during a production investigation.

Tamper evidence does not mean every operator can read every detail, and it does not replace remote API logs. It answers a narrower question: did the local sequence of records remain intact? Keep the test endpoint's request IDs and timestamps so an investigator can compare both sides.

Revocation must stop the next call, not merely close a window

Test revocation while the agent is still running. Approve a session, make one harmless request, revoke the session, and ask the same process to make another harmless request. The second request must fail before it reaches the nonproduction endpoint. If it succeeds because an existing connection or cached credential survives, that is a production blocker.

Then test the vault gate separately. Lock the credential store and attempt the request again. A locked vault should deny every action, including a request that an operator previously approved for the session. This is a stronger control than revoking one run because it halts all action paths that depend on the vault.

Watch the endpoint during both tests. Do not accept an agent-side message that says access was denied as proof. The server-side request log must show that no second request arrived. This simple cross-check catches integrations that report an approval failure after they have already dispatched the HTTP request.

If your agent can perform SSH work as well as HTTP calls, repeat the test against a disposable host. Use a nonprivileged account and a command with an unambiguous result, such as creating a file in a temporary directory. Revocation must prevent a fresh SSH command just as it prevents an API request. A gateway that treats the two channels differently creates a blind spot where operators assume control still exists.

Promotion needs an evidence packet, not confidence from a demo

Move to production only when a reviewer can inspect a compact test record and answer whether the agent stayed inside its intended authority. The packet should contain the test identity and permissions, the endpoint boundary, expected approval behavior, representative success and failure results, the record verification result, and the observed revocation result.

Do not promote every test permission with the agent. Create a production credential separately and begin with the smallest set of operations that supports the first real task. Assign an operator who can approve or revoke runs, and write down which response states require the agent to stop. If the team cannot name that person, they have delegated operational control to chance.

Run the first production task with approvals enabled and inspect its records immediately afterward. Compare the actual request count, destinations, and outcomes to the test run. If the agent contacts an unplanned endpoint, asks for a broader credential, or retries differently under production data, stop the rollout and bring that behavior back to the nonproduction endpoint.

The right first production rollout is deliberately boring. The agent makes a small number of expected calls, a person can stop it, the credential never enters the agent context, and each external effect has a record that matches the service's own request ID. That is enough proof to expand carefully. A polished demo is not.