# Authorization evidence: prove AI agent actions completed An approval record tells you that a human allowed an agent to try something. It does not tell you whether the request reached the service, whether the remote host accepted the command, or whether the intended change happened. Treating approval as proof of completion creates audit trails that look reassuring right up to the moment an incident reviewer asks, "What actually happened?" AI agents make this distinction urgent because they can issue many real actions while working through an ordinary development task. A person may approve an agent process once, then the process creates a ticket, changes a deployment setting, queries a production API, or runs a remote command. Each action needs evidence with a different meaning. The human decision and the resulting outcome belong in the same investigation, but they are not interchangeable. ## An approval records permission, not completion Authorization evidence answers whether a recognized decision maker allowed a defined action at a given time. Execution evidence answers what the action gateway attempted and what the target reported back. An audit design that saves only the first answer has a large hole. Consider an agent that receives approval to call an API that creates an access token. The gateway sends the request, but the connection drops after the service processes it and before the caller gets a response. The approval remains valid. A log that says "approved" does not tell the reviewer whether the token exists. A log that claims "completed" because the gateway sent bytes is worse, because it turns uncertainty into a false fact. The same mistake appears with SSH. A person approves `deploy.sh`. The SSH client authenticates, the remote shell starts, and the network connection dies while the script runs. Did the script finish? Did it partially modify a system? The connection record cannot answer. You need the remote exit status, if the client received it, and a clear unknown-outcome record if it did not. Keep these questions separate: - Did a human or an approved control authorize this agent process? - What exact capability did the authorization cover? - Did the gateway send the action to the destination? - What result did the destination or transport return? - Can later reviewers verify that nobody edited the history? Many teams compress all five questions into a single event named `agent_action`. That is convenient for a dashboard and useless when facts diverge. Store related events, then show them together in the interface. NIST Special Publication 800-53 Rev. 5 makes a similar practical demand in control AU-3. Its audit-record content includes what happened, when it happened, where it happened, the source, the outcome, and the identity tied to the event. The useful part is not the checklist language. It is the insistence that outcome belongs in the record. A permission decision cannot fill that field. ## The two records need different fields A clean audit model uses one authorization event and one or more execution events for a single requested action. They share a correlation identifier, but each keeps the fields that support its own claim. The authorization record should identify the request before execution starts. Capture the calling process identity, the human decision, the credential reference, the intended channel, the destination, and the requested operation. Record the authorization scope precisely enough that a reviewer can see whether the later call stayed inside it. A useful shape looks like this: ```json { "event_type": "authorization.granted", "action_id": "act_01JX7K8N4Q", "session_id": "ses_01JX7JYQ2M", "time": "2025-03-08T14:21:18Z", "agent_process": { "pid": 4812, "code_signing_authority": "Example Development Team" }, "human_decision": { "method": "local_confirmation", "actor": "local_user" }, "requested_action": { "channel": "http", "credential_ref": "billing-api-prod", "method": "POST", "destination": "api.internal.example", "path": "/v1/refunds" } } ``` This record should not contain the API secret, SSH private key, authorization header, or raw request body by default. An audit store full of credentials becomes a second breach path. If reviewers need to distinguish requests with private content, retain a cryptographic digest of the protected portion and a short, redacted summary with documented rules. The execution record starts when the gateway attempts the action. It must say whether the action reached a protocol boundary and what came back. For HTTP, capture the method, destination, response status, transport error, response size, and bounded response classification. For SSH, capture the account, host identity, command representation, exit status, signal if any, and client-side connection result. ```json { "event_type": "execution.finished", "action_id": "act_01JX7K8N4Q", "time": "2025-03-08T14:21:20Z", "channel": "http", "attempt": 1, "delivery": "response_received", "result": { "http_status": 201, "response_bytes": 428, "response_digest": "sha256:..." } } ``` Do not write `success: true` as the only result field. Success changes meaning across protocols and products. HTTP 201 means the server reports a created resource. HTTP 202 means it accepted work for later processing. SSH exit status 0 means the remote command reported success, which may still mean a script ignored an internal failure. A typed outcome gives reviewers facts instead of a vague green label. ## The trusted action boundary must create the correlation The component that holds credentials and performs the action should generate the action ID. The agent may request work, but it cannot be the source of record for what it was allowed to do or what the gateway sent. This matters when an agent process is buggy, compromised, or simply confused by a long tool conversation. If it creates its own identifiers, it can attach a later outcome to an earlier request, omit inconvenient calls, or report an invented status. Even a well-behaved agent can lose context after a restart. The action boundary sees the request, credential selection, outbound attempt, and returned result in one place. Use an identifier that has no business meaning and never reuse it. A random or time-sortable unique ID works. Attach it to every local record. Where the protocol permits it, also carry it outward as a request identifier. That gives service operators a way to join their logs to the gateway record without exposing a credential. For an HTTP request, use a dedicated header that the target system agrees to log, such as `X-Action-ID`. Do not confuse that header with an idempotency key. An action ID helps investigation. An idempotency key tells a supporting server to recognize a duplicate mutation. A single value can sometimes play both roles, but only after the service owner confirms its semantics and retention period. ```text Authorization: action_id=act_01JX7K8N4Q Outbound request: POST /v1/refunds Request header: X-Action-ID: act_01JX7K8N4Q Response: 201 Created Execution record: action_id=act_01JX7K8N4Q, delivery=response_received ``` Do not rely on timestamps alone to correlate records. Clocks drift, requests overlap, and agents can make identical calls in the same second. Timestamps help reconstruct order. They do not prove parentage. A session ID also matters, but it answers a broader question: which agent process run made this request? Preserve both IDs. The session tells you which process received standing permission. The action ID tells you which particular operation got a particular result. ## HTTP status codes are evidence with limits An HTTP response gives stronger execution evidence than a local approval, but it still needs interpretation. Log the final status and enough context to explain it. Do not flatten every status outside the 200 range into failure. A `200 OK` or `201 Created` is an explicit server report. A `204 No Content` often means a successful operation with no response body. A `202 Accepted` is different: the target received the request for asynchronous processing, but the final work may fail later. Your execution record should say `accepted_for_async_processing`, then link a later status check or callback to the same action when the service supports that pattern. Redirects need care. If a gateway follows redirects, log the original destination and the final destination, including whether the credential injection rule applied at each hop. Forwarding an authorization header to an unexpected host is a credential leak, not an ordinary redirect. In practice, reject cross-host redirects unless an operator explicitly configured the target relationship. Client errors and server errors also carry useful facts. A `403` proves that the service denied the request. A `409` may show that a duplicate or conflicting state already exists. A `429` says the target refused work for now. A `500` says the server reported a fault, not that no change occurred. The response body may clarify the result, but store it only after redaction and size limits. Error responses often contain more operational detail than success responses. Transport failures need their own outcome values. Record distinctions such as `dns_failure`, `tls_validation_failure`, `connect_timeout`, `write_interrupted`, `response_timeout`, and `connection_reset`. These outcomes tell a reviewer where certainty ends. The dangerous category is an interrupted request after the gateway starts writing it. The service might have acted. Mark the state as `unknown_remote_outcome`; do not label it failed because the client lacks a response. If the call mutates state, the agent should pause and use a safe reconciliation method. That method might query the service by an idempotency key, read a resource using a request identifier, or ask a human to inspect the target. ## SSH needs remote process evidence SSH audit records often stop at "connected to host." That only proves that a client established an SSH session. It says nothing about the command's result, and it may not even identify the exact host that answered unless you record host key verification. For each SSH action, record the destination hostname, resolved address if available, verified host key fingerprint, requested account, authentication method reference, normalized command, and the final client result. If the remote command starts, capture its exit status and terminating signal. Treat standard output and standard error as sensitive operational data, not automatic log material. A normalized command preserves review value while reducing exposure. For example, instead of retaining a command that contains a secret argument, retain the executable, fixed options, sensitive argument positions, and a digest for protected values. ```text requested_command: /usr/local/bin/rotate-service-token --project payments --token [redacted] command_digest: sha256:... remote_exit_status: 0 remote_signal: null connection_result: clean_close ``` Exit status 0 is evidence from the remote command, not proof that every business effect occurred. A badly written shell script can run `curl`, ignore its error, and still exit 0. If you control the script, make it fail loudly and return nonzero when a required operation fails. If you do not control it, log the command outcome accurately and avoid claiming more. Interactive SSH sessions deserve skepticism. They create an enormous gap between "permission granted" and "commands executed." A bounded remote command produces better evidence than granting an agent a general interactive shell. If a task needs several commands, use a reviewed script with explicit exits, or create separate action records for each command. This is less glamorous than an agent typing freely into a terminal. It is far easier to investigate. Host verification must remain on. A record that says an agent ran a command on `build-01` means little if the client accepted an unverified host during a network attack or after a mistaken host replacement. Preserve the verified host key fingerprint in the execution event, then reviewers can distinguish the hostname label from the cryptographic identity. ## A timeout must end in uncertainty, not a retry storm The failure that catches teams most often starts with a mutating API call and ends with a timeout. The agent received permission. The gateway opened a connection and sent the request. The target completed the change but the response disappeared, or the target never received the final bytes. Both paths look similar to the caller. Imagine an agent creates a production incident ticket through an API. It sends: ```text POST /v1/incidents Idempotency-Key: inc_72f9c X-Action-ID: act_01JX7K8N4Q ``` The client waits, then records `response_timeout`. An approval-only audit trail says the person approved a ticket. A simplistic action log says ticket creation failed. Neither claim is safe. The correct execution record says that the gateway attempted the request and did not receive a response. It should retain the action ID and idempotency key reference. The agent then asks the service for the state associated with `inc_72f9c`, if the service exposes that lookup. If the service reports an existing ticket, log a reconciliation event that points to the original action. If it reports no record and its idempotency contract supports retry, retry once with the same idempotency key, not a new one. The sequence has three separate facts: 1. A person approved the original request. 2. The gateway could not confirm the initial outcome. 3. A later read or idempotent replay established the final state. Do not erase the timeout after reconciliation. It explains why the later action occurred. It also shows whether the team has a recurring transport issue that a polished success count would hide. SSH has an equivalent failure. A remote command may continue after the local client loses its session. Avoid automatic retries unless the command is demonstrably idempotent. Prefer a remote operation ID, a status file with controlled permissions, or a target API that reports state. If none exists, record the unknown outcome and require a human review. Duplicate deployment commands have caused enough damage without an agent retrying them at machine speed. ## Per-session approval and per-call approval answer different risks Per-session authorization establishes that a particular agent process may use approved action channels for the duration of its run. It reduces approval fatigue when a human expects a short, bounded sequence of low-impact work. It does not transform later calls into individually reviewed decisions. Per-call approval records a human decision for each use of a selected credential. Use it for operations where each destination, mutation, or remote command deserves fresh attention. The execution record still remains necessary. A person may approve a destructive API call and the service may reject it, partially process it, or time out. The distinction becomes sharp when reviewers ask, "Who approved this change?" A session record may answer, "This signed agent process had permission to call this channel." A per-call record can answer, "A person approved this exact request at this time." Neither answers, "Did it happen?" Only the resulting execution evidence can address that question. Approval prompts should show enough information for a human to make a meaningful decision: calling process identity, credential reference, destination, method or command, and whether the request changes state. A prompt that says only "Allow agent access?" records a click but captures almost no useful intent. Do not solve this with a giant policy language before you can produce coherent evidence. Teams often reach for rules because they want fewer prompts. Rules can limit actions, but they do not replace an audit model that distinguishes approval, attempt, response, and unknown outcome. First make the facts legible. Then decide where automation is safe. ## Hash chains protect history, not the truth of a claim An append-only, hash-chained log makes later tampering detectable. Each event includes or contributes to a digest that depends on earlier records. If someone alters an old authorization, removes a failed request, or reorders a sequence, verification fails unless they can rewrite the chain from the changed point onward and replace the trusted chain head. That property matters for agent activity because the most embarrassing records are often the ones somebody wants to remove: a denied action, a failed deployment, a request sent to the wrong service, or a session that continued longer than expected. A log that only administrators can quietly edit will not survive serious review. But hash chaining does not make a false entry true. If an untrusted agent supplies "HTTP 201" and the log faithfully chains that lie, the chain proves only that the lie persisted. The collector must observe the event at the action boundary. The gateway should create the execution result after it receives the response or detects the transport failure. NIST SP 800-92, Guide to Computer Security Log Management, warns that logs need protection across generation, transmission, storage, analysis, and disposal. The practical lesson is broader than centralizing text files. Preserve the raw event source, protect the sequence, control who can alter records, and make verification possible without granting broad access to the secrets that actions used. Sallyport projects agent runs and individual calls from one write-blind encrypted, hash-chained audit log. Its `sp audit verify` command verifies the chain offline over ciphertext without requiring the vault key. That design separates a reviewer who checks history integrity from a process that can use production credentials. A verification command should produce an unambiguous result and identify the checked range. For example: ```text $ sp audit verify verified: 1847 records chain: valid first_record: 2025-03-01T08:15:02Z last_record: 2025-03-08T14:21:20Z ``` A valid chain does not settle whether an external API fulfilled its promise. It does establish that the gateway's preserved account of the approval and received outcome has not changed undetected. ## Review the joins, not isolated event counts Audit review fails when teams count approvals, successful calls, and denied calls in separate charts but never inspect the joins between them. The useful review unit is an action timeline: request, authorization decision, execution attempts, result, and any reconciliation. Start with records that lack a matching partner. An authorization with no execution event might reflect a cancelled request, a gateway crash, or a logging fault. An execution event without a prior authorization may expose a bypass. An unknown remote outcome with no reconciliation is unfinished operational work, not a closed failure. Use a small set of queries or reports that force these questions: - Which approved mutating actions ended with an unknown remote outcome? - Which calls received a response outside the authorization scope? - Which SSH commands ended without an exit status? - Which per-session approvals produced actions after the expected task ended? - Which action IDs appear in local records but not in target service logs? The last query needs restraint. Absence from a target log may mean the target did not retain the request ID, a clock window is wrong, or another team owns the logs. Mark it as an investigation signal, not proof of wrongdoing. Sallyport's session journal and activity journal make this split practical: one view shows the agent run and its revocation state, while the other shows individual calls. Reviewers should move between them through the session and action identifiers rather than treat either journal as the whole story. Keep an explicit vocabulary for certainty. `authorized`, `attempted`, `response_received`, `remote_exit_received`, `denied`, `failed_before_send`, and `unknown_remote_outcome` are easier to defend than a single success flag. Once a team uses those terms consistently, the hard cases stop disappearing inside friendly green dashboards. ## Build evidence before granting broader agent access You do not need a complex control plane to start. Put credential use behind a trusted action boundary, create the action ID there, record the human decision separately from the result, and preserve unknown outcomes without rewriting them as failures. That gives incident responders a sequence they can test against service and host records. Then test the record under failure, not only on a happy path. Approve a request and cut the connection after the send. Run an SSH command that returns a nonzero exit. Force a DNS failure. Revoke an agent session and verify that later calls receive denial records. Check whether a reviewer can explain each event without opening a source file or asking the agent what it meant. An agent can be allowed to act without holding a credential. It cannot be allowed to redefine what happened. Keep the decision record, keep the execution result, and leave uncertainty visible when the network refuses to cooperate.