Agent audit logs: session records and call records

Agent audit logs fail when they force two different questions into one record. During an incident, you need to establish both who held authority to act and exactly what action reached the outside world. A record of the run cannot substitute for a record of the call, and a request list cannot explain why that process had permission.

Teams often keep only one of these views because one feels sufficient during a demo. Then an agent opens a ticket, updates a production setting, or runs an SSH command, and the investigation collapses into timestamp guessing. That is avoidable. Keep a session journal for authority and lifecycle. Keep an activity journal for each attempted external action. Join them with an identifier that neither humans nor agents get to improvise.

A run record answers who held authority

A session record should answer whether a specific agent process had permission to act, for what period, and under what human decision. It is the record you use when someone asks, "Which agent instance did this, and why did we let it?"

A useful session record starts when a process first asks for authority, not when a user opens an editor and not when a project directory appears. Agent names are weak evidence. Two processes can call themselves the same thing, and a malicious binary can borrow a familiar name. Capture the process identity available to the host, its code-signing authority where the operating system provides one, the parent process where useful, a generated session ID, start and end times, and the authorization outcome.

The session record also needs lifecycle events. Approval is an event. Revocation is an event. Process exit is an event. Locking the credential store, if that blocks activity, is an event worth correlating. Without these boundaries, an investigator cannot tell whether a call happened inside an approved period or after a supposed shutdown.

Do not turn a session record into a diary of every model thought, terminal line, and file edit. That creates a pile of sensitive material and still misses the boundary that matters: an agent process received authority to ask a gateway to take external actions. Capture the evidence that establishes that boundary cleanly.

Consider this simplified session record:

{
  "type": "session.authorized",
  "session_id": "ses_7f4c2",
  "observed_at": "2025-04-18T14:03:11Z",
  "process": {
    "pid": 8124,
    "signing_authority": "Example Development Team",
    "parent_pid": 8090
  },
  "decision": "approved",
  "approved_by": "local_operator"
}

The record does not need an employee name to be useful. On a shared workstation, local_operator may be the honest level of attribution. Pretending you know more produces confident fiction. If your environment can bind approval to an authenticated person, record that binding and how it was established.

A session record should never claim that a whole run was safe. Approval grants authority; it does not pre-approve every effect a process may create. That distinction sounds picky until a coding agent keeps a session open for hours and makes hundreds of calls under one approval.

A call record answers what touched the outside world

A call record should describe one attempted operation and its outcome. It is the evidence you need when someone asks, "Did the agent send this request, to which target, with what result?"

Record attempts, not just successes. A denied request can show an agent probing for a credential. A failed request can show a misspelled host, an expired secret, or an unsafe command that the remote side rejected. A cancelled request can explain why a session appears to have stopped in the middle of a deployment. Success-only logs tell a flattering but incomplete story.

For an HTTP action, retain the destination identity, HTTP method, path or controlled path representation, credential reference rather than credential value, request time, completion time, status outcome, session ID, and a call ID. For SSH, retain the intended host identity, command or a safe representation of the command, connection result, remote exit status where available, session ID, and call ID.

You must decide carefully what request data to retain. Logging full headers and bodies by habit is an incident waiting to happen. Authorization headers, cookies, signed URLs, access tokens, private customer data, and passwords often hide there. A good record preserves the operation's meaning while redacting secret material. For example, POST /v1/users/123/disable may be enough to reconstruct an administrative change; copying the entire JSON body may expose far more than the investigation needs.

Use target identity, not just a raw URL string. https://api.example.test and https://api.example.test:443 may represent the same destination, while a lookalike host may differ by one character. For SSH, record the host identity used for verification where your system can obtain it. A hostname alone does not settle whether the connection reached the expected machine.

This paired set shows the separation:

{
  "type": "call.completed",
  "call_id": "call_b91d",
  "session_id": "ses_7f4c2",
  "observed_at": "2025-04-18T14:09:27Z",
  "channel": "http",
  "operation": "POST",
  "target": "api.example.test/v1/deployments/42/cancel",
  "credential_ref": "deployment-service",
  "authorization": "session_approved",
  "outcome": "completed",
  "response_status": 202
}

The session ID says whose authority the call used. The target and outcome say what occurred. If you merge them into a single vague event such as agent performed task, you have recorded neither answer well.

Approval and execution are separate facts

Teams often confuse an approval event with evidence that a request ran. They are separate facts, and an audit trail must preserve both.

An operator may approve a new agent process, then walk away. The process may make no calls because the task completes locally. It may make one failed API request. It may make fifty successful requests. The approval record stays the same in each case. Only individual call records show the effect.

The reverse confusion also happens. Someone sees an outbound request in a network log and assumes an approved agent sent it. A network log can establish a connection or a request fragment, depending on where it was captured. It cannot usually show the approval decision, the actual process identity, or whether a gateway injected a credential on behalf of the agent. Do not force it to answer questions it was never designed to answer.

NIST Special Publication 800-92, Guide to Computer Security Log Management, distinguishes the event source, logging infrastructure, and analysis process. Its practical lesson for agent systems is plain: collect the event at the layer that knows the fact. The authorization layer knows whether a session received authority. The action gateway knows which operation it attempted with a protected credential. A firewall knows traffic it observed. Each record has a different evidentiary scope.

A precise authorization state on each call helps. session_approved means the session had a standing approval. per_call_approved means an operator approved this use. denied_locked means the credential vault rejected the attempt while locked. denied_user means an operator rejected it. These are not decorative labels. They tell an investigator whether the operation reached the action executor and whether human intervention stopped it.

Do not label every successful call as "approved." That word conceals the difference between permission granted at session start and permission granted at the moment of use. When an incident reviewer asks whether anyone approved the deletion, the record should answer in one reading.

Per-call approval has a place, but treat it as a narrow control. Put it on credentials where each use carries high consequence or where the target changes state in ways an operator should see. If you require a human click for routine reads and harmless build calls, people learn to approve a blur of prompts. The approval still exists on paper, but it no longer means informed consent.

A failure timeline exposes gaps that summaries hide

A single incident timeline makes the value of two views obvious. Suppose an autonomous coding agent receives a task to clean up stale preview environments. The operator approves its process for the session. The agent discovers an old deployment API credential and asks the gateway to make calls with it.

At 14:03, the session journal records an approved process and assigns ses_7f4c2. At 14:07, the activity journal records a GET request that lists deployments. At 14:09, it records the cancellation call shown above. At 14:10, a second cancellation attempt receives a 403 response. At 14:12, the operator revokes the session after noticing the agent selected the wrong environment group.

Now imagine the records you kept were only session records. You could say a process was approved and later revoked. You could not establish whether it cancelled one environment, several, or none. You could not distinguish a blocked second attempt from a successful one. You would need to ask the deployment service, and its log retention or request detail may not meet your needs.

Now imagine you kept only call records. You could see two cancellation requests. You could not establish which local process initiated them, whether an operator approved that process, whether the approval was still valid at the time, or whether the reviewer acted promptly after revocation.

The timeline should preserve ordering without pretending that wall-clock time is perfect. Machines drift. Remote services report their own timestamps. Write a gateway timestamp when you observe the request and, where relevant, separately retain a remote result timestamp. Use an ordering sequence inside the audit log if you have one. Do not declare causality merely because two events share a second on a clock.

The awkward question is whether a revoked session can have calls complete afterward. It can, depending on when revocation reaches the executor and whether a request already left the machine. Your records should make that visible. Log the revocation time, then log any call completion that follows, with the call's start time and completion time. A system that simply deletes a session makes this analysis impossible.

Correlation IDs need strict ownership

A session ID only works when the gateway assigns it and controls it. Do not let the agent supply a session identifier and treat it as security evidence.

Agents can carry arbitrary text across tool calls. They can reuse a stale ID after a restart, typo an ID, or deliberately claim another session's ID if an interface permits it. The gateway must derive the association from the authenticated local connection or process relationship, then attach the session ID to every call itself.

Call IDs need similar treatment. Generate them at the action boundary before the request leaves. If an HTTP request retries, record whether the retry is a new attempt linked to the original call or part of the same call with several transport attempts. Either model can work. Mixing them destroys count accuracy during an outage.

Use a small, consistent correlation model:

• A session ID groups authority and lifecycle events for one agent process.
• A call ID identifies one requested external operation.
• An attempt ID identifies a transport attempt when retries matter.
• A credential reference identifies the configured secret without exposing its value.
• A target reference identifies the host, service, or command destination.

Do not make every identifier globally meaningful to a human. Their job is to join records reliably. Human-readable labels can sit alongside them, but labels change, collide, and invite casual editing.

For concurrent agents, correlation prevents a familiar failure. An engineer sees a destructive request at 16:21, finds a terminal transcript from an agent at 16:21, and assumes they match. Meanwhile another agent process ran under the same account. The session ID on the action record removes the guesswork. If no stable join exists, state that limitation in the incident report rather than filling the gap with confidence.

The audit log must show rejection as well as use

Denied actions often matter more than completed actions because they reveal what an agent tried to do before a control stopped it. Record them with enough context to explain the decision, but avoid making the rejection log a fresh source of secrets.

A locked credential vault should deny every action that requires it. The resulting call record should say the action was denied before external execution, identify the session and target requested, and identify the reason class. It should not contain a fake token, a partial private key, or a copied authorization header.

Per-call rejection deserves the same care. If an operator rejects an SSH command, record the command representation, target, session, decision time, and decision result. The absence of a remote exit status then has meaning: the executor never launched the command. This is different from a command that launched and returned a nonzero status.

Sallyport uses three fixed controls: an absolute vault gate, per-session authorization by default, and optional per-call approval for an individual credential. That bounded model makes audit interpretation cleaner because each record can identify which decision stopped or allowed the action.

Avoid the fashionable answer of a giant policy language for every agent action. It is popular because it promises complete automation. In practice, a policy engine adds a second program that teams must review, test, update, and explain during an incident. If you need policies for network or service governance, use them at the relevant layer. Do not pretend an unreadable rule set removes the need for clear session and call evidence.

A record should distinguish these outcomes:

• The agent never had an authorized session.
• The session had authority, but the vault was locked.
• The gateway requested a per-call decision and the operator rejected it.
• The gateway executed the action and the remote target rejected or failed it.
• The gateway executed the action and received a successful result.

Those cases require different follow-up. A denied session request may point to an untrusted process. A remote 403 may point to a credential scope issue. A successful but unwanted call may require reviewing the task instruction, the session approval decision, and the target credential's allowed use.

Tamper evidence protects the record after the incident

Ordinary application logs are easy to edit, truncate, or replace after someone gains control of the host. That does not make them useless, but it limits what they can prove. A hash-chained audit log makes later modification evident when a verifier checks the chain against the records it received.

The distinction matters. A hash chain can show that an entry changed or disappeared from the middle of a preserved chain. It cannot prove the system recorded every event that should have existed. It cannot rescue a host that was already compromised before it created the event. It cannot tell you whether an operator understood an approval card. Claims beyond that are security theater.

RFC 5848, Signed Syslog Messages, addresses a related problem: log messages can lose integrity and origin assurance as they travel through systems. Its lesson still applies when you use a local encrypted journal rather than syslog. Protect logs near the point where the event occurs, retain ordering evidence, and verify rather than merely trusting a pretty interface.

Sallyport projects both its Sessions journal and Activity journal from one write-blind encrypted, hash-chained audit log. Its sp audit verify command verifies the chain offline over ciphertext without needing the vault key, which is useful when the reviewer should validate record integrity without receiving the secrets used for the actions.

Run verification before you filter, export, or annotate records for an incident. Preserve the original encrypted evidence first. Then make a working copy for analysis. If verification fails, record the failure and the exact artifact examined. Do not quietly continue with a cleaned-up export, because the integrity question has become part of the incident.

Tamper evidence also changes operational discipline. If your team knows a later edit will be visible, people stop treating the audit journal as a convenient place to rewrite history after a bad deployment. That alone does not prevent mistakes. It does preserve the evidence needed to learn from them.

Retention needs boundaries, not indiscriminate capture

Keep session and call records long enough to investigate delayed discovery, credential misuse, and access reviews, but do not retain every payload forever because storage is cheap. The most dangerous logs are often the ones nobody classified.

Start with the incident questions your team must answer. How long after an agent run might a service owner notice an unwanted change? How long do you need to trace a credential use after a staff departure? Which regulations or contracts impose retention requirements? Those answers set the period. They do not justify collecting raw prompts, full responses, or secrets that you do not need.

Separate operational visibility from forensic preservation. Operators may need a concise current view of active sessions and recent calls. Investigators may need the complete immutable sequence, including denials and timing details. Giving every developer unrestricted access to the latter turns the audit trail into another sensitive dataset.

Use role boundaries around review, but do not use access control as a reason to hide material from the people responsible for incidents. A service owner may need to know that a call changed their service. They do not need a bearer token or unrelated customer payload to learn it.

When a record points to sensitive content held elsewhere, store a controlled reference and retrieval procedure. For example, retain a request ID that the destination service can use to locate a protected payload under its own access rules. This keeps the action journal useful without duplicating sensitive business data into every audit system.

Deletion needs its own record. If retention expires a batch of entries, record the retention event, scope, and authority that caused removal before deleting the data. Without that, a later verifier cannot distinguish authorized expiry from unexplained absence. Keep the retention policy understandable enough that an incident reviewer can apply it without consulting the person who wrote the original system.

Build the incident view by joining outward from the call

During an active investigation, begin with the suspicious call and work outward. It is usually the most concrete evidence: a target, operation, time, and result. Use its session ID to retrieve the authority record, then inspect nearby calls in that session and the session's revocation or exit event.

Use this sequence:

1. Preserve and verify the original audit records before editing or exporting them.
2. Locate the call record by destination, call ID, operation, or incident time window.
3. Retrieve the linked session record and confirm process identity, approval time, and lifecycle state.
4. Review every call in that session around the event, including denials and retries.
5. Compare the action timeline with the destination service's own records, then document gaps rather than guessing through them.

This method catches both the obvious error and the quieter one. The obvious error is a call that should never have happened. The quieter error is a session that remained authorized after the intended task ended, or a retry that repeated an action after a service timeout.

Do not rotate every credential first because it feels decisive. If the vault kept credentials away from the agent and the record shows a gateway-issued call to one known target, broad rotation may create needless disruption. Revoke active sessions immediately when ongoing misuse is plausible. Then use the call records to decide which credential, target, or access scope needs action.

Two views do create more records. They also remove the most expensive sentence in an incident report: "We cannot determine whether the approved agent actually made this change." Make authority visible at the session boundary and effects visible at the call boundary. Anything less leaves your team reconstructing a security event from scraps.