# What do force quit tests prove about an agent gateway?

A gateway that holds credentials for an AI agent has to stay safe when its process disappears at the worst possible instant. A happy-path test can show that an approval appears, a request succeeds, and an audit record exists afterward. It cannot show whether a credential escaped during an interrupted request or whether the audit trail lies about work that never completed.

Force quit tests expose the gaps between statements such as "request started," "credential was attached," "remote system received it," and "the record is durable." Those statements are separate facts. Treating them as one event is how teams ship an action gateway that looks controlled until a crash turns an ordinary retry into a duplicate deployment or an untraceable outbound call.

For a macOS gateway, distinguish a normal quit from process death. Apple documents normal app termination as a lifecycle path where the app can save state and run termination handling. A forced termination may not grant that work at all. Apple also identifies SIGKILL as a termination type that can result when a user force quits an app. Your test plan must assume cleanup handlers, deferred writes, and best-effort telemetry do not run.

The useful question is not "does the app restart?" The useful question is: at every interruption point, can you state exactly which side effects are possible, which are impossible, and what evidence survives?

## A force quit must cut through cleanup

A force quit test is valid only when it removes the gateway's opportunity to tidy up. If the test calls a polite shutdown function, waits for queues to drain, flushes logs, and then exits, you have tested orderly termination. That matters, but it does not map the failure case that keeps security engineers awake.

Use two termination modes and label them accurately:

- A normal terminate request tests cancellation handling, orderly file closure, and how the UI reports an interrupted session.
- A hard kill tests the state of memory, file buffers, open sockets, and in-flight work when no application cleanup runs.

On macOS, a test harness can use `kill -TERM` for the first case and `kill -KILL` for the second. `SIGTERM` gives a process the chance to handle termination. `SIGKILL` does not. Do not call both cases "force quit" in the result table, because they answer different questions.

```sh
# Find the gateway process you started for the test.
pgrep -fl Sallyport

# Graceful termination case.
kill -TERM 48192

# Abrupt termination case.
kill -KILL 48192

# Confirm the process is gone.
ps -p 48192
# PID TTY           TIME CMD
# 48192 ttys003    0:00.42 gateway-test
# After SIGKILL, ps should print no process row.
```

The process ID is not the test identifier. Give every test run a run ID, an action ID, and a remote request ID. Put those IDs in the fixture's records and the gateway's records. Without shared identifiers, you will end up comparing timestamps and guessing whether a remote POST belonged to the run you killed.

Do not test against a production endpoint, even if you believe the call is harmless. The purpose of this work is to create ambiguous outcomes deliberately. Build a receiver that can pause before accepting a body, after reading headers, after recording the body, or before releasing its response. A real third-party API will not reliably give you those boundaries, and it may impose retries or caching you did not ask for.

## The request needs explicit interruption points

You cannot kill a process "during credential injection" unless the implementation defines what that means. Add observable hold points around the action pipeline. These are test controls, not a policy language and not a production authorization mechanism.

For an HTTP action, use a sequence like this:

1. The gateway accepts an authorized action description and assigns an action ID.
2. It writes an intent record, if the design requires one before network work.
3. It resolves the credential inside the protected process.
4. It constructs the outbound request and injects the credential.
5. It opens the connection and writes the request.
6. It receives enough of the response to classify the remote outcome.
7. It writes a completion or unknown-outcome record, then releases a result to the agent.

Install a pause at the boundary after each numbered operation. The pause must be observable from a separate process. A test-only local socket, a named pipe, or a file descriptor controlled by the harness works. A sleep call does not work well because timing drift turns a boundary test into a race.

A compact fixture protocol can look like this:

```json
{
  "action_id": "fq-2026-07-22-017",
  "hold_after": "request_headers_built",
  "release": false
}
```

The gateway pauses after creating headers but before any connection attempt. The harness confirms that state, kills the process, then asks the receiver whether it saw a connection. The expected outcome is zero inbound requests and zero credential-bearing headers. If the receiver saw a request at this point, your checkpoint sits too late or a background task crossed the boundary without being accounted for.

Make the checkpoints narrow. "Before network I/O" is too broad if DNS resolution, connection creation, TLS negotiation, and body transmission run in different tasks. You do not need a checkpoint inside every library call, but you do need enough structure to state whether the remote server could have received credentials or a state-changing body.

For SSH, the same idea applies with different evidence. Hold before `sp-ssh` receives a credential-backed connection request, after the helper starts but before authentication, after authentication but before the command begins, and after the remote command returns but before local completion is committed. Record remote command start and exit in a disposable test host. Do not infer command execution from an open TCP connection.

## Killing before injection must leave no remote trace

The earliest failure case has the cleanest expected result: if the gateway dies before it has attached a credential or started transport, no external service should see anything from that action.

That sounds obvious, but implementations often blur preparation and dispatch. A client library might begin a connection while another task fetches or formats a header. A retry wrapper might allocate an outbound request before the code that writes the intended journal record. A metrics callback might write an action description containing a URL query parameter that your normal redaction path never sees.

Test four pre-dispatch cuts:

- After authorization but before any credential lookup.
- After credential lookup but before request construction.
- After request construction but before a socket connection begins.
- After connection setup begins but before any credential-bearing bytes leave the process.

Each cut has a different assertion. Before credential lookup, inspect memory-safe diagnostics and logs for the absence of values or placeholders that could later be substituted. After lookup, inspect the same outputs and verify the secret never escaped the process boundary. Before connection, the receiver should have no connection record. During connection setup, the receiver may see a failed handshake or connection attempt, but it must not receive an authorization header, basic-auth field, custom credential header, or SSH authentication attempt.

This distinction matters because a connection attempt is not an authenticated action. Do not report "no action occurred" if the remote edge recorded an attempted connection. Report the narrow truth: no credential was sent and no application request was received. Security records become less useful when they smooth away evidence that operators will later need during incident review.

Sallyport's design makes this boundary worth testing directly: the agent should never hold a plaintext credential, while the app performs the credential-backed HTTP or SSH action and returns a result. The test is not complete because the agent lacks the secret. It is complete when a killed gateway also cannot turn a half-prepared action into a secret-bearing outbound request.

## Injection is a local event, not proof of dispatch

Credential injection is where teams make the most damaging bookkeeping mistake. They log "credential used" when the code builds a header or hands an identity to an SSH library. That event tells you the gateway prepared to authenticate. It does not prove that any peer received it.

Keep at least four local states separate:

| State | What you can honestly claim | What you cannot claim |
|---|---|---|
| Credential resolved | The protected process read a credential for this action | A peer received it |
| Request assembled | The process created a credential-bearing request in memory | A connection opened |
| Transport write started | The process attempted to send bytes | The remote application processed them |
| Remote outcome observed | The process received evidence from the remote side | The completion record is durable |

Run a kill after the credential is resolved and another after the request object exists but before the transport write starts. In both cases, the agent must receive no secret, and the remote fixture must record no credential-bearing request. The action journal should show either an interrupted preparation state or no persisted record, depending on when your durability boundary sits. Do not manufacture a completed action just to make the journal look tidy.

Then kill at the first point where the transport library can write. This is the awkward test. The local process may have called a write function, but the kernel, proxy, TLS layer, or remote service may not have received the complete request. Your expected result should be `outcome=unknown` unless the receiver has positive evidence that it did or did not receive the request.

Avoid logging full request headers to make this test easier. That is a shortcut that creates a second secret path. Instead, make the fixture calculate a nonreversible test marker from the received credential value and store only that marker. The gateway can store a credential reference or an internal credential identifier. During the test, compare identifiers and markers without printing the credential itself.

A useful fixture record has this shape:

```json
{
  "action_id": "fq-2026-07-22-017",
  "connection_seen": true,
  "headers_complete": true,
  "credential_marker": "sha256:7b8c...",
  "body_complete": false,
  "response_sent": false
}
```

Use a test-only credential with no value outside the fixture. Even then, keep it out of screenshots, shell history, and ordinary logs. A disposable secret is still a secret-shaped object, and test habits have a way of reaching production code.

## Network I/O creates an honest unknown state

Once a state-changing request can leave the machine, local certainty ends before the remote system necessarily answers. This is the case that must shape retry behavior, UI wording, and audit semantics.

Consider a POST that asks a deployment service to start a release. The gateway writes the full request. The service persists the deployment job. Before the response reaches the gateway, you kill the gateway process. On restart, there are three possible realities:

1. The service never received the request.
2. The service received the request but rejected it.
3. The service accepted the request and created a deployment.

The local gateway may not know which reality occurred. A record marked `failed` is false in case three. A record marked `succeeded` is false in cases one and two. Mark it `unknown` and preserve the action ID, remote request ID, destination, method, and the point at which local observation stopped.

This is also where blind retries cause damage. Teams love automatic retry because transient network failure is common and successful demonstrations look smooth. For a GET, retry is usually acceptable if it does not cause server-side accounting or rate-limit trouble. For a POST, PATCH, remote command, or write-capable API call, retry requires an idempotency mechanism or later reconciliation.

Use the remote system's idempotency feature when it has one. Supply an action-specific idempotency token, not a token reused for the entire agent session. If the remote API does not support it, create a remote lookup path that can answer whether the action ID was accepted before you retry. If neither exists, ask a human to decide. That is less elegant than automatic recovery and far safer than double execution.

Test this exact sequence with your receiver:

```text
Gateway writes intent record
Gateway sends POST with action ID fq-2026-07-22-017
Receiver stores action ID and body
Receiver delays its HTTP response
Harness kills gateway with SIGKILL
Gateway restarts
Agent requests retry
Gateway checks receiver for action ID before another POST
```

The result should show one stored remote action and one local record with an interrupted or unknown outcome that later reconciles to the remote state. If the second POST appears before the reconciliation check, the test has found a retry bug. If the journal overwrites the uncertainty and leaves only a clean success record, it has found an audit bug.

Do not depend on TCP close events as proof that a remote application did not act. A socket can close after the peer has already passed the request to its application code. The receiver's durable action record is the evidence that matters.

## Audit commit must have a defined durability promise

An audit log does not become trustworthy because it contains a hash chain. A hash chain can reveal alteration or removal within the surviving sequence. It cannot recover an event that never reached durable storage before the process died.

Write down the promise in one sentence. For example: "Before a gateway begins a state-changing external action, it durably records the action intent; after it observes an outcome, it durably records that outcome before returning it to the agent." Then test the two verbs, "begins" and "durably," instead of treating a timestamped in-memory object as a record.

POSIX defines `fsync()` as a request to transfer a file's data to associated storage and says it does not return until that action completes or reports an error. The standard also warns that storage guarantees depend on the implementation and configuration. That warning is not an excuse to skip the call. It means you must be precise about what your software can promise and test the recovery behavior it controls.

For a hash-chained encrypted log, test at least these cuts:

- Kill after writing a proposed entry but before its durability barrier.
- Kill after the durability barrier but before the in-memory index updates.
- Kill after a completion entry is durable but before the agent receives its response.
- Kill during compaction, rotation, or any journal projection process.

After each restart, run the offline verification command and inspect both the raw audit sequence and the user-facing projections. Sallyport exposes `sp audit verify` for verification of its encrypted hash-chained audit log without requiring the vault key. That makes it useful in this test set, but verification should be one assertion among several, not the whole test.

Your expected results need nuance. A record absent after a kill before its durability boundary can be acceptable if no external action began. A record absent after the gateway started a state-changing request is not acceptable if the stated design promised write-ahead intent. A record present with an unknown result is often correct. A record that claims completion before the remote response was observed is wrong, even if the test usually happens to pass.

Keep agent-run history separate from action-call history when you inspect recovery. A session can end abruptly while several actions have different outcomes. One run-level entry saying "terminated" does not replace per-call records for a request that reached the remote system and another that never left memory.

## Approval state must not outlive its subject

A restart tests authorization state as much as durability. If a gateway authorizes a particular agent process for a session, killing the gateway must not turn that authorization into a reusable permit for a new process that happens to make a similar request.

The simplest safe rule is that a session approval belongs to an observed process identity and dies with that run. After restart, a new agent process needs a new session authorization. A current process that remained alive while the gateway restarted also needs a fresh decision unless the gateway can re-establish the exact identity binding and the product deliberately documents that behavior. Do not restore approval from a loose cache record such as command name, working directory, or a friendly process label.

Test the restart boundary with two agents that look similar from the command line but differ in signing authority or executable origin. Approve the first. Stop the gateway during an action. Restart it. Then have the second agent issue the same action. The gateway must show a new approval decision rather than inheriting the first agent's session.

Per-call approval has a different test. Kill the gateway while the approval card is visible, then restart and repeat the action. The stale UI event must not authorize the new call. Kill after the user approves but before credential resolution, then confirm that the new process cannot reuse the old approval. These tests catch a common error: persisting the boolean "approved" without binding it to an action ID, a specific session, and an expiry condition.

Sallyport's fixed controls give this test a clear shape. Its vault gate denies actions while locked, its session authorization is tied to a new agent process, and selected credentials can require approval on every use. Force quit testing should prove those boundaries remain true when the UI, vault state, and action pipeline restart at different moments.

## Build a result matrix before you run the suite

A test that merely says "kill at stage X" leaves too much room for interpretation. Create a matrix with rows for interruption points and columns for observable outcomes. Review the expected result before you write the test code. If the team cannot agree what should happen, the implementation does not yet have a defined failure contract.

Use these columns:

| Cut point | May remote see a connection? | May remote see credentials? | Permitted local action status | Agent result after recovery | Required evidence |
|---|---:|---:|---|---|---|
| Before credential lookup | No | No | Not started or interrupted | New approval or retry path | Gateway trace only |
| After credential resolution | No | No | Interrupted preparation | New approval or retry path | Gateway trace only |
| During request write | Yes | Possibly | Unknown | Reconcile before retry | Receiver record and local record |
| After remote acceptance | Yes | Yes | Unknown until observed or reconciled | Reconcile before retry | Remote durable record |
| After durable completion | Yes | Yes | Completed | Return cached or reconciled result | Verified audit record |

The words "possibly" and "unknown" are not weaknesses. They are an honest description of distributed work. The dangerous word is "failed" when the gateway has no evidence that the remote service failed to act.

Run each row repeatedly, but do not turn repetition into a substitute for control. One hundred random kills can miss the line between a journal append and its durability barrier. A single controlled kill at a hold point can establish what that line does. Repeat afterward to catch races, scheduling mistakes, and accidental movement of the checkpoint.

Collect artifacts from both sides on every run: the harness event timeline, the remote fixture's durable action list, process exit information, gateway recovery diagnostics, audit verification output, and agent-visible output. Put the run ID in the filename. If a test fails, preserve the evidence before rerunning it. The second run often destroys the only useful clue.

## Treat discrepancies as design work, not test noise

When a force quit test produces a disagreement between the gateway journal and the remote fixture, resist the urge to patch the assertion until it passes. The disagreement is usually the test finding.

If the fixture reports an accepted request and the gateway records no intent, move the durable intent boundary earlier or stop the action before that boundary. If the gateway reports completion and the fixture has no action, identify whether the gateway confused a local write with a remote result. If a restarted agent can act without a fresh approval, fix identity binding rather than adding a longer timeout.

The best outcome is not a dashboard full of green tests. It is a failure contract that an operator can use under pressure: this action definitely did not leave the machine; this action may have reached the remote system and needs reconciliation; this action completed and its record survived verification. Those are the only categories that make a post-crash decision defensible.

Run this suite whenever you change credential plumbing, transport libraries, journaling, retry behavior, process supervision, or approval handling. A gateway earns trust by behaving predictably when it gets no chance to explain itself.