# What desktop execution gateway latency should teams require?

A desktop execution gateway earns its place only if its machine delay stays beneath the noise of interactive work and its approval delay is reported separately. I would not approve a rollout from an average, a single warm run, or a chart that mixes a person reaching for Touch ID with software execution time. Those numbers may look tidy, but they cannot tell an engineering team whether the gateway will interrupt an edit-test loop.

For ordinary HTTP and already connected SSH work, I use a provisional adoption gate of no more than 25 ms added at p50 and 75 ms at p95 on the target developer Macs. For fresh SSH connections, use a separate, larger budget: no more than 50 ms added at p50 and 150 ms at p95. Treat those as starting requirements, not universal constants. A team should tighten or relax them against its own direct baseline, call frequency, network, and perception tests. The test below produces the evidence needed to make that decision without hiding a slow tail behind a decent median.

## Percentiles must describe the same operation

The p50 and p95 figures mean something only when every sample represents the same boundary. p50 is the median: half the observed calls finished at or below it. p95 is the value at or below which 95 percent finished. In a run of 200 sorted samples, the p95 is the 190th value under the nearest-rank definition. That leaves ten slower observations visible in the tail instead of averaging them away.

Define the boundary as the time from the caller starting an operation until the caller receives the complete result. That is the delay a coding agent experiences. It includes local process work, transport to the target, remote execution, response transfer, and any gateway work that sits on that path. It excludes benchmark setup and, in the unattended track, excludes human confirmation.

Teams often compare unlike boundaries. A direct HTTP chart might use curl's `time_starttransfer`, while the gateway chart uses total wall time around a process invocation. The first stops at the initial response byte; the second stops after the result has been serialized and returned. The gateway loses before the test begins. Pick total completion for both paths, then keep phase timings as diagnostic columns.

The same rule applies to SSH. A new TCP connection, SSH handshake, authentication, remote command, and disconnect form one operation. A command sent over an existing multiplexed connection is a different operation. Report them in separate rows. Mixing them creates a distribution whose p50 mostly describes reuse and whose p95 mostly describes connection setup. No threshold can repair a confused population.

Report at least these columns for each operation class:

- direct p50 and p95 wall time
- brokered p50 and p95 wall time
- added p50 and p95, calculated from paired samples
- sample count, failures, and timeout count
- host, network condition, gateway version, and approval mode

The paired added latency matters more than subtracting two headline percentiles. For sample `i`, calculate `delta_i = brokered_i - direct_i` under the same test condition, then take p50 and p95 of the deltas. `p95(brokered) - p95(direct)` is useful context, but it is not the p95 overhead and can even conceal correlation between network spikes and gateway work.

## A benchmark contract prevents convenient results

Write the benchmark contract before running it, because nearly every accidental optimization changes the question. Record the Mac model and macOS version, whether it is on power, thermal state, network path, target host region, HTTP protocol, SSH cipher negotiation, payload sizes, response sizes, timeout, concurrency, and connection reuse. Keep background load representative of a developer machine. A pristine laptop with every editor and build process closed answers a laboratory question.

Use one local target and one realistic remote target. The local target exposes machine overhead because network variance is tiny. The remote target shows whether that overhead still matters amid real transport delay. A gateway that adds 40 ms to a 3 ms loopback request feels very different from the same addition to a 300 ms remote call, yet both results belong in the report.

Create operation classes rather than one blended suite. A useful minimum is a small HTTP response on a reused connection, a small HTTP response on a new connection, a fresh SSH connection that runs `true`, and an SSH command on an already available connection. Add one representative payload and one representative response from the team's coding workflow. Do not use a destructive API or a production SSH host for a latency test.

Run a warmup that never enters the sample set. Then collect at least 200 measured calls per class, alternating direct and brokered order. If every direct call runs first, a later thermal change, network change, DNS cache, or server cache becomes gateway overhead. Random order is acceptable, but strict alternation is easier to inspect.

Keep concurrency at one for the interactive benchmark. An agent waiting for a tool result is a serial user. Run a second load test if several agents will share the gateway, but label it as capacity work. A 20-way test can reveal queueing limits; it cannot replace the single-call latency result.

Failures stay in the report. Record a timeout as a failure with its elapsed timeout value, and publish the failure rate next to percentiles. Quietly removing errors rewards a path that fails quickly or drops its slowest calls. If either path fails enough that its p95 is unstable, stop debating milliseconds and fix reliability first.

## Measure HTTP without changing its semantics

Use the same method, URL, headers, body, timeout, and response consumption for direct and brokered HTTP. The credential source may differ because credential isolation is the point of the gateway, but the server must see requests with equivalent semantics. Confirm that both variants receive the same status, selected response headers, and body hash before collecting timing data.

curl exposes useful phase clocks in its documented write-out variables. `time_connect` ends when the network connection completes, `time_appconnect` includes the TLS handshake, `time_starttransfer` reaches the first response byte, and `time_total` reaches completion. Those values help locate a regression, but caller wall time remains the comparison boundary because a broker can do work before curl or after the upstream response.

This direct probe prints four seconds-based fields without adding a link or parsing progress output:

```sh
curl -sS -o /dev/null -w '%{time_connect} %{time_appconnect} %{time_starttransfer} %{time_total}
' -H "$AUTH_HEADER" "$ENDPOINT"
```

For the paired measurement, put the complete direct invocation in `DIRECT_CMD` and the complete gateway invocation in `BROKER_CMD`. The following runner alternates their order, uses a monotonic clock, preserves failures, and emits one JSON object per observation. It deliberately treats each command as an argument vector, so shell operators and expansions do not slip into the benchmark.

```python
import json
import os
import shlex
import subprocess
import time

samples = int(os.environ.get("SAMPLES", "200"))
timeout = float(os.environ.get("TIMEOUT", "10"))
commands = {
    "direct": shlex.split(os.environ["DIRECT_CMD"]),
    "brokered": shlex.split(os.environ["BROKER_CMD"]),
}

for n in range(10):
    label = "direct" if n % 2 == 0 else "brokered"
    subprocess.run(commands[label], stdout=subprocess.DEVNULL,
                   stderr=subprocess.DEVNULL, timeout=timeout)

for n in range(samples):
    order = ("direct", "brokered") if n % 2 == 0 else ("brokered", "direct")
    for label in order:
        started = time.monotonic_ns()
        try:
            result = subprocess.run(commands[label], capture_output=True,
                                    timeout=timeout)
            ok = result.returncode == 0
            code = result.returncode
        except subprocess.TimeoutExpired:
            ok = False
            code = None
        elapsed_ms = (time.monotonic_ns() - started) / 1_000_000
        print(json.dumps({"pair": n, "path": label, "ms": elapsed_ms,
                          "ok": ok, "code": code}, separators=(",", ":")))
```

Capture the output in a JSON Lines file. Before calculating latency, compare success codes and response hashes outside the timed loop. A brokered request that returns less data will appear faster; a direct request that follows redirects while the broker does not will measure different work.

Connection reuse deserves explicit control. If the agent protocol and gateway retain an upstream HTTP connection, compare that with a direct client that also retains one. Starting a new curl process for every direct sample may forfeit connection reuse and flatter the brokered path. For a process based test, either force new connections on both sides or build two small persistent clients. Publish which choice you made.

## SSH needs cold and reused measurements

SSH setup often dominates a short remote command, so a single SSH percentile says little. The OpenSSH client manual documents connection sharing through `ControlMaster` and `ControlPersist`. With sharing enabled, later sessions can reuse an existing network connection instead of repeating transport setup and authentication. That is a valid production optimization, but only if both paths receive an equivalent chance to use it.

For a fresh connection class, disable sharing, run a harmless command such as `true`, and disconnect. A direct command can look like this on a dedicated test host:

```sh
ssh -F /dev/null -o BatchMode=yes -o ControlMaster=no -o ConnectTimeout=10 "$TEST_HOST" true
```

Use the gateway's normal SSH action for the paired brokered call, with the same host, user, and remote command. Do not force the gateway through an artificial setup that the agent would never use. Instead, make the direct path match the gateway's documented connection behavior, or declare the architectural difference in the result.

For the reused class, establish the connection before warmup and run only the remote `true` command during sampling. Verify reuse rather than assuming it. OpenSSH verbose diagnostics can show whether the client contacts a control socket; the gateway should expose enough diagnostic timing or logs to establish whether it reused transport. If reuse cannot be proven, label the class "unknown connection state" and do not compare it with a known reused path.

Then repeat with one actual non-destructive task, such as reading a small fixed file or asking a test service for status. `true` isolates connection and execution plumbing, but it does not cover output transfer, decoding, or result limits. Keep the file content fixed and hash the returned bytes.

SSH authentication also changes the comparison. An agent should not receive a private key merely to make the direct baseline convenient. Run the direct path in an isolated harness with equivalent network access, and treat its credential exposure as a benchmark fixture, not an endorsed design. Latency tells you the cost of mediation. It does not decide whether giving an autonomous process a key is acceptable.

## Instrument the gateway around its own work

An external stopwatch proves the user-visible result, while internal timestamps explain it. Add monotonic timestamps at receipt, request validation complete, authorization decision complete, credential operation complete, upstream dispatch, first upstream byte, upstream completion, audit commit, and result delivery. Record durations or a shared trace identifier, not secrets or request bodies.

Those timestamps divide added machine delay into parts a team can act on:

- local protocol parsing and serialization
- authorization lookup and any queue wait
- vault or credential operation
- upstream client setup and connection acquisition
- audit persistence and result return

Do not compare wall clock timestamps from separate processes unless the measurement design handles clock differences. On one Mac, a monotonic clock in each process is stable for durations, but the origins may not line up across runtimes. The safest design has each component report its own duration and joins records by an opaque test identifier.

Audit work belongs inside the measured boundary if the gateway promises that an action is recorded before it reports success. Moving the write after the response can win a benchmark by weakening the guarantee. Batching may be legitimate, but the published durability point must match production. The test should measure the product people will run, with its ordinary journal and security controls enabled.

Profile only after phase data identifies a suspect. If p50 rises with response size, inspect copies and serialization. If p95 rises while p50 stays flat, inspect locks, queue waits, audit flushes, connection pool misses, and operating system scheduling. A CPU profile of a quiet median call rarely explains a tail stall.

Protect the benchmark itself from observer cost. Detailed logging to a terminal can add blocking I/O. Write compact records to a file or an in-memory buffer, measure once with diagnostic events enabled, and confirm that the difference is negligible. Keep the diagnostic mode consistent across direct and brokered runs where possible.

## Human confirmation is a separate service level

Confirmation time starts when the approval card becomes visible and ends when the decision reaches the blocked call. It is not gateway machine latency. Combining it with automatic calls produces a p95 driven by hand position, attention, screen state, and interruption. That may be an important workflow measure, but it answers whether approval fits the work, not whether the execution path is efficient.

Report three distributions for protected actions. First, record gateway machine time before the prompt. Second, record visible prompt to decision. Third, record machine time after the decision until result delivery. The end-to-end figure can appear beside them, provided nobody uses it to diagnose transport overhead.

Do not automate clicks and call that human latency. Automation can measure presentation and decision plumbing, which belongs in the machine buckets. A human study needs consenting participants, a defined task, and event timestamps that avoid capturing sensitive input. Report participant count and whether they expected the prompt. An expected Touch ID confirmation during a scripted test will be faster than an unexpected approval during real coding.

Approval modes also need separate rows. An action allowed for an already authorized session, a first call that asks to authorize the process, and a per-call confirmation have different paths by design. Blend them and the result depends on their accidental frequency in the sample.

A useful workflow budget describes interruption rather than pretending every approval must fit under 100 ms. For example, require the card to appear within 150 ms of the call reaching the gateway at p95, then measure human decision time without a pass or fail target until the team has observed real use. Set a later target based on abandonment, mistaken approvals, and task disruption. Fast approval is bad if the card withholds the identity or action detail needed for a sound choice.

## Calculate deltas before drawing the chart

The result calculator should pair observations, reject incomplete pairs from the latency distribution, and report those incomplete pairs as failures. This prevents a successful direct call from being compared with the next brokered call after its actual partner timed out. Keep the pair number from the runner as the join field.

Use one percentile definition everywhere. The nearest-rank method sorts values and selects rank `ceil(p * n)`, with ranks starting at one. Libraries offer interpolated percentiles that can return a value nobody observed. Interpolation is legitimate for some analysis, but changing methods between a notebook and a dashboard creates needless arguments near a threshold. State the method in the report.

This small calculator consumes the JSON Lines emitted by the earlier runner. It prints counts plus nearest-rank p50 and p95 for direct time, brokered time, and paired added time:

```python
import json
import math
import sys

rows = [json.loads(line) for line in sys.stdin if line.strip()]
by_pair = {}
failures = {"direct": 0, "brokered": 0}

for row in rows:
    if not row["ok"]:
        failures[row["path"]] += 1
        continue
    by_pair.setdefault(row["pair"], {})[row["path"]] = row["ms"]

direct = []
brokered = []
deltas = []
for pair in sorted(by_pair):
    values = by_pair[pair]
    if "direct" not in values or "brokered" not in values:
        continue
    direct.append(values["direct"])
    brokered.append(values["brokered"])
    deltas.append(values["brokered"] - values["direct"])

def percentile(values, fraction):
    ordered = sorted(values)
    rank = max(1, math.ceil(fraction * len(ordered)))
    return ordered[rank - 1]

result = {"complete_pairs": len(deltas), "failures": failures}
for name, values in (("direct", direct), ("brokered", brokered),
                     ("added", deltas)):
    result[name] = {
        "p50_ms": percentile(values, 0.50),
        "p95_ms": percentile(values, 0.95),
        "max_ms": max(values),
    }
print(json.dumps(result, separators=(",", ":")))
```

The output shape is deliberately plain: `complete_pairs`, failure counts for each path, and a block for each distribution containing `p50_ms`, `p95_ms`, and `max_ms`. Add confidence intervals if the team already uses them, but do not let an interval replace raw tail inspection. A percentile can move between runs because the system changed or because sampling varied; the raw rows tell you which explanation fits.

Negative deltas are valid. They can appear when the broker reuses a connection that the direct command opens again, or when ordinary network noise favors the brokered member of a pair. Do not clamp them to zero. Instead, fix an unfair connection setup or let the paired distribution retain the variance. Clamping biases every percentile upward and makes the calculation harder to audit.

Preserve full precision in the data and round only presentation values, preferably to one decimal millisecond. Threshold evaluation should use the unrounded number. A reported 75.0 ms can represent a value just above the 75 ms gate if formatting happens before comparison.

## Tail latency tells you where trust erodes

Interactive coding punishes irregular delay more than a stable small tax. A developer can adapt to an extra 15 ms that appears on every tool call. A random 800 ms pause feels like a broken agent, especially when several calls occur in sequence. This is why p95 is an adoption requirement rather than a decorative chart.

Inspect the raw tail observations. Label whether each coincided with an HTTP connection miss, SSH handshake, vault unlock, audit flush, app wake, CPU pressure, or network spike. Do not delete an outlier merely because you can explain it. If production will contain that condition, it belongs in the distribution. Remove only a proven test fault, preserve the original data, and publish the exclusion rule.

p99 can help during engineering, but small samples make it noisy. With 200 calls, p99 rests on roughly the two slowest observations. I require p50 and p95 for adoption, then inspect maximum and raw tail rows for failure patterns. A longer overnight run can support p99 work after the interactive gate passes.

Warm and cold states must remain visible. Measure app launch or vault unlock as its own operation if users encounter it, but do not mix it into steady calls. Likewise, report Mac sleep and wake recovery separately. An always running desktop app can look excellent after warmup and still mishandle the first action after lunch.

Sequence cost belongs in the decision too. If a coding task makes twelve serial calls, added delays accumulate. Replaying a captured, scrubbed action sequence can expose costs that isolated requests miss, such as pool churn or a journal that slows as a session grows. Keep the individual operation distributions, then add task completion delta as a secondary measure.

## Set the gate from interaction cost

The gateway passes when added machine delay is both small in absolute terms and small relative to the direct path. I start with four limits for a current desktop on representative load:

- HTTP with reuse: 25 ms at p50 and 75 ms at p95
- HTTP with a new connection: 30 ms at p50 and 100 ms at p95
- SSH with a reused connection: 25 ms at p50 and 75 ms at p95
- SSH with a fresh connection: 50 ms at p50 and 150 ms at p95

Apply the relative check when the direct p95 is at least 100 ms. On a 3 ms loopback call, a 20 percent rule permits less than a millisecond and mostly measures noise. On a slow remote call, an absolute limit alone may permit an implementation that adds a large fraction of the direct time. The two checks cover different regimes.

Require a failure rate no worse than direct plus a small, declared tolerance, and make any gateway-caused timeout an automatic investigation. I would not trade reliability for a 10 ms median win. Also require correct result equivalence and the intended audit durability. A fast path that changes the response or records it later has failed a different, more serious test.

Run the gate on the slowest supported Mac class the team actually uses, not just the newest machine on the bench. Repeat on power and under a representative editor plus build load. Require three runs on different days or network periods for remote targets. This is not statistical ceremony; it prevents one friendly network window from becoming a product promise.

Teams can adjust the provisional numbers with a blinded perception test. Inject fixed delays into a mock gateway, replay a typical agent task, and ask developers to rate interruption without telling them the delay. Choose the smallest delay that produces a consistent complaint, then place the p95 limit below it with margin. Preserve the 150 ms fresh SSH allowance only if the direct handshake already dominates and the complete task still feels immediate.

Reject the rollout if one common operation misses the p95 limit, even when the combined chart passes. A weighted aggregate lets frequent fast HTTP calls bury slow SSH actions. Approve by operation class, then review the captured task sequence to confirm that several acceptable taxes do not add up to an unacceptable pause.

## Make adoption reversible and evidence based

Adoption should begin with a canary group and a recorded rollback condition. Freeze the benchmark script, fixtures, gateway configuration, and result calculator in version control. Save raw observations rather than screenshots. Anyone reviewing the decision should be able to recompute percentiles and see every failure.

For Sallyport, run the same harness through its HTTP and SSH channels with the vault, session authorization, per-call setting, and audit behavior configured exactly as the intended deployment requires. Its encrypted hash-chained audit log can be checked offline with `sp audit verify`; run that after the benchmark so the latency report and integrity check describe the same calls.

Advance only when each operation class passes on representative Macs, result hashes match, audit verification succeeds, and developers accept the full task replay. Keep a direct escape path during the canary, but record every use and its reason. If people bypass the gateway because the p95 feels erratic, the benchmark missed the workflow. Fix the measurement before widening the rollout.