MCP agents off environment variables: migrate credentials

Putting credentials in an MCP server's environment is a shortcut that becomes dangerous once an agent can execute commands, inspect files, troubleshoot failures, or spawn helpers. The issue is not that every agent will deliberately print a token. The issue is that you have given a process built to explore and act a reusable secret, then asked it to behave as though it cannot see it.

Move the authority out of the agent process. Let the agent request a defined HTTP or SSH action, and let a local component that holds the credential execute it. That change sounds small, but it forces you to identify what each tool actually does, which identity it needs, and how you will prove the token did not sneak into the new path.

I have seen migrations fail because someone removed API_TOKEN from one config file while leaving it in a shell profile, a task runner, or a copied project template. The API call still worked, everyone relaxed, and the agent still had the old credential. A clean migration treats discovery, replacement, and proof as separate jobs.

Environment variables give the agent more authority than the tool needs

An environment variable does not belong to a single request. It belongs to a process and often to everything that process launches. If an MCP client starts a server with SERVICE_TOKEN in its environment, that server can read it. So can child processes that inherit it, unless somebody carefully strips it. Debug commands, crash reports, test fixtures, and accidental env output can turn a local convenience into a durable leak.

This differs from a tool receiving an authenticated result. A result might contain a repository list, a deployment status, or an error response. The agent needs those to continue its work. It does not need the bearer token that made the request possible.

The distinction gets blurred because both designs can produce the same successful request. They are not equivalent:

• Credential possession means the agent can use, copy, transform, or exfiltrate a secret outside the intended tool call.
• Action authority means the agent can ask a trusted local executor to perform a request under defined controls.
• Result access means the agent sees the response needed to decide its next action.

Getting that distinction wrong leads to a common bad recommendation: put tokens in a secret manager, then inject them into the agent's environment at startup. That may improve storage at rest, but it does not change the runtime boundary. The agent still receives the token.

The MCP specification describes a protocol for clients, servers, and tools. It does not declare environment variables a credential boundary. Treat them as transport for local configuration only when the process receiving them is already trusted with the underlying secret. Autonomous coding agents often do not meet that standard.

Build a tool inventory before changing any configuration

Start with a written inventory. Do not begin by editing JSON files, because configuration files rarely tell the whole story. A tool may read one variable directly, invoke a wrapper that reads another, or rely on a command-line client that loads credentials from a home-directory file.

For every MCP tool, record the tool name, command, target, authentication method, credential owner, permission scope, and the harmless request you can use for testing. Also record where the secret currently enters the process: client config, shell startup file, .env file, CI export, password manager command, or a helper script.

A compact inventory might look like this:

Do not hide broad permissions inside vague names such as prod-token or default-key. Give the credential record a name that tells an operator what it can do and where it goes. deploy-api-production-read is less pretty and much safer during a hurried review.

The inventory also exposes whether a credential should exist at all. I have found write-capable tokens attached to tools that only read project metadata because somebody copied a development setup. A migration is the right time to issue narrower credentials. It is not a reason to preserve every old privilege in a nicer container.

Remove secret injection instead of disguising it

A migrated configuration must stop delivering the secret to the agent or its MCP server. Replacing a literal token with ${SERVICE_TOKEN}, $(secret-tool lookup ...), or a path to an unprotected file does not meet that bar. You changed the spelling, not the authority.

First, find current references. On a project directory, this catches many obvious cases:

rg -n --hidden --glob '! .git' 'API[_-]?KEY|API[_-]?TOKEN|SECRET|PASSWORD|PRIVATE[_-]?KEY|Authorization: Bearer' .

The expected output shape is a list of file:line:matching text entries. Do not paste that output into an issue if it includes live values. Use it to make a remediation list, then search the usual user-level locations separately, such as shell profiles and MCP client settings.

Next, compare the environment visible to the old agent process with the environment visible to the new one. On a controlled test shell, list names without printing values:

env | cut -d= -f1 | sort | rg 'TOKEN|KEY|SECRET|PASSWORD'

You want the old names gone from the process that launches the agent. If SERVICE_TOKEN appears there, the migration is incomplete even if the new gateway path works.

Avoid the tempting halfway design where the MCP server has the token but the primary agent does not. That reduces one exposure path, yet the server still receives unbounded credential possession. If that server can execute arbitrary commands, load plugins, or write logs, you have simply moved the problem into a process that often receives less scrutiny.

Use nonsecret configuration for target selection. A base URL, host alias, account identifier, and credential label may belong in config if they do not grant access. Keep the authenticated material in a local vault that the agent cannot query as data.

Model the new path as requests, credentials, and results

The new flow should have a hard boundary: the agent names an action and supplies ordinary request data, the local executor selects and injects the credential, then the executor returns the response. The agent never receives a placeholder that it can resolve into the secret.

For an HTTP tool, separate the public request shape from the private authentication step. The agent may ask to perform this request:

GET https://api.example.internal/projects/atlas/issues?state=open

The local executor attaches the appropriate bearer, basic, or custom-header credential and returns the response body and status. If the agent asks for an unapproved host, method, or account, the executor should deny the request rather than guessing which credential fits.

For SSH, the request has a host and command, while the private key stays local. This matters because SSH tooling has a habit of smuggling authority through paths, agent forwarding, config includes, and inherited SSH_AUTH_SOCK. A private key path in the tool configuration is not a safe replacement. The agent can often read the file, copy it, or alter the command that uses it.

Sallyport uses this shape through its bundled sp mcp stdio shim: agents make MCP calls, while the app performs HTTP API calls and SSH commands without handing API keys or SSH keys to the agent. That boundary is useful only if you remove the old environment injection as well.

Do not make the local executor a general-purpose proxy with one all-powerful credential. An agent's request should identify a configured destination and credential, not supply an arbitrary URL plus a token choice. Otherwise a prompt-injected agent can turn a legitimate credential into a request signer for somewhere you never intended.

Choose approval points that people can still evaluate

Human approval works when a person can understand what they are approving. It fails when a long-running agent generates a pile of near-identical prompts until the person clicks through them. That failure is predictable, and it is a design fault, not an operator failing a vigilance test.

Use a session-level approval when you need to establish that a specific agent process may use the configured actions during one run. The approval should identify the process in a way that helps you distinguish the real client from an imitation. A process name alone is weak evidence because any program can choose a familiar name.

Reserve per-call approval for credentials with consequences that need fresh human judgment. Production write access, a destructive host command, and a payment-adjacent API merit that friction. Read-only issue lookup usually does not. If every tool call demands a decision, operators stop reading the decision.

A locked vault should deny requests, even if a previously approved agent remains running. That is the point of a vault gate. An unattended process must not inherit authority merely because it had it earlier in the afternoon.

Sallyport has three fixed controls rather than a policy language: a vault gate, authorization for each new agent process, and an optional per-key approval requirement. The fixed model is deliberately narrower than a rules engine, which means there are fewer clever rules for a tired operator to miswrite.

Test success and secrecy as separate claims

A green tool response proves only that something authenticated the request. It does not prove the agent could not obtain the credential. Run a migration test that checks both claims, with a low-risk action first.

Use this sequence for each tool:

1. Lock the local vault and invoke the tool. The call must fail without falling back to an environment token.
2. Unlock the vault, start a fresh agent process, and approve that process if your setup requires it. Run the harmless request identified in the inventory.
3. Inspect the action journal or server-side audit record for the exact destination, account, method, and result status. Confirm that it records the action without recording the secret.
4. From the agent's allowed execution path, inspect its environment for the old variable name and search its workspace for the token prefix. The token must not appear in either place.
5. Restart the agent process and repeat the request. This checks that you did not accidentally rely on inherited state from the old shell.

The first step catches a subtle but serious error. Teams sometimes configure a vault path but leave the old variable in place as a fallback. When the vault is locked, the tool succeeds anyway. That feels reliable until the same fallback appears in a CI worker, copied repository, or an agent transcript.

Do not ask the agent to print all its environment variables as your main proof. That test itself creates a disclosure route and may place the old token in conversation history. Check names from a controlled shell, and use a deliberately nonsecret marker variable during dry runs if you need to validate inheritance behavior.

For SSH, test a command that returns harmless system identity information rather than a command that changes state. Confirm the result, then confirm that the agent configuration contains a host reference rather than the private key material. Also inspect SSH configuration for ForwardAgent yes. Agent forwarding can give a remote host a path to use local identities, which is a separate risk from exposing the private key file.

Audit records must let you reconstruct a disputed action

An action log is useful when it answers a bad Monday-morning question: which agent run made this request, through which configured credential, and did a human approve it? A vague line such as tool succeeded will not settle anything.

Keep a session record for agent runs and a call record for individual actions. The session record tells you when a process began, what identity you approved, and when you revoked it. The call record tells you what happened under that session. Do not blend the two into a single flat event stream if you need to investigate one run among many.

Tamper evidence matters because an agent with local command execution may try to erase its tracks after an undesirable action. A hash-chained log gives you a way to detect changed or removed history, but only if verification does not depend on the agent being cooperative.

Sallyport projects both journals from an encrypted, write-blind, hash-chained audit log and supports offline chain verification over ciphertext with this command:

sp audit verify

A passing result should identify that verification completed successfully; a failure should make you treat the journal as suspect until you understand the break. Verification does not tell you whether the action was wise. It tells you whether the record still has continuity.

Keep audit data out of model prompts and normal chat transcripts. The record may contain sensitive request context or response metadata even when it never contains the credential. Investigation access should not become a back door for casual browsing.

Rotation is the cleanup that proves you meant the migration

After each new path passes its tests, rotate the old credential. Leaving it valid because "we might need to roll back" extends the period in which forgotten configs can still authenticate. Build rollback around a separately controlled replacement, not around a secret you have already sprayed through local environments.

Rotation order matters. Create the new scoped credential, store it locally, validate the new path, remove old secret injection, then revoke the old credential. If an old token may have appeared in source control, a chat message, a support ticket, or build output, revoke it first and accept the outage while you restore safe access.

Repeat the discovery search after revocation. You are looking for stale references, not live secret values. Remove dead variable names from sample files, onboarding documents, shell profiles, task scripts, and test instructions. Future developers copy examples with surprising loyalty.

Finally, leave one deliberate failure test in your operational notes: lock the vault and run a harmless tool call. If the call succeeds, someone has reintroduced a bypass. That single check catches more bad migrations than another polished architecture diagram ever will.

Treat broad credentials as a tool design flaw

Moving a token into a vault does not make a broad token appropriate for an agent. It only changes who holds it. A tool that can query project issues should not quietly carry permission to delete projects, change billing, or deploy production code.

Split tools according to real work and consequence. Read-only discovery actions can use a limited credential and modest approval. State-changing actions deserve tighter scopes, explicit targets, and sometimes per-call consent. This also makes an agent easier to supervise because its available verbs match the task you gave it.

Be suspicious of a universal admin credential justified by tool simplicity. That setup is popular because it reduces configuration work today. It makes incident response worse tomorrow because you cannot tell which requests needed that power and which merely inherited it.

The migration succeeds when an agent can complete its intended work, a locked vault stops it, a new process must earn the authority you configured, and no old token remains in its environment. If any one of those claims lacks a test, you have a working demo, not a credential boundary.