Testing MCP tools with fake credentials that fail safely

Testing MCP tools safely requires more than replacing a production token with a string called TEST_TOKEN. A tool can parse the fake value, return a friendly success message, and still fail the first time an agent meets a real permission boundary, an approval denial, a slow provider, or a half completed write.

The test environment must let you prove two things at once: the tool sends the intended request, and it cannot create a real production consequence when the test goes wrong. Disposable accounts and fake credentials solve different parts of that job. Treat them as separate controls.

Fake credentials test parsing, not authority

A fake credential proves only that the tool handles a credential shaped value correctly. It does not prove that the provider accepts the credential, that the credential has the intended scopes, or that a revoked value fails cleanly.

That distinction gets blurred because many teams call every nonproduction secret a fake secret. There are three materially different things:

• A synthetic string exists only in a local stub. It cannot authenticate anywhere.
• A test credential authenticates against a real provider, but only inside a disposable tenant or project.
• A restricted production credential can authenticate against production, even if its scope looks small.

The first option belongs in unit and contract tests. The second belongs in integration tests. The third does not belong in an automated agent test suite. A credential that can read a production customer record has already crossed the boundary you meant to protect.

Make fake values resemble the format your code actually receives. If an API client rejects tokens without a prefix or a minimum length, use a synthetic value that passes that local validation. Do not copy a real token and alter one character. People paste fixtures into issue trackers, terminal transcripts, and chat threads. A nearly real secret gives you all the handling risk without any testing benefit.

A useful fixture names the authority it does not have. For example, stub_token_no_network says more than token123. A test credential reference such as billing_test_writer says that a secret store, rather than the agent, resolves the actual value. Keep the reference stable and rotate the underlying test credential when needed.

Do not confuse a successful mock with authorization. When a stub returns 200, it has accepted whatever contract you programmed into it. That is useful evidence about your own request construction. It says nothing about the provider's scope model.

Separate invalid input, rejection, and operational failure

An agent needs different guidance for a bad argument, a denied action, and a broken dependency. If your MCP tool turns all three into request failed, the agent will retry actions it should stop and abandon actions it could repair.

Use a small outcome vocabulary and preserve it through the tool result. I use five classes:

• Validation error: the agent supplied an invalid or incomplete argument. The agent can correct the call.
• Authentication failure: the credential is absent, expired, malformed, or revoked. Retrying the same call will not help.
• Authorization rejection: the identity authenticated but lacks permission, or a human denied an approval. The agent must not probe around the boundary.
• Conflict: the request was valid but cannot apply to the current resource state. The agent may need to fetch current state first.
• Operational failure: timeout, connection failure, rate limit, or provider fault. A retry may make sense if the action is safe to retry.

HTTP defines the familiar pieces of this split. RFC 9110 describes 401 Unauthorized as an authentication challenge, despite its historically confusing name, and 403 Forbidden as a refusal to fulfill the request. Your provider may use those codes imperfectly, so test the body and documented error codes too. Do not build agent behavior from status code alone.

The Model Context Protocol tool result supports an isError flag alongside content. Use it when the tool call failed, but put the actionable category in the text or structured content your client expects. A transport level JSON RPC error and a tool level error are also different. Reserve JSON RPC errors for malformed protocol requests or unavailable methods. Return a normal tools/call result with isError: true when the tool ran and the upstream action failed or was rejected.

This response shape gives an agent something concrete to act on:

{
  "jsonrpc": "2.0",
  "id": 42,
  "result": {
    "content": [
      {
        "type": "text",
        "text": "authorization_rejected: identity billing_test_writer cannot create invoices in tenant test-acme. Request a role change or stop."
      }
    ],
    "isError": true
  }
}

Do not include the bearer token, authorization header, full upstream request, or raw provider error in this result. Error messages are part of the agent's context window, which means they may be retained by systems you do not control. The error needs enough detail to guide behavior, not a forensic dump.

Disposable accounts need a hard boundary

A disposable account is safe only when it has no path to production resources. A separate email address and a test label do not create that boundary.

Start with the provider's strongest isolation unit. That may be a separate organization, tenant, cloud project, database, or self hosted instance. Put the test account inside that unit, and verify the account cannot switch into a production unit through an identifier, a shared role, or a default configuration value.

Then give the account permissions that make the intended test possible and nothing else. If a tool creates invoices in tests, its test identity needs permission to create and list test invoices. It does not need export permissions, administrator access, or access to a shared payment configuration. Broad test access is popular because it shortens setup. It also hides which permissions the tool actually needs.

Use an explicit test run marker for every resource your suite creates. Put it in a supported metadata field, description, tag, or name. A marker makes cleanup safer and makes leaked test artifacts recognizable. Do not delete everything that happens to sit in a test tenant. Another developer may be reproducing a bug there.

A minimal configuration can look like this:

run_id: mcp-it-20250308-7f3c
account: [email protected]
allowed_tenant: test-acme
resource_prefix: mcp-it-20250308-7f3c-
cleanup_after_minutes: 90

The config prevents a mundane but expensive failure: a test runner points at the wrong tenant because an environment variable from a developer shell wins over the checked configuration. Your setup code should fetch the current tenant identity before it creates anything and compare it to allowed_tenant. If they differ, stop before the first write.

Disposable does not mean anonymous or unowned. Assign an owner, record how the credential is issued, and make expiry intentional. A forgotten test identity is still an identity with access.

Build contracts around observable requests

A good contract test checks the request your tool sends, the result it returns, and the information it refuses to expose. It does not merely assert that a function was called.

Put a local HTTP stub in front of your client and make it inspect method, path, headers, query parameters, and body. The stub should reject unexpected fields. Permissive mocks train a tool to send accidental arguments until a real provider rejects them.

For a create_invoice tool, a focused test can make this call:

{
  "jsonrpc": "2.0",
  "id": 42,
  "method": "tools/call",
  "params": {
    "name": "create_invoice",
    "arguments": {
      "tenant": "test-acme",
      "customer_id": "cus_mcp_it_7f3c",
      "amount_cents": 500,
      "currency": "USD"
    }
  }
}

The stub should expect POST /v1/invoices, confirm that the authorization header contains its synthetic test value, and confirm that the request includes the run marker. It can return a fixed identifier such as inv_mcp_it_001. The MCP result should expose the invoice identifier and status, but never echo the authorization header.

Write at least one negative contract assertion for every sensitive request field. Send a caller supplied authorization, base_url, account_id, or tenant field if your schema might admit one. Confirm that the tool rejects or ignores values that could redirect an action to another account. Many credential leaks begin as a harmless looking escape hatch for a custom endpoint.

Test request serialization at the bytes that matter. A provider may distinguish omitted fields from null, empty strings from missing values, and numeric values from numeric strings. Agents produce unexpected argument shapes, especially when a tool description fails to state units. Your schema should say amount_cents, not amount, if the provider expects minor units.

Contract tests are also where you enforce log hygiene. Capture the structured log event and assert that it contains a credential reference or a redacted marker, never the synthetic secret itself. A fake secret becomes a real disclosure problem once engineers normalize printing it everywhere.

Test permissions as a matrix, not a happy path

A single authorized test identity cannot tell you whether a tool handles permissions correctly. You need several identities with deliberately different rights and a set of cases that names the expected outcome.

Keep the matrix small enough to maintain. For a write tool, these cases usually earn their place:

This matrix catches a common defect: the tool checks that a token exists but never checks which tenant the token actually represents. The happy path passes because the writer has broad access. The failure appears when an agent receives a tenant identifier in a task and the client honors it without binding it to the selected credential.

Run each matrix case against the real disposable provider when you can. Provider semantics around inherited roles, default scopes, and delayed revocation often differ from the documentation. Keep the cases isolated. If one test upgrades a role that another test expects to be absent, parallel runs create failures that look like authorization bugs.

Do not manufacture forbidden responses solely in a stub and call the suite complete. A stub confirms that your error mapper recognizes a 403. The real test identity confirms that your credential issuance process, provider setup, and tool routing produce a genuine rejection.

There is one useful exception: test provider responses you cannot reliably trigger, such as malformed JSON or an invalid content type. A stub owns those cases. The point is not purity. The point is to know which evidence each test provides.

Writes need cleanup plans before test code

For every tool that changes state, decide how the test will remove or neutralize that state before you write the success test. If you cannot describe cleanup, choose a different target or a different operation.

Prefer operations that create records inside a short lived test project. Avoid test calls that send email, charge a card, rotate a shared secret, trigger deployment, or contact a real third party. A provider's sandbox may still send webhooks to an endpoint you configured years ago. Check the surrounding effects, not only the API label.

Use a unique run marker and clean up in a finally block or its equivalent. Cleanup must tolerate a resource that was never created, a resource that was created twice after a retry, and a partially configured account. Idempotent cleanup saves time when a test fails halfway through provisioning.

A failure worth testing looks like this. The tool sends a create request. The provider creates the record but drops the connection before returning the response. The agent sees an operational failure and retries. Without an idempotency value, the retry creates a duplicate. Without a run marker, cleanup cannot safely find both records.

Give the create request an idempotency value derived from the run identifier and the logical action, not from the transport attempt. The first attempt and retry must use the same value. Then test the dropped response case with a stub that records the first request, closes the connection, and returns success when it receives the same idempotency value again. Assert that the provider side has one record.

If the provider does not support idempotency, make the tool perform a lookup using a unique external reference before it retries a write. That approach can race under concurrency, so document the remaining risk. Do not silently retry money movement or irreversible writes because a generic HTTP client considers POST retryable.

Timeouts and rate limits reveal bad agent behavior

A tool that handles success and 403 correctly can still cause damage when its dependency becomes slow. Agents tend to retry because they are trying to finish the task. Your tool must give them a bounded, truthful response.

Test a connection timeout before any request reaches the server, a response timeout after the server receives the request, and a provider 429 response. These cases differ. The first usually means no side effect occurred. The second may mean the server completed the write. A 429 may include a retry instruction, but only use it if the provider documents the field and your tool preserves its meaning.

Set short test timeouts so the suite remains usable, but do not replace production timeouts with test values in code. Inject a clock or transport configuration. A hard coded two second timeout is a test convenience that will turn into an outage when someone deploys it.

Your assertions should cover agent facing behavior as well as the HTTP client. For a rate limit, return an operational failure that names the provider and says whether retrying after a delay is permitted. For a response timeout after a write, say that outcome is unknown and instruct the caller to look up the operation by its idempotency value or external reference. Calling that case a simple failure invites duplicate writes.

Do not test retries by asserting a count alone. Record whether the retry reused the idempotency value, whether it waited when it should, and whether it stopped after the configured limit. A retry loop that ends eventually can still create a burst that exhausts a small test tenant and conceals the actual defect.

Approval denial must leave the target untouched

If a human can approve agent actions, test the denial path against an endpoint where a side effect would be obvious. A screen that says denied does not prove the execution layer stopped.

Set up a disposable target with a counter, a marker record, or an append only test event list. Start the agent process, issue the tool call, and deny the session or call. Then query the target directly with a separate test observer identity. The expected count is unchanged.

This test detects a bad ordering bug: the tool starts the upstream request, then asks for approval while it waits for the response. That flow may look fine in a demonstration if the provider is slow. It fails the purpose of approval. The authorization decision must happen before the action dispatcher opens the network connection or starts an SSH helper.

Test a fresh agent process separately from a second call in the same process. Those are distinct security promises for systems that grant authorization per run. Also test a process whose signing identity differs from the expected one. The approval prompt must show enough origin information for the person approving to distinguish the intended agent from an arbitrary local process.

For a local Mac workflow, Sallyport keeps API and SSH credentials in its encrypted vault and can require an approval for each new agent run or for each use of a selected credential. Treat those controls as things to test against disposable endpoints, not as a reason to skip upstream permission tests.

SSH tests need a target that can be discarded

SSH adds failure modes that HTTP examples hide: host verification, command quoting, environment inheritance, remote shell behavior, and files left behind after a dropped connection. Never point automated agent SSH tests at a developer workstation or a shared administration host.

Create a controlled test machine or short lived virtual machine with a dedicated account. Give that account a home directory with no useful data, a restricted command set when feasible, and no credentials that can reach other systems. Use a dedicated test SSH key and discard it when the environment expires.

Exercise command arguments that break naive shell construction. Test spaces, quotes, newline characters, paths beginning with a dash, and data that resembles shell syntax. The tool should pass arguments to the remote command without concatenating user supplied values into a shell string. If the required remote interface only accepts shell text, constrain the allowed command grammar and reject everything outside it.

A safe test can ask the target to create a file named with the run marker and then read that exact file. A rejection test can request a path outside the test account's allowed directory and expect the target to deny it. An operational test can terminate the SSH connection after the command starts, then inspect whether the remote process continued.

Collect exit status, bounded standard output, and bounded standard error. Do not return unlimited command output to an agent. Large outputs consume context, and command output often contains configuration values that were never meant to leave the host.

Audit evidence must connect the agent call to the side effect

When an agent test fails, you need to answer whether the tool made the call, whether the provider received it, and whether cleanup removed it. A pile of console text does not answer those questions reliably.

Give every test run one identifier and carry it through the MCP request, tool log, provider metadata, and cleanup log. Do not put credentials in that identifier. A useful evidence record includes the tool name, action category, credential reference, target tenant, request correlation identifier, outcome category, and returned resource identifier.

Keep agent session records separate from action records. A session tells you which process made a run and when you revoked it. An action record tells you which external call occurred. Joining them through a correlation identifier makes a denial test auditable: you can show the agent attempted an action, authorization denied it, and no matching external action record exists.

Tamper evidence matters when you use the test results to review a new tool boundary. Sallyport projects its session and activity journals from an encrypted, hash chained audit log, and sp audit verify can verify that chain offline without a vault key. That does not replace provider logs, but it gives local tests a way to detect altered action history.

Do not treat audit output as a secret sink. Record credential references and redacted attributes, then assert those rules in tests. The audit trail should help you reconstruct an action without becoming the easiest place to steal the access that enabled it.

A release candidate earns production access slowly

A tool should graduate through increasingly realistic boundaries: local synthetic stubs, a real disposable provider, denied and expired identities, failure injection, and a human approval test if the workflow uses one. Skipping straight to production because the sandbox differs is how teams discover their safety behavior under pressure.

Keep one short release suite that runs on every change and a deeper suite that provisions disposable resources less often. The short suite should cover tool schema, request construction, redaction, and representative failures. The deeper suite should cover actual scopes, lifecycle cleanup, revocation, and target isolation.

Before you allow a new action against production, inspect the evidence from a deliberately denied call and a deliberately ambiguous write timeout. Those cases reveal whether the tool respects a boundary and whether it tells an agent the truth when the provider may have acted. Success paths are easy to stage. The refusal and uncertainty paths are where production access becomes either controlled or reckless.