Do production backups need separate backup credentials?

A backup is only a recovery copy if an incident that damages production cannot also erase the copy. That sounds obvious until you inspect the credentials behind an automated job. In many setups, one token reads live data, writes a backup, lists old backups, deletes them for retention, changes the destination, and starts restores. Give that token to an autonomous agent and one mistaken or compromised API call can turn a production outage into a recovery failure.

The fix is not a larger permission policy written in cleverer language. Give live data and recovery data separate action boundaries. The credential that operates production should not remove recovery copies. The credential that writes a backup should not rewrite its retention. A restore should carry its own approval because it moves sensitive data back into an active environment. Each boundary should make a bad request fail before it has consequences.

This is a design problem, not a vendor problem. The same pattern applies whether the copies land in object storage, a managed backup vault, snapshots, a physical appliance, or a second cloud account. Storage immutability matters. Separate administration matters. But neither helps if an agent's normal working identity can still exercise the destructive controls around them.

A backup copy is not separate if one identity can erase it

A recovery copy has to survive the failure modes you expect from production, including misuse of a privileged credential. If the same agent identity can call both delete production database and delete recovery vault, you have duplicated data, not isolated recovery.

Teams often call this separation because production and backups use different buckets, folders, regions, or resource names. That is storage layout. It is not access separation. A single principal with broad access can traverse all of those boundaries in a few requests.

The test is simple: take the credential available to the production agent and ask what it can do without another human, another identity, or a physically distinct control plane. If it can do any of the following, it has too much influence over recovery:

• Permanently delete a recovery point or object version.
• Shorten retention, bypass retention, or remove a hold.
• Disable replication or redirect future copies.
• Change encryption access so restoration fails.
• Delete a backup vault, project, account, or storage container.

The awkward part is that an agent may never receive an explicit instruction to delete backups. A model can select an overly broad cleanup command. A tool wrapper can map an innocent sounding operation to a destructive endpoint. A compromised repository can persuade an agent to use whatever credentials it already has. The protection has to hold when the request is wrong, not only when the prompt was sensible.

There are two different blast radii to control. The first is data plane damage: an agent changes or deletes business data. The second is recovery plane damage: an agent removes the copies, disables the path to them, or makes the copies impossible to decrypt. Most teams put effort into the first and leave the second attached to the same administrator role.

That decision usually comes from convenience. Retention cleanup needs permission to delete expired backups, so the backup job receives broad delete rights. A restore test needs a privileged role, so the same integration receives it. An engineer wants one secret in CI, so the role gathers every permission. Each shortcut is understandable. Together they give a routine automation credential authority over the last line of recovery.

Writing a backup and managing its lifetime are different jobs

The backup writer needs a narrow, repeatable path. The retention administrator needs authority to change or remove copies. A recovery operator needs authority to read a selected copy and introduce it into a controlled target. Treating these as one job is how a backup identity becomes dangerous.

A useful split has four action classes:

1. Capture reads the named production source and creates a new recovery artifact.
2. Deposit writes that artifact to one defined destination with the required retention attributes.
3. Recover reads a selected artifact and restores it only to an allowed target.
4. Administer changes retention, holds, vault settings, replication, encryption access, or deletion rules.

Capture and deposit can often run unattended. Recover should usually require a fresh approval because it can move a large amount of sensitive information into a new runtime. Administration should sit outside the agent's ordinary path entirely, except for narrowly defined and separately approved emergency procedures.

Do not confuse cleanup with capture. Automated expiration is desirable, but it does not mean the writer must carry permanent deletion authority. Prefer lifecycle rules owned by the recovery system or a separate retention role. If the platform forces the writer to delete its own old copies, give it permission only within a short, dedicated staging area. Replicate completed copies into a protected destination where that writer cannot delete or alter them.

This distinction catches a familiar failure. A database agent runs an export every hour. Its role can write to recovery/incoming/, list the prefix, and delete old files. Months later, the storage team changes the destination to recovery/. The old prefix limit disappears, and the agent now has delete rights across current recovery data. The job still reports green. The problem appears only when somebody needs the copy.

Use separate names that express intent in the permissions themselves. backup-writer, backup-retention-admin, and restore-operator are clearer than a single role named backup-service. Clear names do not enforce access, but they make a review much harder to wave through.

Separate credentials must lead to separate authorities

Creating two API keys for the same broad administrator role does nothing useful. Separate credentials only matter when they lead to authorities that differ in a way an attacker, a broken script, or an agent cannot collapse.

Start with a small access map. Write down every source, destination, credential, and destructive operation. Include the cloud account or subscription, not just the storage path. The map should answer questions that application diagrams routinely skip:

• Which identity creates the copy?
• Which identity can delete an existing copy before its scheduled expiry?
• Which identity can reduce retention or invoke a bypass?
• Which identity can alter replication, a vault lock, or the encryption keys needed to restore?
• Which identity can restore data into a network that reaches production?

If the answer to three or more questions is the same service identity, separate the actions before you add more automation.

A practical minimum design looks like this:

The identities can live in the same provider at first, but they must have distinct roles, credentials, and approval paths. Better separation puts the protected destination in another account controlled by a different administrative group. Stronger separation adds an independent identity provider or a recovery environment that production administrators cannot silently modify. Do not delay the initial split while you wait for the perfect account structure.

A separate account still fails if a production super-admin can assume the recovery administrator role whenever it wants. That may be acceptable for a small organization that has no other option, but call it what it is: administrative separation by convention. It is weaker than a boundary that needs a separate person, hardware factor, or external approval.

Immutability stops one class of deletion, not every recovery failure

Immutable storage protects existing copies against modification or deletion for a retention period. It does not prove that new copies continue to arrive, that they contain the right data, that encryption keys remain available, or that an operator can restore them. It is an important control, but it cannot carry the whole recovery plan.

CISA's StopRansomware guidance tells organizations to keep backups offline and to ensure backup data is encrypted and immutable. That recommendation is sound because attackers frequently pursue backup systems after they reach production. The guidance should not be read as permission to put a vault administrator credential in the same automation that operates the application.

Amazon S3 Object Lock makes the distinction concrete. In compliance mode, a protected object version cannot be overwritten or deleted by any user, including the account root user, before its retention date. In governance mode, a caller with s3:BypassGovernanceRetention can override protection if it explicitly makes a bypass request. Amazon's documentation even notes that the console automatically includes that bypass header for a caller who has permission.

Governance mode is useful, particularly while you learn what retention period you can afford. It is not the same as a hard boundary if your agent's credential holds bypass authority. Do not hand an agent s3:BypassGovernanceRetention because somebody wants a cleanup task to stop failing. Fix the lifecycle design instead.

Compliance retention has a cost: an incorrect retention period may preserve data longer than expected, and you cannot shorten it. Decide that deliberately. Pick retention lengths from recovery requirements, legal obligations, data sensitivity, cost, and how long it takes to discover an intrusion. A blanket setting copied from another team is not a plan.

There is another trap in versioned object storage. A simple delete request may create a delete marker rather than permanently remove an older object version. That can make a restore look broken to an operator browsing the latest view, even though the protected version still exists. Your recovery procedure must say how to identify and retrieve the required version. An undeletable object that nobody can find during an incident is only partly protected.

Put destructive backup controls behind a different approval

Routine backup creation should be boring. A new agent session may need approval to use a backup writer credential, but the individual capture and deposit calls should not demand human attention if they are constrained to their intended source and destination. People quickly learn to approve repetitive prompts without reading them.

Destructive or irreversible actions deserve a different treatment. Removing a recovery copy, shortening retention, changing a replication destination, disabling a vault lock, exporting decryption material, or restoring into a production-reachable environment should stop for a fresh, specific approval. The approval should describe the requested action in plain language and identify the credential, target, and impact.

Bad approval: Allow backup operation?

Useful approval: Allow backup-retention-admin to remove 14 expired recovery points from archive-vault? This action cannot be reversed for copies not under immutable retention.

The wording matters because it lets the reviewer reject a request that is technically authorized but operationally wrong. A production agent that suddenly asks to modify a recovery vault should look strange before anybody has to interpret an IAM action name.

Approval is not a substitute for permissions. A reviewer can approve the wrong request, especially at 2 a.m. when an incident has already filled the screen with alerts. The permission boundary must make dangerous actions unavailable to the normal writer. Approval then addresses the smaller set of actions that remain intentionally possible.

Sallyport fits this pattern when a coding agent needs to call a backup API or run an SSH-based backup task: keep the credential in the app's vault, allow the normal session only to use the writer credential, and mark recovery administration credentials for approval on every use. The agent receives the result of the action, not the secret itself.

That arrangement also handles a practical concern with long-running agents. Do not approve a process once and assume every future action from it deserves the same trust. A new process should establish its own session. A sensitive credential should require its own per-call consent even when the process already has routine backup access.

A policy fragment should make the bad call impossible

Permission reviews become much clearer when you test the request you never want to succeed. The following example illustrates an S3-style deposit role. It can place a new object only under an assigned prefix. It has no DeleteObject, no retention bypass, no bucket-policy authority, and no permission to read back the archive.

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "WriteNewRecoveryArtifacts",
      "Effect": "Allow",
      "Action": ["s3:PutObject"],
      "Resource": "arn:aws:s3:::recovery-archive-prod/incoming/database/*"
    },
    {
      "Sid": "DenyRecoveryAdministration",
      "Effect": "Deny",
      "Action": [
        "s3:DeleteObject",
        "s3:DeleteObjectVersion",
        "s3:BypassGovernanceRetention",
        "s3:PutObjectRetention",
        "s3:PutObjectLegalHold",
        "s3:PutBucketPolicy",
        "s3:DeleteBucket"
      ],
      "Resource": "*"
    }
  ]
}

This is an example shape, not a policy to paste blindly. A real deployment may need encryption headers, multipart upload actions, a bucket restriction, or a separate role for a replication service. The point is that no one reading it should wonder whether this identity can delete a recovery artifact. The answer is visible.

Run a negative test after every permission change. With the deposit credential active, a delete should fail in a way your logs can record:

aws s3api delete-object \
  --bucket recovery-archive-prod \
  --key incoming/database/2026-07-22/backup.sql.zst

A healthy result resembles this:

An error occurred (AccessDenied) when calling the DeleteObject operation:
User is not authorized to perform: s3:DeleteObject on resource:
arn:aws:s3:::recovery-archive-prod/incoming/database/2026-07-22/backup.sql.zst

Then test the action the role should perform. Upload a harmless canary artifact, verify that its retention settings appear as expected, and confirm that the agent cannot alter them afterward. A permission design that has not survived a negative test is still a diagram.

Do not put the retention administrator's credential beside this writer credential "for emergencies." Emergencies create the most pressure to use the broad secret. Keep that credential in a separate store and require a separate approval route or a second operator.

Recovery can leak data even when deletion is blocked

Teams usually describe restore as the safe direction of travel. It is safer than deleting a backup, but it is still a privileged action. Restoring a customer database into an ad hoc development environment can expose production secrets, personal data, payment records, or internal tokens. Restoring an image into a network with production connectivity can also introduce stale credentials and unsafe services.

A restore action needs constraints that fit the system being recovered. At minimum, specify the source recovery point, the destination account or project, the target network, and the intended access group. If the platform supports a restore sandbox, use it. If it does not, build a restricted destination with outbound controls and no route to production by default.

Separate restore approval from backup approval for another reason: the recovery operator may need access to read protected data, while the backup writer should not. That reading authority can be more sensitive than write authority. An export job may produce encrypted data without ever seeing the plaintext. A restore job often materializes it.

A good restore drill answers more than "did the command finish?" It checks whether:

• The selected timestamp matches the incident scenario.
• The artifact can be decrypted with the intended recovery identity.
• The application starts with isolated configuration.
• The expected records and schema appear.
• The temporary restored environment is destroyed or retained under its own access rules.

Do not run drills only against yesterday's easiest copy. Pick older recovery points, different data sources, and conditions where an object version or encryption key must be selected deliberately. The hard restore is the one that teaches you whether the runbook describes reality.

The failure path usually starts with a harmless request

Consider a coding agent that has access to a production database and a cloud provider CLI. It receives a request to reduce storage costs after a test environment accumulated old exports. The agent lists a broad storage prefix, finds large objects, and submits a deletion command. The engineer intended to clear staging files. The credential reaches both staging and the archive because a wildcard was convenient.

If the archive uses ordinary versioning, the command may add delete markers and make current copies disappear from a normal listing. If the archive uses governance retention and the credential includes bypass authority, the request may remove versions outright. If it uses compliance retention, the delete request fails, which is exactly the kind of failure you want.

Now change only the credential design. The agent can use a staging-cleaner credential for the test path and a deposit-only credential for new archive writes. Neither credential can list or delete protected recovery objects. The request fails before it can turn into an incident. A recovery administrator can later inspect the cost issue with a separate approval and decide whether lifecycle rules need adjustment.

This is why broad storage credentials are worse than they appear. The command itself may be ordinary. The destructive outcome comes from an identity that crosses boundaries the task never needed to cross.

Keep an audit record of both allowed and denied recovery actions. The denied record proves that a control fired. The allowed record tells you which process used which credential, what it touched, and when. In Sallyport, the session journal and activity journal give those two levels of evidence, and sp audit verify can check the hash chain offline without vault access. That is useful after an incident because a backup report that says "success" cannot explain a suspicious administrative request.

Backup reporting should expose authority, not just success

Most backup dashboards answer whether a job completed and how much data it copied. Add a second view: which authority made the change, what operation it attempted, and whether the system accepted it. Recovery security fails quietly when all reports reduce authorization to a green or red job badge.

For each backup run, record the source identifier, destination identifier, artifact version or recovery point ID, credential class, retention state, and result. For each denied action, record enough context to investigate without storing secrets or sensitive payloads. The record should make it possible to distinguish a full backup from a failed upload, a routine lifecycle expiration from a manual deletion, and a blocked destructive call from a missing permission that needs review.

Do not grant broad read access to recovery storage merely so an agent can produce a detailed report. Many providers offer metadata endpoints, inventory reports, or narrowly scoped status calls. If your agent must read a manifest, write a separate manifest that contains identifiers, checksums, capture times, and retention status rather than a catalog of customer data.

A useful weekly review asks four questions:

1. Did every expected source create a recoverable artifact?
2. Did any identity attempt a retention, deletion, or replication change?
3. Can the current writer credential access recovery administration actions?
4. Did a restore drill prove that a selected older copy can start the application in isolation?

If the team cannot answer those questions from its records, improve the records before assuming the backup design works.

Build the boundary before you automate more of it

Start with the credential already held by your agent or CI job. Remove its ability to delete recovery copies, bypass retention, change holds, change replication, alter vault settings, and administer the recovery account. Then create a deposit identity that can write only where it should write. That one change closes a common path from a bad instruction to irreversible damage.

Next, move lifecycle and retention controls to a distinct administrative identity. Add immutable retention for the recovery window that your organization can defend. Put restore into a constrained destination and require a specific approval before sensitive data reappears anywhere outside its normal runtime. Finally, run a drill that attempts both the expected backup and the forbidden deletion.

Your backup design is ready for agents when an agent can create a recovery copy without possessing the authority to destroy recovery. Until then, automation only makes the shared mistake faster.