Weekly agent security review: a practical 35-minute routine

A weekly agent security review should take less than an hour, produce a small number of decisions, and leave behind evidence that someone can inspect later. If it turns into a hunt through thousands of entries with no clear question, the design has already failed.

I have watched teams make the same mistake with automated work: they collect activity records because they know they should, then only open them after an uncomfortable surprise. By then, the useful context has gone cold. The engineer who launched the run has moved to another task, the temporary credential is gone, and nobody can say whether that strange command was a legitimate repair or the first sign of a bad permission boundary.

A weekly pass solves a narrower problem. It catches access drift while the people and tasks are still identifiable. It also forces a distinction that gets blurred far too often: a record that an agent was allowed to act is not a record that its actions made sense.

A weekly review catches drift before it becomes normal

Weekly review works because agent access tends to change in small, forgettable increments. Someone adds a token to unblock a test. A coding task expands into a deployment change. An agent retries an API call until a fallback path succeeds. None of these events necessarily deserves an incident response. Together, over several weeks, they can create an access pattern nobody deliberately approved.

Reviewing every event in real time sounds safer, but most teams cannot maintain the attention it needs. Reviewers begin approving or dismissing records by pattern. They stop asking why an agent that was editing documentation connected to a production host. That is approval fatigue wearing a security badge.

Waiting for a quarterly review fails for the opposite reason. A quarter contains too many runs, too many changed repositories, and too many expired memories. You end up counting events instead of understanding them.

Set a fixed weekly window and review the previous seven days at the same time. Pick a time when the person reviewing can contact the people who ran unusual jobs. Friday afternoon works for some teams; Monday morning works better if weekend automation needs attention. The calendar choice matters less than keeping it stable.

The review should answer five questions:

• Which new agent processes received authority?
• Which actions differed from the task or the normal destination?
• Which failures indicate a broken integration or probing behavior?
• Which sessions did a human revoke, and did revocation actually stop use?
• Which credentials need an owner decision before their next use?

Do not add a sixth question merely because a dashboard can display it. A weekly routine survives when it has a narrow contract. It dies when it becomes a generic security meeting with logs on one screen.

NIST SP 800-92, Guide to Computer Security Log Management, makes a point that remains useful here: organizations need defined processes for analyzing logs, not merely a place to store them. Agent records make that point sharper because an agent can act quickly and repeatedly. Storage gives you evidence. A routine gives that evidence a chance to change a decision.

Start with new sessions, not individual calls

Start by reviewing new sessions because a session is the unit where authority begins. You want to know which process obtained permission, what identity it presented, when the run began, and whether the run had a plausible purpose.

A session review is not an inventory exercise. Do not read a list, recognize a familiar developer name, and move on. A signed process identity tells you something useful about the process, but it does not tell you that the process was launched for a sensible task. Treat process identity as evidence of origin, not evidence of intent.

For every new session, answer these questions in a review note:

1. Who initiated or owns the run?
2. What repository, ticket, maintenance task, or investigation justified it?
3. Which credential classes could the run use?
4. Did the session end when the task ended?
5. Did another session appear that repeats the same job with a different identity?

The last question catches a failure that teams often miss. An engineer sees a tool fail under a restricted account, reruns it through another agent process, and gets the result. The activity record may show two ordinary sessions. The security meaning is different: the first boundary did its job, while the second run may have bypassed the reason for that boundary.

Mark a session for follow-up if it has no recognizable owner, no task reference, an unusual duration, or authority unrelated to its stated job. “Unusual duration” does not mean every long run is suspicious. Large refactors and slow test suites run long. A session that remains active long after its human context has disappeared deserves attention because stale authority is easy to forget.

Keep a small allowlist of recurring automation by purpose, not by a vague label such as “trusted.” For example, a scheduled dependency update may legitimately contact a package registry and open pull requests. That description gives a reviewer something testable. “Trusted coding bot” gives them nothing.

An unusual command needs context before it needs blame

An unusual SSH command or HTTP request is evidence to investigate, not a verdict. Reviewers get this wrong in both directions. Some dismiss odd commands because the agent was authorized. Others treat every unfamiliar command as hostile. Both reactions make the log less useful.

Build the expected context from the task. A request to read a staging deployment status may fit a release investigation. The same request during a task to adjust a README does not fit without an explanation. A command that archives build output may be ordinary. A command that archives a home directory, reads shell history, or changes remote startup files asks for a closer look.

For SSH activity, compare the command against four boundaries:

• The host the task should touch.
• The account and directory that task should require.
• The kind of change the task permits.
• The expected consequence if the command succeeds.

The fourth boundary matters. git status on a build host has little consequence. A command that edits a service definition, changes file ownership, or creates a scheduled job changes future behavior. Those calls should have a specific task reference and an accountable human.

HTTP records need the same treatment, though the clues differ. Look at the destination, request method, path, response class, and request volume. A new GET against an expected service may be routine. A burst of failed authorization responses, attempts against administrative paths, or a write request to a service unrelated to the task needs an explanation.

Do not create a huge list of forbidden strings and call it review. Agents can call legitimate tools in inappropriate contexts, and an innocent command can appear alarming without its arguments. The reviewer needs enough surrounding evidence to tell what the agent was attempting.

A useful finding reads like this: “Session S-184 ran a repository maintenance task. It used SSH on the deployment host and changed a service configuration. The task record did not include deployment work. The session owner confirmed the change was an accidental command selection. We revoked the session and restored the prior configuration.” That gives another reviewer facts, an explanation, and a completed action.

A weak finding reads: “Suspicious command observed.” That sentence creates anxiety and leaves the next reviewer to reconstruct the entire event.

Failed calls show both breakage and boundary testing

Failed calls deserve their own pass because failures carry a different kind of signal than successful actions. A successful write can change a system. A failure can reveal that an agent tried to reach something it should never have considered.

First separate ordinary integration failure from suspicious repetition. An expired token, a changed API path, a network timeout, or a provider rate limit will produce failures during normal work. The fix may be operational, not security related. Record the failure trend, identify the owner, and repair it before an agent learns to work around a broken route.

Then look for patterns that change the interpretation:

• The same rejected call repeats many times without a useful backoff or task change.
• The agent tries adjacent paths after an authorization rejection.
• A run changes destinations after a denial rather than reporting the denial.
• A failed call targets a host, service, or account outside the task boundary.
• A failure appears immediately before a successful action through a broader credential.

That final pattern is where weak credential design hides. Suppose an agent tries to update a deployment through a narrow token and receives an authorization error. It then uses a general operations token and succeeds. The logs may show a “recovered” task. The review should call it a scope failure. The narrow token correctly denied the request; the broad token hid a mismatch between the task and the allowed access.

Do not tell teams to rotate a credential every time an API request returns 401 or 403. That advice is popular because it feels decisive. It is also wasteful. Rotation does not fix a missing scope, a bad endpoint, or an agent repeatedly selecting the wrong action. It can make the next review harder by replacing the evidence trail with a new credential that has the same bad permissions.

Instead, classify a failed call as one of four outcomes: expected operational failure, configuration defect, task boundary violation, or possible credential misuse. The reviewer should write why they chose the category. If the evidence cannot support a category, ask the session owner while the run is still fresh.

Revocation must close the route that caused concern

A revoked session should stop an active process from using its granted authority, but it does not automatically repair every related risk. The weekly review needs to check the revocation event itself and the surrounding access routes.

For each revoked session, record the trigger. A human may revoke because the task ended, because an approval was accidental, because the process identity looked wrong, or because the run behaved unexpectedly. These reasons lead to different repairs. A normal end-of-task revocation may need no further action. An unexpected process identity may require investigation of the workstation and the launch path.

Then check for post-revocation activity. Any call after revocation needs an explanation. It may come from a timestamp misunderstanding, an independently authorized process, or a defect in how the reviewer linked records. Do not assume the worst, but do not hand-wave it away. The point of revocation is to make an authority boundary observable.

Also check for parallel authority. An agent can lose one session and still act through another active process, a different credential, an open SSH connection, or a separate automation account. The review does not need to prove that no alternate route exists across the entire company. It should determine whether the same task could continue through an obvious route that was left open.

Write a revocation record in plain language:

Review date: 2025-03-07
Session: [session reference]
Reason: SSH command exceeded the approved maintenance task
Action: session revoked
Post-revocation calls: none observed
Related credential: reviewed, no rotation required
Owner follow-up: update maintenance run instructions

Use your actual date and references. The template matters because it forces the reviewer to state whether they checked what happened after revocation. “Revoked” by itself describes an action, not the result.

Avoid turning every revocation into a disciplinary event. If engineers expect punishment for stopping a run, they will hesitate until they can prove intent. Revocation exists to stop an uncertain action quickly. The later review decides whether the cause was a bad prompt, a mistaken approval, a credential design problem, or misconduct.

Rotation starts with ownership and scope

Credentials due for rotation should appear in the weekly review as decisions waiting for an owner, not as a panic list. A token has a rotation date, a system owner, a stated purpose, and a scope. If any of those are missing, the credential is already harder to manage than it should be.

Create a small rotation queue from credentials used during the review window. For each one, capture the owner, service, intended use, next rotation date, and whether the week’s activity still matches that purpose. You do not need an elaborate database to begin. A durable record with named owners beats an impressive inventory nobody updates.

Prioritize rotation when one of these conditions applies:

• The credential is past its required rotation date.
• The owner cannot explain a recent use.
• The credential has more permissions than the task needs.
• The credential was used after an access concern or unknown process event.
• The team cannot identify the service account or human responsible for it.

Rotation without a cutover plan causes avoidable outages. Before replacing a credential, identify the agent runs, scripts, and integrations that use it. Issue the replacement with the minimum scope the current task needs. Test the intended action. Move the known consumers. Then retire the old credential and verify that no new activity uses it.

Do not treat “we have not seen it lately” as proof that a credential is unused. Some maintenance jobs run monthly or only during an incident. Check the owner and the documented purpose before removal. If neither exists, disable it in a controlled window and watch for the resulting failure. That is often the fastest honest answer.

There is another distinction worth preserving: rotation reduces the useful lifetime of a secret, while scope limits what that secret can do. Teams often use frequent rotation to compensate for broad permissions. It does not compensate. A newly issued credential with excessive access is still excessive access.

Verify the evidence before you interpret it

A weekly judgment is only as trustworthy as the record behind it. Exported screenshots and manually copied rows are convenient, but they are poor evidence if someone needs to determine whether they were edited later.

Sallyport records agent runs and individual calls from one encrypted, hash-chained audit log, while showing them in separate session and activity journals. That separation is useful because it lets a reviewer move from a process approval to the concrete calls made during that run without confusing the two records.

Run the available offline integrity check before you begin a formal review or when you preserve evidence after a concerning event:

sp audit verify

The command verifies the audit chain over ciphertext and does not need the vault key. A successful verification says the chain is intact according to the recorded hashes. It does not prove that an authorized human made a wise approval, that the destination was appropriate, or that the credential scope was sensible. Integrity and judgment are separate jobs.

If verification reports a problem, stop treating the affected export as settled evidence. Preserve the files and record the exact command result, then investigate the storage and application state. Do not “clean up” records to make the report look normal. You need to know whether the issue came from corruption, an incomplete copy, or interference before you draw conclusions from the journal.

For ordinary weekly reviews, retain the verification result beside the review note when your team needs a defensible trail. For low-risk local experimentation, you may choose a lighter practice. The decision should be explicit. Quietly skipping verification during the one week that contains a serious finding is the pattern you want to avoid.

The review needs a fixed 35-minute operating rhythm

A short review works when you reserve time for decisions, not when you promise to “look at the logs” after other work. The following routine is enough for a team with a manageable number of agent runs. Increase the time only after the volume or risk actually demands it.

1. Spend five minutes verifying the audit record and setting the date range. Open the prior review note so unresolved items are visible.
2. Spend ten minutes on new sessions. Match each unfamiliar process or unusually long run to an owner and a task.
3. Spend ten minutes on unusual successful actions and failed calls. Pull the surrounding context before classifying any event.
4. Spend five minutes on revoked sessions. Check the trigger, post-revocation activity, and obvious parallel authority.
5. Spend five minutes on credential rotation. Assign an owner and due date for every item that needs action.

Keep the output to one review record with three headings: findings, decisions, and open owners. A finding is an observed fact. A decision says what you will do. An open owner names the person who must finish something. Mixing those categories creates notes full of vague verbs such as “monitor” and “review.”

Here is a concise format that holds up well:

Period: [start time] to [end time]
Reviewer: [name]
Audit verification: pass or follow-up required

Findings
- [record reference] [observed event and context]

Decisions
- [action taken and reason]

Open owners
- [person] will [specific action] by [date]

A weekly review that produces no findings can still be good work, but write what you checked. “No issues” tells nobody whether you examined session origin, revocations, failures, or rotation. A few concrete lines give next week’s reviewer a baseline.

Do not hold the review hostage to perfect attendance. One informed reviewer can complete the evidence pass and assign questions to owners afterward. Waiting for a meeting room full of people is how a seven-day routine becomes a six-week gap.

The record should change access decisions

The review has failed if it only creates a report. Each recurring finding should change one of four things: the task instruction, the approval boundary, the credential scope, or the automation itself.

Repeated failed calls to an expected endpoint may need a repaired integration. An agent that keeps selecting deployment work during code review may need a narrower task instruction and no deployment credential. A recurring unknown session may require changes to how developers launch the agent. Repeated use of a broad credential may justify splitting it by purpose.

Sallyport can require approval for a new agent process and can require approval on every use of a selected credential. Use the second control for actions where the human should examine each attempt, rather than trying to express that judgment through a sprawling set of rules.

Do not react by adding approvals everywhere. If a reviewer sees the same harmless call every week and cannot distinguish it from a risky call, the approval design is poor. Move low-risk recurring work into a bounded credential and preserve per-use approval for actions with consequences that need a human decision.

Keep a short list of recurring findings and their repair status. When the same category appears three reviews in a row, stop treating it as an isolated observation. Someone needs to change the operating design. The log has already told you that the current boundary does not match the work.

A good first review may feel slightly awkward because it exposes missing task references, unnamed credentials, and unclear ownership. That discomfort is useful. Write down the unanswered questions, assign owners, and run the same routine next week. The goal is not a spotless journal. The goal is an agent environment where a human can still explain who acted, why it acted, what happened, and what access remains.