Can macOS UI automation testing defeat an agent approval?

An approval card is only a security control if the person, rather than another process, causes the approval. That sounds obvious until you place the card inside a normal macOS desktop where Accessibility clients can inspect controls, automation tools can activate applications, and any foreground window can compete for the next click.

Do not test this by asking whether a card looks modal or whether its button has a reassuring label. Test whether a process that already has realistic local privileges can cause a protected action without the intended person making a deliberate, informed choice. The useful outcome is not a dramatic exploit video. It is a repeatable answer to a narrower question: which input paths can cross the approval boundary, under which permissions, and what evidence proves the result?

This is a red team exercise for software you own, on a Mac and account you control. Keep the requested action harmless. Use a dummy HTTP endpoint, a nonproduction SSH host, or an action that returns a fixed test value. A security test that can accidentally spend money, delete data, or change production access is badly scoped before anyone writes a script.

Accessibility permission changes the attacker model

A process with Accessibility permission is not an ordinary background process. Apple describes Accessibility access as allowing apps to control a Mac, and macOS requires the user to grant that access in Privacy & Security settings. Apple separately describes Automation permission as permission to access and control other apps. Those are different permissions, but both belong in the threat model when an approval card lives on the desktop.

The distinction that teams routinely blur is this: an approval UI can be accessible without treating every accessibility-originated action as proof of human intent. Accessibility has to expose meaningful labels, roles, state changes, and focus so assistive technology users can operate the application. That is an inclusion requirement. It does not require the application to let a foreign process invoke the irreversible confirmation action with no local user signal.

Start the test by writing down the attacker you are actually simulating. Avoid the vague label "malware." It hides the permissions that decide the result. A useful baseline attacker is a locally running, signed helper under the same logged-in user that has already received Accessibility permission. A stronger variant also has Automation permission and can request application activation. Neither variant gets administrator credentials, a screen unlock bypass, or a modified copy of the approval application.

That boundary makes the test fair. If your conclusion is "an attacker with full control of the user account can control the user account," you learned nothing. If the conclusion is "a helper with the same permissions a user might grant to a window manager, test runner, macro tool, or assistive utility can approve a credential use," you found a design problem with a clear fix path.

Build a capability sheet before the first run:

Do not grant a permission merely because a test tool asks for it. Each additional permission changes the claim you can make afterward. Record the exact combination in the result title, such as "Accessibility-only helper cannot confirm per-call approval" rather than "approval is safe." The second statement cannot survive contact with a different desktop configuration.

Also capture the helper's bundle identifier, code-signing authority, process ID, parent process, and launch path. A good approval system should display enough process identity for a person to judge the request, but your test record needs more detail than a card can reasonably show. The test operator needs to answer whether a child process inherited a trusted-looking parent, whether a terminal wrapper launched the helper, and whether relaunching created a new process identity.

A clickable button is not proof of a weak approval

A button exposed through the macOS accessibility hierarchy is not automatically a vulnerability. AppKit expects accessibility elements to participate in that hierarchy so assistive clients can find a functional control, and Apple documents that controls expose properties such as labels, titles, frames, and whether they are enabled.

The weak design is one where invoking that exposed button is sufficient evidence for a sensitive action. That design makes a synthetic accessibility action indistinguishable from a conscious click. It also tends to fail quietly because the team tests with a mouse, sees the expected callback, and never asks what else can produce the same callback.

Make the approval decision depend on a local authorization event, not just on the button's action handler. The UI control can initiate the request, but the protected action should wait for an authorization result that records how the confirmation occurred. For an ordinary session approval, that may be a deliberate click after the application verifies that its own approval surface is active and current. For a high-risk action, require a system-mediated local authentication event, such as Touch ID where available.

Do not make the common mistake of hiding or renaming the Approve button in the accessibility tree. That punishes users who need assistive technology and does little against a capable local attacker who can still manipulate focus or invoke alternate paths. Instead, test whether the confirmation pipeline rejects a synthetic path, a stale card, an inactive card, and a request whose displayed details no longer match the action about to run.

A useful internal decision record looks like this:

{
  "request_id": "test-4d8f",
  "decision": "approved",
  "decision_method": "touch_id",
  "card_generation": 7,
  "request_generation": 7,
  "app_active": true,
  "card_frontmost": true,
  "requester_pid": 48102,
  "requester_signing_authority": "Example Development Team",
  "action_started_after_decision": true
}

The field names are less important than the separation. decision says what happened. decision_method says how it happened. The generation values stop an old card from approving a newer request after a redraw, retry, or action substitution. The activation fields record whether the user was looking at your card when the decision occurred. The final field lets the test assert the order of operations.

That separation also catches a subtle implementation failure: accepting an approval early and then showing the card as a courtesy. In that bug, every visual test passes. The protected action already started, so the card supplies theatre instead of control. Make the action executor require a decision token that only the approval subsystem creates after the relevant checks finish.

The test rig should produce harmless, comparable runs

Build one test fixture that creates a predictable approval request and one observer that collects evidence. Do not begin with a general desktop automation framework pointed at a live credential. A test rig should reduce ambiguity, not add it.

Your fixture needs a request with a stable identifier, visible details that vary per run, a short expiry, and a harmless completion effect. For example, have the protected action call a local endpoint that returns approved:test-4d8f only after the approval system releases it. If the action succeeds without a valid approval record, your test has a clear failure. If it times out or records a denial, you have a clear nonapproval result.

Use a short-lived nonce in both the card and the protected request. An approval card that says only "Allow agent access?" cannot reveal a stale-window bug. A card that says "Allow test request 4d8f to call the staging echo service?" can. Change the nonce for every run, then assert that the action result contains the same nonce.

Keep three independent clocks in the evidence:

1. The request creation time.
2. The decision time.
3. The protected action start time.

Use monotonic time inside the application where possible. Wall-clock time is useful for a human timeline but can move because of network synchronization, sleep, or manual changes. The ordering rule is simple: the action must start after a valid decision for the same request generation. A screenshot cannot prove that ordering.

Your observer should capture the active application, the focused window, the approval card's visible request ID, and the eventual action result. A screen recording helps diagnose surprising behavior, but it does not replace structured events. Screen recordings lose process identifiers, collapse fast transitions, and often hide the event that matters behind a pointer movement.

Use a test record like this for each attempt:

case=AX-invoke-approve
request=test-4d8f
helper=TestHelper pid=48102
permissions=accessibility
approval_surface=active
attempt=accessibility_action
decision=denied
protected_action=not_started
artifact=activity-event-0192

The attempt value should describe the mechanism, not the desired result. Write activate-then-click, stale-window, keyboard-focus-shift, or voiceover-navigation rather than attack-1. Six weeks later, that plain description will save someone from re-running the wrong case.

Run the same fixture first with no helper permissions. That is your control. Then enable one capability at a time. A red team result without a control run makes every failure difficult to interpret. Maybe your card was never keyboard reachable. Maybe the endpoint failed. Maybe the test helper lacked the entitlement you thought it had. The control turns those excuses into visible differences.

Test synthetic approval without turning the test into an exploit kit

The point of this case is to determine whether a permitted accessibility client can invoke the approval control and cause the protected action to run. You do not need a public-ready clicker or a pile of brittle coordinate scripts to answer it.

Use your own test helper with a narrow job: find your test application's approval window by its application identity and test-only request nonce, request the same semantic action that an accessibility client would request, then report whether the protected action completed. Do not search every window on the desktop. Do not target third-party dialogs. Do not store credentials or interact with real service accounts.

Test these paths separately because they prove different things:

• A semantic accessibility invocation of the Approve control.
• Keyboard navigation that lands on Approve followed by a synthetic activation.
• A pointer action after the helper brings the approval application forward.
• A pointer action after the helper does not bring it forward.
• A delayed action after the card expires or the request changes.

Coordinate clicking is worth keeping only as a regression test for visual overlap. It tells you whether a click at a screen position can land on the wrong control after a layout shift. It does not tell you whether a foreign process can identify and activate the intended element. Semantic UI automation is the stronger test for authorization logic; coordinates are the stronger test for layout and focus accidents.

Expect the results to differ by approval mode. A session-level approval is usually designed to authorize a recognized agent process for the life of that process. If your session approval accepts a plain click, then an Accessibility-permitted helper may be able to imitate it unless the app distinguishes the input path or binds approval to a stronger local event. That may be an accepted tradeoff for low-risk actions, but say so openly.

Per-call approval should withstand more. If the action uses a credential that can change data, publish a release, or reach a production host, a button press alone is too thin. Ask for Touch ID or another system-mediated local authentication signal. The test then becomes: can the helper cause the prompt to appear, invoke every accessible control around it, and still fail to complete the protected action without the person authenticating? That is a much cleaner security property.

Do not report "blocked" simply because the button did not respond. Report the full chain: whether the helper found the card, whether it invoked the control, whether the app recorded a decision, whether the authorization subsystem issued a token, and whether the action executor began. A helper might fail to press the button today while an alternate keyboard path succeeds tomorrow. The decision token is the boundary that matters.

Focus stealing can turn an honest click into the wrong decision

Direct scripted clicking gets attention because it looks malicious. Focus stealing is often the more realistic failure because it can convert a genuine user gesture into approval of a different request.

Apple's AppKit documentation makes clear that application activation is a request rather than a guarantee, and application activation state can change as macOS manages the desktop. Apple also provides APIs that launch or open applications with activation behavior. That means approval code must treat frontmost state as an event it observes and verifies, not a permanent fact it assumed when it drew the card.

Test the race deliberately. Show a harmless approval card with a unique nonce. Move the pointer over the card's confirmation control without clicking. Trigger your controlled helper to activate a second test application or display an innocuous test window. Then perform the intended click timing in repeated runs. Record which application was active at mouse-down and at mouse-up, which window owned the focused element, and whether the protected action started.

Do not accept "the card stayed visible" as a pass. A visible card can be inactive. It can sit behind another window. Its controls can remain drawn while keyboard focus moved elsewhere. Your approval path should re-check that the current request still owns the displayed card and that the app is active at the point it consumes the approval.

A solid focus-stealing test set includes these cases:

The phrase "require a new interaction" matters. Automatically restoring focus and accepting the original click after a brief interruption is dangerous. The person may think the click dismissed the interruption, while your application treats it as approval. Withdraw the pending decision or visibly reset it. Make the person initiate a new confirmation after your app is unquestionably active again.

Do not solve this by aggressively forcing your app to the foreground. That creates a different problem: surprise foreground behavior trains people to click through it. The card should surface clearly when a request arrives, but it should not wage a focus war with the desktop. If it loses activation at the wrong moment, it can fail closed and ask again.

Stale cards and request swaps deserve their own attack cases

A card can resist a direct synthetic click and still authorize the wrong action. This happens when the UI represents one request while the backend has moved on to another.

The familiar version is a retry bug. An agent requests an HTTP call. The card appears. The agent disconnects, reconnects, and submits a second request with slightly different headers. The UI reuses the existing card because it looks similar. A click on what the user believes is request A releases request B. The identity text can be perfect and the approval can still be wrong.

Bind the approval to an immutable request digest. Include the operation type, destination, credential identity or alias, relevant target details, agent process identity, and nonce. When any protected field changes, invalidate the current card. Do not update the label in place and retain an already-ready Approve control. Replace the request generation and make the previous UI instance incapable of completing anything.

For a test fixture, submit two requests in quick succession:

A: POST staging.example.invalid/echo
   header X-Test-Nonce: alpha-4d8f

B: POST staging.example.invalid/echo
   header X-Test-Nonce: bravo-7a21

Leave request A's card visible long enough for the test operator to identify it. Then submit B through the same agent process. Attempt confirmation through mouse, keyboard, accessibility action, and after a focus change. The only acceptable completions are: A completes with alpha-4d8f, B completes after its own fresh approval with bravo-7a21, or neither completes. If A's visible card releases B, treat it as a release-blocking authorization defect.

Do the same for a target change that looks harmless to a hurried person. An SSH action can preserve the command while changing the host. An HTTP action can preserve the host while changing the path or injected credential. Your card does not need to print every byte of a request, but it must show the details that distinguish the security decision. The backend must bind the full canonical request, not only the details you chose to display.

Expiry tests matter here too. Do not count down merely for visual effect. When a request expires, revoke the backend authorization path for that request generation. Then test an Accessibility action fired at the exact expiry boundary and a mouse-up that arrives just after expiry. The expected result is denial with a reason that tells you whether expiration, focus loss, generation mismatch, or authorization failure stopped it.

Your logs need to settle the argument after the screen recording ends

Approval tests become arguments when the only evidence is "I clicked and it worked." You need records that let another engineer reconstruct the causal chain without trusting the tester's interpretation.

Record two related streams: the agent run and the individual protected call. The run record answers who initiated the work and whether that run was later revoked. The call record answers exactly what action was requested, how the approval resolved, and whether execution began. Keep the approval events in the same ordered audit story as request creation, focus transitions, authorization outcomes, and execution.

Sallyport keeps a Sessions journal for agent runs and an Activity journal for individual calls, both projected from an encrypted hash-chained audit log. Its sp audit verify command verifies that chain offline over ciphertext without needing a vault key. That gives a test team an integrity check on the evidence instead of a mutable export that someone can quietly edit after a surprising result.

For any red team run, export or preserve these facts together:

• The test case name and permission matrix.
• The requesting process identity and its code-signing authority.
• The request digest, visible nonce, and request generation.
• The decision method, decision result, and reason for denial if denied.
• The first execution event, or explicit proof that no execution event exists.

Hash chains do not prove that the approval policy is correct. They prove something narrower and still useful: after the log was written, an investigator can check whether the record has been altered or reordered. The policy test supplies the other half by demonstrating which events the application allows to reach execution.

Make log review part of the test pass criteria. A run that correctly blocks an action but records only "cancelled" is not finished. You need to distinguish a synthetic accessibility invocation from a user cancellation, a stale generation from an expired request, and a focus-loss refusal from an app crash. Those distinctions turn one defect report into a family of regression tests.

A pass means the action stayed behind the human decision

Do not claim that an approval surface is secure because your first automation script failed. A good red team result has a small, exact scope: on a controlled Mac, with specified permissions, each tested synthetic activation and focus-interference path failed to produce a valid authorization token or start the protected action.

The strongest tests leave you with an uncomfortable but useful list of exceptions. Perhaps session approval allows a plain click because the risk is low and the user has already approved the agent process. Perhaps per-call approval requires Touch ID for selected keys. Perhaps a card cancels whenever the app loses activation, which annoys a few users but prevents a focus race. Put those decisions in the product behavior and test them. Security that only exists in a design meeting will not survive a desktop full of helpers.

Run this suite whenever you change approval rendering, add a new action channel, alter process identity handling, or refactor the code that transitions from UI decision to execution. The regression that matters will not announce itself as "Accessibility bypass." It will arrive as a harmless cleanup that moves a callback, reuses a window, or treats an old request as equivalent to a new one.

Keep the final assertion blunt: the protected action starts only after the current request receives the required local authorization. If any UI automation path can make that assertion false, the approval card is decoration until you fix it.