Agent Desktop test: Codex wrote and revised a TextEdit document, then got stuck at Save

The short answer

Agent Desktop lets an AI agent use apps on your computer by reading their controls and acting on those, instead of working only from screenshots. In this test Codex researched a topic in the browser, then used Agent Desktop to put a briefing into TextEdit, read it back, make a requested revision and read the revised text back again. All of that went through TextEdit’s own controls.

Saving was where it got stuck. Agent Desktop could read the Save sheet, but its checks blocked it from finishing, so Codex switched to its separate computer controls to save, close and reopen the document. The reopened file had the revised text. So this is a promising result for writing and editing inside a native app, not an unattended workflow that Agent Desktop completed on its own.

Which tool did what

Agent Desktop is from Vercel Labs. You give it to an agent you already use; it doesn’t have its own AI model. Here, Codex decided what to do and wrote the text, Agent Browser read the websites in a separate browser session, and Agent Desktop worked in TextEdit. It’s one assistant doing the job, calling separate tools for the browser and the desktop.

The test used Agent Desktop 1.2 and Agent Browser 0.27, both driven by Codex on a Mac. Newer versions may behave differently, so check the documentation for the version you install.

How it reads an app’s controls

Agent Desktop uses the operating system’s accessibility information: what is a text field, what’s in it, what a button is called and which actions are available. It puts that into a snapshot with a reference for each control, so the agent can pick the text field it needs without finding it in a screenshot.

That only works as well as the app’s accessibility information. Many apps don’t put much effort into it, although agents give developers a new reason to. And reading a control isn’t the same as being able to use it: in this test it could see the Save button but couldn’t click it.

Setup and Mac permissions

Codex installed Agent Desktop 1.2 in a local project folder and read the instructions bundled with that version. Its status check reported that Accessibility and Screen Recording were already granted to Codex.

On a Mac, Accessibility access belongs to the application that launches Agent Desktop (your terminal, editor or agent app), which is why it worked here for Codex. Screen Recording is a separate permission, needed only for screenshots. Installing Agent Desktop doesn’t give it control of your apps by itself: nothing happens until you grant those permissions and an agent calls it. The macOS page in the official docs explains how to check and grant them.

The test: browser research into TextEdit

Codex used Agent Browser to open the official docs and research how desktop control works, how it fits with browser automation and where the AI model runs, with a source link for each finding so anyone opening the finished document could check it.

Codex then used Agent Desktop to target TextEdit’s editable text area, set its contents and read the contents back from the app. The text went in through TextEdit’s controls; it wasn’t a file generated in the background and opened afterwards. Writing a text file directly would have been easier for a document like this, but the point was to see whether the agent could work through the application itself.

The revision: local control isn’t local inference

The first version said Agent Desktop runs locally. That’s true, but not specific enough: it still needs an AI model, and unless you run that model on your own machine the setup isn’t fully local. Codex, for example, wasn’t running locally here. So we asked for that to be made clearer.

The revised paragraph explains that Agent Desktop contains no model, the model belongs to whichever agent is calling it, and local desktop control doesn’t automatically mean local inference. Agent Desktop replaced the text in TextEdit and read back the revised version. We told it what to change, so it hadn’t spotted the issue itself, but it made the change inside the app and checked the text that was actually there.

Where Save got stuck, and what finished the job

Agent Desktop could read TextEdit’s native Save sheet, including the file name and the Save button, but its visibility and focus checks blocked it from finishing. It had already had trouble identifying the TextEdit window; restarting the empty app through Codex’s separate computer controls helped it reach the writing stage but didn’t fix Save. Codex also tried an unmodified build of the project’s newer source, and saving still didn’t work.

Codex then used its separate computer control tool to choose the folder, save the document, close it and open it again. The reopened file had the revised paragraph, and the text extracted from it matched the revised input. Agent Desktop put the text into the app and revised it; the other controls completed the save and the reopen. For a dependable workflow you’d want Save to work too.

The little hack: let your agent set it up

The quickest way to try Agent Desktop is the one from the video: paste this guide into the agent you already use, such as Codex or Claude, and ask it to set Agent Desktop up and test it. Keep the first run small and stay at the computer while it works:

1. Paste this guide (or its link) into Codex or Claude and ask: “Read this guide and the official Agent Desktop docs it links to, install Agent Desktop for this machine, and tell me which permissions it needs and why before you change anything.”
2. Let it run the status or permissions check. On a Mac, grant Accessibility only to the app that launches Agent Desktop, and Screen Recording only if you want screenshots.
3. Open TextEdit with a new, disposable document. Ask the agent to snapshot TextEdit, find the text area by its reference, enter a short paragraph and read it back.
4. Ask for one specific change. Have it take a fresh snapshot first rather than reusing old references, since they can change after the app does, then read the result back.
5. Ask it to save the document and watch what happens. If Save is blocked, as it was here, decide yourself whether to finish by hand or with another tool.
6. Only point it at apps and documents you’re happy for it to touch.

Documentation links

This guide is free, with no email needed. Start with the project’s own setup instructions for the version you install: the Agent Desktop repository on GitHub (https://github.com/vercel-labs/agent-desktop), the documentation overview (https://agent-browser.dev/desktop) and the quick start (https://agent-browser.dev/desktop/quick-start).

For the parts this test depended on: the macOS page covers Accessibility and Screen Recording (https://agent-browser.dev/desktop/platforms/macos), Snapshots and refs explains the control references (https://agent-browser.dev/desktop/snapshots-and-refs), and Observe, act, verify describes the check-after-every-action loop (https://agent-browser.dev/desktop/guides/observe-act-verify).

The takeaway

Agent Desktop let Codex write, read back and revise a document inside TextEdit through the app’s own controls, without screenshots. Save was blocked, so separate computer controls finished the job, and the AI model still ran wherever Codex runs. Paste this guide into your agent, keep the first test small and attended, and see which of your apps it can actually drive.