agent-desktop

View on GitHub

Agent Desktop gives any agent reliable computer use on the desktop. Built with Rust, it sees any app's real UI structure through OS accessibility trees and operates it — refs stay stable and actions stay safe to retry, instead of guessing from pixels.

★ 1.7K117 forksRustApache-2.0lahfir

Rust CLI and C-ABI library that gives agents reliable computer use on macOS, Linux, and Windows by reading native accessibility trees instead of pixels. Offers ref-stable snapshots, progressive token-saving traversal, session tracing, and CDP interop for Chromium apps.

Use Cases

Let an AI agent drive native macOS apps like Finder, Safari, Xcode, or SlackObserve app UI structure via accessibility trees instead of pixel/screenshot guessingProgressive skeleton snapshots to cut token usage in dense UIs by 78-96%Click, type, and press shortcuts by stable element refs with retry-safe actionsRun multi-agent desktop workflows with shared sessions and per-agent cursorsDrive Chromium web contents via CDP while native windows stay on accessibility pathRecord and export session traces as single-file HTML for debugging agent runsCall the C-ABI cdylib from Python, Swift, Go, Ruby, Node, or C for in-process automationStructured JSON CLI output for agent loops: snapshot, decide, act, re-observe

Built With

Language: Rust
Frameworks: clap · serde · tracing · Playwright · Puppeteer · chrome-remote-interface

agent-desktop

Use Cases

Built With

Tags