Computer Use & GUI Agents

Agents that operate desktops, phones and apps.

Browser & desktop agents

microsoft/magentic-ui - Experimental web-browsing agent that works across the browser.
anthropics/claude-quickstarts - Includes the reference computer-use demo.
alibaba/page-agent - In-page JavaScript GUI agent that controls web interfaces with natural language.
web-infra-dev/midscene - Vision-based GUI agent for browser, mobile and desktop automation and testing.
microsoft/fara - Family of computer use agent models from Microsoft.
Skyvern-AI/skyvern - Automate browser based workflows with AI.
yuruotong1/autoMate - Like Manus, Computer Use Agent(CUA) and Omniparser, we are computer-using agents.AI-driven local.
OpenAdaptAI/OpenAdapt - Compiles a demonstrated GUI task into a program that reports VERIFIED only if an independent.
openinterpreter/openinterpreter - A coding agent for open models like Kimi K3 and GLM 5.3.
microsoft/UFO - UI-focused agent for Windows OS interaction.
bytedance/UI-TARS-desktop - Desktop app for the UI-TARS GUI agent model.
trycua/cua - Open-source computer-use stack: drivers, cross-OS VM fleets, models, and benchmarks.
AmberSahdev/Open-Interface - Control Any Computer Using LLMs.
mediar-ai/terminator - playwright for windows computer use.
Jeomon/Windows-Use - An AI Agent that interacts with Windows OS at GUI level.
CursorTouch/Windows-MCP - MCP Server for Computer Use in Windows.
OthersideAI/self-operating-computer - A framework to enable a multimodal model to operate a computer.
openclaw/Peekaboo - Peekaboo is a macOS CLI & optional MCP server that enables AI agents to capture screenshots.
peakmojo/macos-visual-agent - On-device screen understanding for macOS.
screenpipe/screenpipe - YC (S26) | Open Computer History | Record your screen continuously locally and provide.
showlab/ShowUI-Aloha - Human-taught Computer-use Agent Designed for Real Windows and MacOS Desktops.
askui/python-sdk - Enable AI to control your desktop, mobile and HMI devices.
OpenGVLab/ScaleCUA - Open-sourced computer-use agent that operates across Windows, macOS, Ubuntu, and Android.
simular-ai/Agent-S - Open agentic framework that uses computers like a human.
TurixAI/TuriX-CUA - Computer-use agent that controls desktop apps from natural language.
e2b-dev/open-computer-use - Computer-use agent running open-source LLMs in an E2B desktop sandbox.
lahfir/agent-desktop - Rust CLI giving agents reliable desktop control through accessibility APIs.
iFurySt/open-codex-computer-use - Open-source alternative to Codex computer use, with an MCP server.
webbrain-one/webbrain - Open-source AI browser agent for Chrome and Firefox.

GUI grounding models & research

microsoft/OmniParser - A simple screen parsing tool towards pure vision based GUI agent.
showlab/ShowUI - Vision-language-action model for GUI agents.
xlang-ai/OpenCUA - [NeurIPS 2025 Spotlight] OpenCUA: Open Foundations for Computer-Use Agents.
OSU-NLP-Group/UGround - [ICLR'25 Oral] UGround: Universal GUI Visual Grounding for GUI Agents.
THUDM/SCALE-CUA - Open-source framework for computer use agents: VeriGen verifiable task synthesis, online RL.
mlfoundations/Gelato - 🍨 Gelato — From Data Curation to Reinforcement Learning: Building a Strong Grounding Model.
UCSB-AI/Screen-Point-and-Read - Code repo for "Read Anywhere Pointed: Layout-aware GUI Screen Reading with Tree-of-Lens.
likaixin2000/ScreenSpot-Pro-GUI-Grounding - GUI Grounding for Professional High-Resolution Computer Use.
inclusionAI/UI-Venus - General-purpose foundation GUI agent for mobile, web, and desktop using screenshots as input.
GUI-Libra/GUI-Libra - Official code for NeurIPS 2026 paper "GUI-Libra: Training Native GUI Agents to Reason and Act.
Tongyi-MAI/MAI-UI - Foundation GUI agent models for real-world mobile and desktop tasks.
ZJU-REAL/ClawGUI - Framework to train, evaluate and deploy GUI agents with online RL.

Mobile agents

minitap-ai/mobile-use - Agent framework that operates real Android and iOS apps.
X-PLUG/MobileAgent - Mobile-Agent: The Powerful GUI Agent Family.
zai-org/Open-AutoGLM - An Open Phone Agent Model & Framework. Unlocking the AI Phone for Everyone.
OpenBMB/AgentCPM-GUI - AgentCPM-GUI: An on-device GUI agent for operating Android apps, enhancing reasoning ability.
mobile-next/mobile-mcp - MCP server for automating iOS and Android devices, emulators and simulators.

Benchmarks & environments

ServiceNow/BrowserGym - Gym environment for building and evaluating web automation agents.
web-arena-x/webarena - Realistic web environment benchmark for autonomous web agents.
google-research/android_world - Official Google Research Android environment and benchmark for autonomous GUI agents.
microsoft/WindowsAgentArena - Scalable Windows OS platform for testing and benchmarking multimodal AI agents.
google-deepmind/android_env - RL research on Android devices.
xlang-ai/OSWorld - Benchmark for multimodal agents acting in real computer environments.

RPA

chelslava/rpaforge - Open source Python RPA studio with visual process designer, recorder, debugger, and orchestrator.
iflytek/astron-rpa - Agent-ready RPA suite with out-of-the-box automation tools for individuals and enterprises.
aisingapore/TagUI - Free RPA tool by AI Singapore.
open-rpa/openrpa - Free Open Source Enterprise Grade RPA.
botcity-dev/botcity-framework-core-python - BotCity Framework - Python.
tebelorg/RPA-Python - Python package for doing RPA.
robocorp/rpaframework - Open-source RPA libraries and tools designed for use with Robot Framework and Python.
oculix-org/Oculix - OculiX — Visual Automation IDE. If you can see it, you can automate it.
intra2net/guibot - A tool for GUI automation using a variety of computer vision and display control backends.
robotframework/robotframework - Generic automation framework for acceptance testing and RPA.

Desktop automation libraries

SeleniumHQ/selenium - The W3C WebDriver standard for browser automation that GUI agents build on.
autokey/autokey - AutoKey, a desktop automation utility for Linux and X11.
AutoHotkey/AutoHotkey - AutoHotkey - macro-creation and automation-oriented scripting utility for Windows.
Jmgr/actiona - Actiona 3: cross-platform automation tool.
pywinauto/pywinauto - Windows GUI Automation with Python (based on text properties).
enigo-rs/enigo - Cross platform input simulation in Rust.
yinkaisheng/Python-UIAutomation-for-Windows - Python 3 wrapper of Microsoft UIAutomation for MFC, WinForms, WPF, Qt, and browsers.
go-vgo/robotgo - Go-native cross-platform library for RPA, GUI automation, automated testing, and computer use.
FlaUI/FlaUI - UI automation library for .Net.
jordansissel/xdotool - fake keyboard/mouse input, window management, and more.
ReimuNotMoe/ydotool - Generic command-line automation tool.
boppreh/keyboard - Hook and simulate global keyboard events on Windows and Linux.
ra1nty/DXcam - High-performance Python screen capture library for Windows using the Desktop Duplication API.
kwhat/libuiohook - A multi-platform C library to provide global keyboard and mouse hooks from userland.
BoboTiG/python-mss - Ultra-fast cross-platform screenshot module in pure Python using ctypes.
autopilot-rs/autopilot-rs - A simple, cross-platform GUI automation module for Rust.
octalmage/robotjs - Node.js Desktop Automation.

Mobile device automation

honeynet/droidbot - A lightweight test input generator for Android. Similar to Monkey, but with more intelligence.
openatx/uiautomator2 - Android Uiautomator2 Python Wrapper.
appium/appium-mcp - Appium MCP on Steroids!.
mobile-dev-inc/Maestro - Painless E2E Automation for Mobile and Web.
devicelab-dev/maestro-runner - Fast mobile UI test automation for Android, iOS, React Native, Flutter & Expo. Open-source.
alibaba/tidevice - tidevice can be used to communicate with iPhone device.
doronz88/pymobiledevice3 - Pure python3 implementation for working with iDevices (iPhone, etc...).
Genymobile/scrcpy - Display and control your Android device.
danielpaulus/go-ios - This is an operating system independent implementation of iOS device features. You can run UI.
barry-ran/QtScrcpy - Android real-time display control software.
libimobiledevice/libimobiledevice - A cross-platform protocol library to communicate with iOS devices.
budtmo/docker-android - Android in docker solution with noVNC supported, video recording, mcp server and AI-agent.
AirtestProject/Airtest - UI Automation Framework for Games and Apps.
appium/appium - Cross-platform automation for mobile apps on the WebDriver protocol.
callstack/agent-device - Mobile app automation and verification for AI coding agents.

VMs & remote desktop

waydroid/waydroid - Waydroid uses a container-based approach to boot a full Android system on a regular GNU/Linux.
sickcodes/Docker-OSX - Run macOS VM in a Docker! Run near native OSX-KVM in Docker! X11 Forwarding! CI/CD for OS X.
kholia/OSX-KVM - Run macOS on QEMU/KVM. With OpenCore + Monterey + Ventura + Sonoma support now! Only.
quickemu-project/quickemu - Quickly create and run optimised Windows, macOS and Linux virtual machines.
novnc/websockify - Websockify is a WebSocket to TCP proxy/bridge. This allows a browser to connect to any.
TigerVNC/tigervnc - High performance, multi-platform VNC client and server.
apache/guacamole-server - The Apache Guacamole proxy daemon (guacd), C API (libguac), and protocol support.
apache/guacamole-client - The Apache Guacamole web application, official extensions, and JavaScript library.
FreeRDP/FreeRDP - FreeRDP is a free remote desktop protocol library and clients.
Ylianst/MeshCentral - A complete web-based remote monitoring and management web site. Once setup you can install.
FreeRDP/Remmina - Mirror of https://gitlab.com/Remmina/Remmina The GTK+ Remmina Remote Desktop Client.
rustdesk/rustdesk - Open-source remote desktop application designed for self-hosting, an alternative to TeamViewer.
neutrinolabs/xrdp - xrdp: an open source RDP server.
kasmtech/KasmVNC - Modern VNC Server and client, web based and secure.
feschber/lan-mouse - mouse & keyboard sharing via LAN.
deskflow/deskflow - Share a single keyboard and mouse between multiple computers.
LizardByte/Sunshine - Self-hosted game stream host for Moonlight.
veyon/veyon - Cross-platform computer monitoring and classroom management.
any1/wayvnc - A VNC server for wlroots based Wayland compositors.
dwservice/agent - DWService agent for Linux, Mac and Windows.
selkies-project/selkies - Low-Latency Accelerated Web Remote Desktop Streaming Platform for Self-Hosting, Containers.
Xpra-org/xpra - Persistent remote applications for X11; screen sharing for X11, MacOS and MSWindows.
rdesktop/rdesktop - 🚨 rdesktop is in need of a new maintainter. Please see the home page for more details. 🚨.
novnc/noVNC - HTML5 VNC client for remote desktop access from any browser.