Tools · curated
Activity Recorder
Windows tray utility that records primary-display screenshots, keyboard key codes and mouse events as local JPEG and JSON files for training-data collection.
Release date not recorded · desktop, windowsRESOURCE CATEGORY
Computer-use agent tools
Browse 22 reviewed tools connected to computer-use, GUI, web, and mobile agents. Each entry links to primary evidence and records its scope.
Review means AI-assisted editorial inspection, not independent reproduction. Unknown licenses and specifications remain unknown.
Tools · curated
Agent Browser Shield
Browser extension for filtering potentially unsafe agent input.
Release date not recorded · webTools · curated
agent-browser
A browser-automation command-line interface that exposes browser state and interaction commands for AI agents.
Release date not recorded · webTools · curated
AUITestAgent
Generates mobile GUI interactions from natural-language test requirements and checks application behavior against those requirements.
Release date not recorded · unspecifiedTools · curated
Browser Use WebUI
A Gradio interface for configuring and running Browser Use agents, including persistent browser sessions and interaction recordings.
Release date not recorded · webTools · curated
BrowserTrace
Local, cloud-free replay debugger that records each Browser Use step's screenshot, URL, action and model I/O as a timeline and reports the step at which a failed run diverges from a known-good one.
Release date not recorded · webTools · curated
Chrome DevTools MCP
MCP tools for browser control and inspection through Chrome DevTools.
Release date not recorded · webTools · curated
GUI Grounding Evaluation Harness
ScreenSpot-Pro evaluation harness with explicit coordinate transforms, prompt and crop-based inference comparisons, and per-sample grounding accuracy and cost records.
Release date not recorded · unspecifiedTools · curated
Mobile Harness
Portable agent instructions and integration tooling for local or cloud phone control through mobilerun-core.
Release date not recorded · android, iosTools · curated
Mobile Next MCP
An MCP server exposing mobile accessibility snapshots, screenshots and device interactions across real devices, simulators and emulators.
Release date not recorded · android, iosTools · curated
Mobilerun Portal (formerly Droidrun Portal)
An Android accessibility service that exposes UI-element inspection, device-control interfaces and screen feedback to mobile automation agents.
Release date not recorded · androidTools · curated
OculiX (SikuliX continuation)
A maintained continuation of SikuliX that locates on-screen targets with image matching and OCR, drives desktop or Android interfaces, and exposes scripting and optional MCP interfaces.
Release date not recorded · desktop, windows, macos, linux, androidTools · curated
OmniParser
Screenshot parser that detects interactable regions and captions icons so vision-language models can ground actions, released as V1.5 and V2 weights alongside OmniTool for driving a Windows 11 VM.
Release date not recorded · cross-platformTools · curated
Playwright MCP
MCP server that lets LLM clients drive Playwright browsers through structured accessibility snapshots instead of screenshots, with opt-in vision, PDF and devtools capabilities.
Release date not recorded · webTools · curated
Set of Mark
Visual-prompting toolbox that labels image regions for multimodal reference and grounding, including GUI applications.
Release date not recorded · unspecifiedTools · curated
Sikuli legacy research implementation
The original Sikuli source distribution combines screenshot targets with a Jython scripting library and IDE for visual GUI automation and testing.
Release date not recorded · desktop, macos, linux, windowsTools · curated
TAG attention-driven GUI grounding
Grounding and evaluation code that aggregates a pretrained vision-language model’s attention maps to localize requested interface elements without fine-tuning.
Release date not recorded · webTools · curated
Tarsier
Perception utilities for web agents that tag interactable elements with typed IDs an LLM can act on, and use OCR to render page screenshots as whitespace-structured text for models without vision.
Release date not recorded · webTools · curated
UI Defect Mocker
Synthesizes visual UI defects from mobile screenshots and Android layout metadata to support GUI-testing experiments.
Release date not recorded · androidTools · curated
Web-Shepherd inference and evaluation
Process-reward inference and benchmark-evaluation utilities that use task checklists to assess individual web-agent decisions.
Release date not recorded · webTools · curated
WebMarker
A browser library that labels interactive elements and their boxes for visual prompting, preserving element mappings that automation code can use to execute selected actions.
Release date not recorded · webTools · curated
Windows-MCP
A Windows MCP bridge exposing application, window, keyboard and mouse operations plus native UI-state inspection for agents.
Release date not recorded · windows, desktop, web