Evaluation
Source-reviewed research and assets tagged with Evaluation.
Membership follows reviewed tags; it does not establish a dependency between neighboring records. Dates preserve source precision. Annotations retain their original language.
Web Context Bench (Python)
Local-page benchmark comparing six browser representations on token use, factual reading and target-element selection, with deterministic answer scoring.
WebClick
A web-interface localization benchmark with screenshot instructions and target boxes, including agent browsing, human browsing and calendar cases.
WebGames
150 client-side web challenges in five capability domains, built to be easy for humans but hard for browsing agents, each revealing a password when solved.
WebGuard
Codebase, annotation toolkit, and action-level risk dataset for training and evaluating guardrail models that flag unsafe browser-agent actions before execution.
WebRewardBench
An offline preference-evaluation set for web process reward models, pairing page and trajectory context with preferred and rejected candidate decisions.
WebTailBench
Live-web task definitions and rubrics for evaluating computer-use agents, with preserved original tasks, a refreshed V2 task suite and a separate refusal set.
WebTestBench
An end-to-end web-testing benchmark where agents derive test checklists from development instructions, interact with deployed applications and report functional or constraint-related defects.
Windows Agent Arena
Windows 11 benchmark for multimodal desktop agents in a reproducible VM that runs locally or in parallel on Azure ML, with a harder mode where agents must set tasks up themselves.
WindowsAgentArena-V2
Windows benchmark revision with task/evaluator fixes and VM-initialization support, documented alongside PC-Agent-E.
WONDERBREAD
An offline benchmark of workflow documentation, quality assessment and knowledge transfer using human web-navigation demonstrations, action traces and written procedures.
WorldGUI
611 desktop GUI tasks that agents must complete from varied starting points, shipped with instructional videos and project files, plus the self-reflective WorldGUI-Agent baseline.
GUI-R1 data bundle
Training and evaluation Parquet bundles accompanying GUI-R1, including GUI grounding and mobile/web action-prediction data.
ScienceBoard evaluation trajectories
Archives of ScienceBoard agent evaluation runs, separated by agent configuration and screenshot, accessibility-tree or set-of-marks observation mode.
ScreenDrag
GUI dragging data with screen observations, prior state, and continuous pointer actions, provided as trajectory files and Parquet representations.
ScreenSpot-Pro data
High-resolution professional interface screenshots and target annotations for evaluating instruction-based GUI localization.
UI-AGILE-Data
UI-AGILE training annotations and screenshots together with converted GUI-grounding and AndroidControl evaluation files.
UIBert AppSim and RefExp annotations
Two Rico-derived UI component retrieval datasets: AppSim pairs semantically similar elements across screens, while RefExp links referring expressions to target elements.
UICrit
Mobile-interface critiques and quality ratings with region coordinates and RICO screen references, including a larger three-annotator release than the original paper dataset.
WebPRM Collection preference data
Checklist-annotated web decision records with page observations, prior actions, one preferred continuation and rejected alternatives for process-reward learning.
WebJudge 7B
A visual trajectory judge that retains selected intermediate screenshots to evaluate whether a web agent completed its task.
AdvAgent (formerly AdvWeb)
Research code for constructing and evaluating adversarial inputs to web agents, with a black-box red-team pipeline using Mind2Web data and SeeAct.
AgentStudio
Toolkit and Ubuntu-based environment for building and evaluating generalist computer-use agents through GUI and API interaction.
AndroTMem inference and evaluation code
A public Android GUI evaluation framework with action-history, summary and anchored-memory prompt modes, plus step-action and dependency-aware task scoring.
AUITestAgent
Generates mobile GUI interactions from natural-language test requirements and checks application behavior against those requirements.
Cua
Monorepo behind Cua Spaces, which gives agents full macOS or Linux desktops, plus Cua Driver for operating native apps, Lume for Apple Silicon VMs, Cua Bench and the CUA-S1 decision models.
D-PoT evaluation code
Offline prompting and evaluation code for Dynamic Planning of Thoughts on Android-in-the-Wild, including action extraction and comparison with reference actions.
DroidCall generation and evaluation code
Data-generation, fine-tuning and evaluation utilities for teaching small language models to invoke Android intents through predefined functions.
GUI Grounding Evaluation Harness
ScreenSpot-Pro evaluation harness with explicit coordinate transforms, prompt and crop-based inference comparisons, and per-sample grounding accuracy and cost records.
HUD
SDK for RL environments and evals spanning coding, browser, computer-use and robotics agents, whose environments expose SSH, MCP, CDP or VNC capabilities behind a minimal start-and-grade protocol.
LearnAct
Released inference and evaluation code for demonstration-conditioned mobile GUI agents on LearnGUI-Offline. The repository describes a multi-agent adaptation method and links the LearnGUI demonstrations.
MemGUI-Agent
MemGUI-Agent training and offline evaluation tools provide trajectory conversion, LoRA SFT templates, and step-level checks of GUI actions, memory updates and context folding.
Mobile Env
AndroidEnv-based platform for building GUI agent benchmarks and training agents, where task definition files emit instructions, rewards and episode ends from screen text, view hierarchy or log events.
MobileForge
Android GUI-agent adaptation framework combining app exploration, generated curricula and rollout feedback with hint-conditioned step-level GRPO; the documented public training path targets Qwen3-VL-8B.
Mobster engine
iOS agent engine organized around an observe, decide, verify and act loop, using WebDriverAgent screen state and phone actions such as taps, typing, swipes and app switches.
OpenMobile-2 code
Evaluation adapters and training guidance for OpenMobile-2 across Android emulators and browser-rendered mobile simulation.
ORBIT framework
Inspect-based multi-agent evaluation framework with role, scheduling and communication controls, BrowserART accessibility-tree actions, and desktop mouse, keyboard and screenshot tools.
RedTeamCUA
A controlled testing framework that combines desktop virtual machines and replicated websites to evaluate computer-use agents against indirect prompt injection.
ScienceBoard VMware environment
A prebuilt Ubuntu virtual-machine archive containing the software environment used for ScienceBoard scientific computer-use tasks.
ScreenSuite
A shared evaluation harness for GUI-model perception, offline action prediction and multi-step interaction, using simple smolagents-based agents across existing benchmarks.
SheetCopilot
Spreadsheet automation agent for Windows/Excel, with benchmark assets and a separately released headless Ubuntu evaluation backend.
STEVE / STEVE-R1 research code
Research code and linked models for step-verified computer-use training; the current repository describes STEVE-R1 and its trajectory releases.
TAG attention-driven GUI grounding
Grounding and evaluation code that aggregates a pretrained vision-language model’s attention maps to localize requested interface elements without fine-tuning.
UI Defect Mocker
Synthesizes visual UI defects from mobile screenshots and Android layout metadata to support GUI-testing experiments.
VideoCAD training and evaluation code
VideoCADFormer training, action-log preprocessing and evaluation code for learning CAD interface action sequences from screen recordings of browser-based Onshape.
Web-Shepherd inference and evaluation
Process-reward inference and benchmark-evaluation utilities that use task checklists to assess individual web-agent decisions.
WebAgent-R1
Environment setup, multi-turn reinforcement-learning and evaluation tooling for browser agents interacting with WebArena.
WebLlama
Training, evaluation and integration tooling for dialogue-guided web agents, including Llama fine-tuning workflows and BrowserGym examples built around WebLINX.
WebVLN research implementation
Training and evaluation code for question-guided website navigation that combines visual observations with web-specific HTML information.
WinDOM code
GUI-grounding training and evaluation code with same-family distillation, GRPO recipes and point-in-box scoring, using a Windows-style web environment.
Showing 661–720 of 720 resources.
Get full records and evidence through the topic export manifest. Source records can be passed to the standard RAG and citation exporters after cloning the catalog.