CUAVerifierBench
Human-annotated browser-agent trajectories for comparing automated task-success and process-quality verifiers against reviewer judgments.
AI-assisted editorial review against a primary source. This is a catalog annotation, not an independent reproduction or endorsement.
What is this resource?
- Resource type
- Benchmarks
- Environment
- web
- Topics
- evaluation, reward-models, web-navigation
- Record publication date
- Not recorded
- Evidence retrieved
- 2026-10-17T02:42:32Z
- Resource ID
- url:d5140d438466c1e1348a
What licenses are recorded?
Code, training data, weights, and documents can have different terms. A blank declaration means unknown, not unrestricted use.
- Code
- Not recorded
- Data
- mit (publisher model/dataset card metadata; third-party assets retain their own terms)
- Document
- Not recorded
- Weights
- Not recorded
License review notes
- Data: The benchmark's Hub dataset card declares MIT in its metadata header (license: mit). Source 1
Evaluation protocol
- Focus
- verifier-human agreement, process quality
- Setting
- offline
- Observations
- screenshots, action logs, final answers
- Actions / outputs
- success verdict, process score
- Evaluation
- Agreement metrics including Cohen’s kappa, accuracy and F1 against human annotations; review stages must be kept distinct.
Scale and counting units
- 106 trajectories — fara7b_om2w_browserbase split source
- 154 trajectories — internal split source
- 215 annotation rows — fara7b_om2w_browserbase split source
- 154 annotation rows — internal split source
Trajectory rows and reviewer rows have different grains. Internal task labels do not make the released Parquet files private. No payload data was downloaded.
Compare source-reviewed profiles
Recorded specifications
- disabled
- false
- download url
- https://huggingface.co/datasets/microsoft/CUAVerifierBench/tree/main
- gated
- false
- hub id
- microsoft/CUAVerifierBench
Explore related topics
Scope and limitations
Public Parquet configs separate trajectories from per-reviewer annotations. The card declares MIT. The two splits have different review stages: Online-Mind2Web includes blind and verifier-informed annotations, whereas the internal split is blind-only. This measures verifier agreement rather than agent task success.
Related resources
- documented by: The Art of Building Verifiers for Computer Use Agents
Primary evidence and verification
Source retrieval is separate from experiment reproduction. Use the original publication or publisher page when citing claims about the resource.
- editorial primary review
Retrieved 2026-10-17T02:42:32Z. Supports: name, kind, summary, environments, topics, scope, relations, details, licenses.data.
Content fingerprint
44d7190b72ae489297f59d6fadff635ced6dd067c845c77b04f187a7b2d7bec7 - editorial primary review
Retrieved 2026-10-17T02:42:32Z. Supports: identity, release_status, details, licenses.data.
Content fingerprint
fb1fe8a9fd6e45fc82157c65c08d5c682d08bead5196308776200ef3c39c1e41
Suggest a correction
Open a source-backed correction. Include this record ID and the original source. Read our editorial policy and coverage boundaries.