# An Illusion of Progress? Assessing the Current State of Web Agents

Introduces Online-Mind2Web, with 300 tasks across 136 live websites, and examines how benchmark design affects assessments of web agents. It also develops an LLM judge for task completion and compares its judgments with human evaluation.

- Catalog URL: https://awesomecua.github.io/resources/arxiv-2504-01382/
- Original source: https://arxiv.org/abs/2504.01382
- Kind: paper
- Review: curated
- Scope: core
- Environments: web
- Topics: evaluation, web-navigation
- Evidence checked: 2026-10-16T06:36:54Z
- ID: arxiv:2504.01382

## Limitations

The annotation and scope were reviewed against the primary paper abstract. Experimental results and released artifacts have not been independently reproduced.

- Record JSON: https://awesomecua.github.io/resources/arxiv-2504-01382/record.json

## Authors

Tianci Xue, Weijian Qi, Tianneng Shi, Chan Hee Song, Boyu Gou, Dawn Song, Huan Sun, Yu Su

## Citation exports

- BibTeX: https://awesomecua.github.io/resources/arxiv-2504-01382/citation.bib
- CSL JSON: https://awesomecua.github.io/resources/arxiv-2504-01382/citation.csl.json

## Recorded arXiv snapshot

- Version: 4
- Version source: https://arxiv.org/abs/2504.01382v4
- First submission: 2025-04-02T05:51:29Z
- Version updated: 2025-10-08T02:22:45Z
- Abstract SHA-256: ccad667f7c5bb75e58d75d5eab5cc24c7c3610be9eed1a09079de5e23c85af04
- Abstract hash normalization: sha256-utf8-whitespace-collapsed-v1

## License declarations

- code: not recorded
- data: not recorded
- document: not recorded
- weights: not recorded

## Primary evidence

- https://export.arxiv.org/api/query?search_query=%28ti%3A%22computer+use%22+OR+ti%3A%22computer-use%22+OR+ti%3A%22GUI+agent%22+OR+ti%3A%22GUI+agents%22+OR+ti%3A%22GUI+grounding%22+OR+ti%3A%22web+agents%22+OR+ti%3A%22browser+agent%22%29+AND+submittedDate%3A%5B200001010000+TO+202610160636%5D&start=700&max_results=100&sortBy=submittedDate&sortOrder=descending (retrieved 2026-10-16T06:36:54Z; SHA-256 4953a8e0f029016aef668bce98cec4d2873a7ae49c1fe33c4311fa5eb8d24089)
- https://export.arxiv.org/api/query?search_query=%28ti%3A%22computer+use%22+OR+ti%3A%22computer-use%22+OR+ti%3A%22GUI+agent%22+OR+ti%3A%22GUI+agents%22+OR+ti%3A%22GUI+grounding%22+OR+ti%3A%22web+agents%22+OR+ti%3A%22browser+agent%22%29+AND+submittedDate%3A%5B200001010000+TO+202610160636%5D&start=700&max_results=100&sortBy=submittedDate&sortOrder=descending (retrieved 2026-10-16T06:36:54Z; SHA-256 4953a8e0f029016aef668bce98cec4d2873a7ae49c1fe33c4311fa5eb8d24089)