# Tur[k]ingBench: A Challenge Benchmark for Web Agents

Evaluates web agents on multimodal annotation tasks adapted from HTML interfaces originally built for crowdsourcing workers.

- Catalog URL: https://awesomecua.github.io/resources/arxiv-2403-11905/
- Original source: https://arxiv.org/abs/2403.11905
- Kind: paper
- Review: curated
- Scope: core
- Environments: web
- Topics: evaluation, web-navigation
- Evidence checked: 2026-10-16T06:36:58Z
- ID: arxiv:2403.11905

## Limitations

The annotation and scope were reviewed against the primary paper abstract. Experimental results and released artifacts have not been independently reproduced.

- Record JSON: https://awesomecua.github.io/resources/arxiv-2403-11905/record.json

## Authors

Kevin Xu, Yeganeh Kordi, Tanay Nayak, Adi Asija, Yizhong Wang, Kate Sanders, Adam Byerly, Jingyu Zhang, Benjamin Van Durme, Daniel Khashabi

## Citation exports

- BibTeX: https://awesomecua.github.io/resources/arxiv-2403-11905/citation.bib
- CSL JSON: https://awesomecua.github.io/resources/arxiv-2403-11905/citation.csl.json

## Publication record

- Title: TurkingBench: A Challenge Benchmark for Web Agents
- Authors: Kevin Xu, Yeganeh Kordi, Tanay Nayak, Adi Asija, Yizhong Wang, Kate Sanders, Adam Byerly, Jingyu Zhang, Benjamin Van Durme, Daniel Khashabi
- Publication date (recorded precision): 2025-04
- Publication source: https://aclanthology.org/2025.naacl-long.188/
- Publication BibTeX: https://awesomecua.github.io/resources/arxiv-2403-11905/publication-citation.bib
- Publication CSL JSON: https://awesomecua.github.io/resources/arxiv-2403-11905/publication-citation.csl.json
- venue: Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers)
- publisher: Association for Computational Linguistics
- pages: 3694-3710
- doi: 10.18653/v1/2025.naacl-long.188
- date basis: Explicit publisher citation_publication_date: 2025/4. Preserve supplied precision; do not substitute conference event dates.
- identity basis: Same work: normalized title, ordered author correspondence and the specific contribution in the complete publisher abstract were reviewed against the retained arXiv identity.
- notes: Publisher proceedings edition; retained separately from the arXiv snapshot.
- Publication evidence: https://aclanthology.org/2025.naacl-long.188/

## Recorded arXiv snapshot

- Version: 4
- Version source: https://arxiv.org/abs/2403.11905v4
- First submission: 2024-03-18T16:06:30Z
- Version updated: 2025-02-22T01:33:54Z
- Abstract SHA-256: d82424b84ebddfeb005ffae38c130c6a9c7dae7adcdc59e2b7d001284b785573
- Abstract hash normalization: sha256-utf8-whitespace-collapsed-v1

## License declarations

- code: not recorded
- data: not recorded
- document: not recorded
- weights: not recorded

## Primary evidence

- https://export.arxiv.org/api/query?search_query=%28ti%3A%22computer+use%22+OR+ti%3A%22computer-use%22+OR+ti%3A%22GUI+agent%22+OR+ti%3A%22GUI+agents%22+OR+ti%3A%22GUI+grounding%22+OR+ti%3A%22web+agents%22+OR+ti%3A%22browser+agent%22%29+AND+submittedDate%3A%5B200001010000+TO+202610160636%5D&start=800&max_results=100&sortBy=submittedDate&sortOrder=descending (retrieved 2026-10-16T06:36:58Z; SHA-256 c87e97eb397da04806d53145cf9c13be739fc657bb54dca9318c8c4ed2a36249)
- https://aclanthology.org/2025.naacl-long.188/ (retrieved 2026-10-18T04:09:50Z; SHA-256 56537fe44b4655949e8c70cc49a4ed7304a8201a12f75ddb3caa5df92d6f8e16)
- https://export.arxiv.org/api/query?search_query=%28ti%3A%22computer+use%22+OR+ti%3A%22computer-use%22+OR+ti%3A%22GUI+agent%22+OR+ti%3A%22GUI+agents%22+OR+ti%3A%22GUI+grounding%22+OR+ti%3A%22web+agents%22+OR+ti%3A%22browser+agent%22%29+AND+submittedDate%3A%5B200001010000+TO+202610160636%5D&start=800&max_results=100&sortBy=submittedDate&sortOrder=descending (retrieved 2026-10-16T06:36:58Z; SHA-256 c87e97eb397da04806d53145cf9c13be739fc657bb54dca9318c8c4ed2a36249)