# CRAB: Cross-environment Agent Benchmark for Multimodal Language Model Agents

Introduces CRAB, a framework for cross-environment agent tasks with graph-based evaluation and a benchmark spanning desktop and phone interactions.

- Catalog URL: https://awesomecua.github.io/resources/arxiv-2407-01511/
- Original source: https://arxiv.org/abs/2407.01511
- Kind: paper
- Review: curated
- Scope: core
- Environments: desktop, cross-platform
- Topics: evaluation, cross-app
- Evidence checked: 2026-10-16T06:31:32Z
- ID: arxiv:2407.01511

## Limitations

The annotation and scope were reviewed against the primary paper abstract. Experimental results and released artifacts have not been independently reproduced.

- Record JSON: https://awesomecua.github.io/resources/arxiv-2407-01511/record.json

## Authors

Tianqi Xu, Linyao Chen, Dai-Jie Wu, Yanjun Chen, Zecheng Zhang, Xiang Yao, Zhiqiang Xie, Yongchao Chen, Shilong Liu, Bochen Qian, Anjie Yang, Zhaoxuan Jin, Jianbo Deng, Philip Torr, Bernard Ghanem, Guohao Li

## Citation exports

- BibTeX: https://awesomecua.github.io/resources/arxiv-2407-01511/citation.bib
- CSL JSON: https://awesomecua.github.io/resources/arxiv-2407-01511/citation.csl.json

## Publication record

- Title: CRAB: Cross-environment Agent Benchmark for Multimodal Language Model Agents
- Authors: Tianqi Xu, Linyao Chen, Dai-Jie Wu, Yanjun Chen, Zecheng Zhang, Xiang Yao, Zhiqiang Xie (谢志强), Yongchao Chen, Shilong Liu, Bochen Qian, Anjie Yang, Zhaoxuan Jin, Jianbo Deng, Philip Torr, Bernard Ghanem, Guohao Li
- Publication date (recorded precision): 2025-07
- Publication source: https://aclanthology.org/2025.findings-acl.1113/
- Publication BibTeX: https://awesomecua.github.io/resources/arxiv-2407-01511/publication-citation.bib
- Publication CSL JSON: https://awesomecua.github.io/resources/arxiv-2407-01511/publication-citation.csl.json
- venue: Findings of the Association for Computational Linguistics: ACL 2025
- publisher: Association for Computational Linguistics
- pages: 21607-21647
- doi: 10.18653/v1/2025.findings-acl.1113
- date basis: Explicit publisher citation_publication_date: 2025/7. Preserve supplied precision; do not substitute conference event dates.
- identity basis: Same work: normalized title, ordered author correspondence and the specific contribution in the complete publisher abstract were reviewed against the retained arXiv identity.
- notes: Publisher proceedings edition; retained separately from the arXiv snapshot. Publisher spelling, initials, name annotations or author lists differ from the retained preprint.
- Publication evidence: https://aclanthology.org/2025.findings-acl.1113/

## Recorded arXiv snapshot

- Version: 4
- Version source: https://arxiv.org/abs/2407.01511v4
- First submission: 2024-07-01T17:55:04Z
- Version updated: 2025-07-20T13:42:07Z
- Abstract SHA-256: 12e5464ea0fc62bd566f19d301b887826b7ff317cfc27c86fa0cda8d3de271d9
- Abstract hash normalization: sha256-utf8-whitespace-collapsed-v1

## License declarations

- code: not recorded
- data: not recorded
- document: not recorded
- weights: not recorded

## Primary evidence

- https://export.arxiv.org/api/query?id_list=2406.12814%2C2406.13719%2C2406.14056%2C2406.14250%2C2406.14314%2C2406.18082%2C2406.19263%2C2407.00993%2C2407.01476%2C2407.01489%2C2407.01511%2C2407.01558%2C2407.03037%2C2407.03913%2C2407.04346%2C2407.05291%2C2407.07061%2C2407.08850%2C2407.09018%2C2407.09295%2C2407.10956%2C2407.13032%2C2407.13063%2C2407.15673%2C2407.15711%2C2407.16741%2C2407.17490%2C2407.18901%2C2407.19056%2C2407.20183%2C2408.00203%2C2408.02544%2C2408.06327%2C2408.07199%2C2408.11824%2C2408.15978%2C2409.02453%2C2409.04566%2C2409.07429%2C2409.08264%2C2409.09354%2C2409.10741%2C2409.11295%2C2409.13711%2C2409.14337%2C2409.14818%2C2409.15441%2C2409.15637%2C2409.17140%2C2409.20566&max_results=50 (retrieved 2026-10-16T06:31:32Z; SHA-256 b9a83994d8ac36ccdb606a5219511f8bb35ad3948ade4f9f3e28d386221aedfb)
- https://aclanthology.org/2025.findings-acl.1113/ (retrieved 2026-10-18T04:09:51Z; SHA-256 a8cf178f2faa088fd94b77d03785b8c6e2908db34b411287984066b4e5d1ef3f)
- https://export.arxiv.org/api/query?id_list=2406.12814%2C2406.13719%2C2406.14056%2C2406.14250%2C2406.14314%2C2406.18082%2C2406.19263%2C2407.00993%2C2407.01476%2C2407.01489%2C2407.01511%2C2407.01558%2C2407.03037%2C2407.03913%2C2407.04346%2C2407.05291%2C2407.07061%2C2407.08850%2C2407.09018%2C2407.09295%2C2407.10956%2C2407.13032%2C2407.13063%2C2407.15673%2C2407.15711%2C2407.16741%2C2407.17490%2C2407.18901%2C2407.19056%2C2407.20183%2C2408.00203%2C2408.02544%2C2408.06327%2C2408.07199%2C2408.11824%2C2408.15978%2C2409.02453%2C2409.04566%2C2409.07429%2C2409.08264%2C2409.09354%2C2409.10741%2C2409.11295%2C2409.13711%2C2409.14337%2C2409.14818%2C2409.15441%2C2409.15637%2C2409.17140%2C2409.20566&max_results=50 (retrieved 2026-10-16T06:31:32Z; SHA-256 b9a83994d8ac36ccdb606a5219511f8bb35ad3948ade4f9f3e28d386221aedfb)