# Harnessing Webpage UIs for Text-Rich Visual Understanding

Builds MultiUI by generating instructions from webpage accessibility trees and pairing them with screenshots for multimodal training. Evaluation examines web-interface understanding, including VisualWebBench and Mind2Web, and transfer to other text-rich visual tasks.

- Catalog URL: https://awesomecua.github.io/resources/arxiv-2410-13824/
- Original source: https://arxiv.org/abs/2410.13824
- Kind: paper
- Review: curated
- Scope: core
- Environments: web
- Topics: data-generation, screen-understanding, grounding
- Evidence checked: 2026-10-16T06:31:35Z
- ID: arxiv:2410.13824

## Limitations

Reviewed against the retained original arXiv abstract. Experimental results and artifact availability have not been independently reproduced.

- Record JSON: https://awesomecua.github.io/resources/arxiv-2410-13824/record.json

## Authors

Junpeng Liu, Tianyue Ou, Yifan Song, Yuxiao Qu, Wai Lam, Chenyan Xiong, Wenhu Chen, Graham Neubig, Xiang Yue

## Citation exports

- BibTeX: https://awesomecua.github.io/resources/arxiv-2410-13824/citation.bib
- CSL JSON: https://awesomecua.github.io/resources/arxiv-2410-13824/citation.csl.json

## Publication record

- Title: Harnessing Webpage UIs for Text-Rich Visual Understanding
- Authors: Junpeng Liu, Tianyue Ou, Yifan Song, Yuxiao Qu, Wai Lam, Chenyan Xiong, Wenhu Chen, Graham Neubig, Xiang Yue
- Publication date (recorded precision): 2025-05-01
- Publication source: https://proceedings.iclr.cc/paper_files/paper/2025/hash/2da53cd1abdae59150e35f4693834f32-Abstract-Conference.html
- Publication BibTeX: https://awesomecua.github.io/resources/arxiv-2410-13824/publication-citation.bib
- Publication CSL JSON: https://awesomecua.github.io/resources/arxiv-2410-13824/publication-citation.csl.json
- venue: International Conference on Learning Representations
- volume: 2025
- pages: 17971-18005
- date basis: Explicit publisher citation_publication_date: 2025-05-01. Preserve supplied precision; do not substitute conference event dates.
- identity basis: Same work: normalized title, ordered author correspondence and the specific contribution in the complete publisher abstract were reviewed against the retained arXiv identity.
- notes: Publisher proceedings edition; retained separately from the arXiv snapshot.
- Publication evidence: https://proceedings.iclr.cc/paper_files/paper/2025/hash/2da53cd1abdae59150e35f4693834f32-Abstract-Conference.html

## Recorded arXiv snapshot

- Version: 3
- Version source: https://arxiv.org/abs/2410.13824v3
- First submission: 2024-10-17T17:48:54Z
- Version updated: 2024-11-06T08:29:22Z
- Abstract SHA-256: 2a3bcd26dff4f1d21a8a2772929c0c97cd1cc180a7606b0744e630dabae94609
- Abstract hash normalization: sha256-utf8-whitespace-collapsed-v1

## License declarations

- code: not recorded
- data: not recorded
- document: not recorded
- weights: not recorded

## Primary evidence

- https://export.arxiv.org/api/query?id_list=2410.00467%2C2410.00689%2C2410.02810%2C2410.02907%2C2410.05243%2C2410.06703%2C2410.08164%2C2410.09006%2C2410.09024%2C2410.11843%2C2410.11871%2C2410.11872%2C2410.13232%2C2410.13757%2C2410.13824%2C2410.13825%2C2410.13886%2C2410.14803%2C2410.15164%2C2410.16464%2C2410.17236%2C2410.17401%2C2410.17520%2C2410.17883%2C2410.18603%2C2410.18963%2C2410.18967%2C2410.19054%2C2410.19100%2C2410.19461%2C2410.19609%2C2410.20745%2C2410.22552%2C2410.23218%2C2410.24024%2C2411.00820%2C2411.02337%2C2411.02391%2C2411.04468%2C2411.04890%2C2411.06559%2C2411.07618%2C2411.10323%2C2411.13451%2C2411.13591%2C2411.15004%2C2411.17465%2C2411.18279%2C2412.00402%2C2412.01268&max_results=50 (retrieved 2026-10-16T06:31:35Z; SHA-256 b3e67984f59a1b05ccd946eb54e2d37b0bb78ab0ca796fbb6d544793dafea3e0)
- https://proceedings.iclr.cc/paper_files/paper/2025/hash/2da53cd1abdae59150e35f4693834f32-Abstract-Conference.html (retrieved 2026-10-18T04:10:02Z; SHA-256 899100e1d0cad8d949c9075f62ab3506995587b01aab3981ac4e26eaf6d88339)
- https://export.arxiv.org/api/query?id_list=2410.00467%2C2410.00689%2C2410.02810%2C2410.02907%2C2410.05243%2C2410.06703%2C2410.08164%2C2410.09006%2C2410.09024%2C2410.11843%2C2410.11871%2C2410.11872%2C2410.13232%2C2410.13757%2C2410.13824%2C2410.13825%2C2410.13886%2C2410.14803%2C2410.15164%2C2410.16464%2C2410.17236%2C2410.17401%2C2410.17520%2C2410.17883%2C2410.18603%2C2410.18963%2C2410.18967%2C2410.19054%2C2410.19100%2C2410.19461%2C2410.19609%2C2410.20745%2C2410.22552%2C2410.23218%2C2410.24024%2C2411.00820%2C2411.02337%2C2411.02391%2C2411.04468%2C2411.04890%2C2411.06559%2C2411.07618%2C2411.10323%2C2411.13451%2C2411.13591%2C2411.15004%2C2411.17465%2C2411.18279%2C2412.00402%2C2412.01268&max_results=50 (retrieved 2026-10-16T06:31:35Z; SHA-256 b3e67984f59a1b05ccd946eb54e2d37b0bb78ab0ca796fbb6d544793dafea3e0)