# Read Anywhere Pointed: Layout-aware GUI Screen Reading with Tree-of-Lens Grounding

Introduces Screen Point-and-Read and a hierarchical layout approach that interprets screen regions indicated by a user-selected point.

- Catalog URL: https://awesomecua.github.io/resources/arxiv-2406-19263/
- Original source: https://arxiv.org/abs/2406.19263
- Kind: paper
- Review: curated
- Scope: core
- Environments: cross-platform
- Topics: screen-understanding, grounding
- Evidence checked: 2026-10-16T06:31:32Z
- ID: arxiv:2406.19263

## Limitations

The annotation and scope were reviewed against the primary paper abstract. Experimental results and released artifacts have not been independently reproduced.

- Record JSON: https://awesomecua.github.io/resources/arxiv-2406-19263/record.json

## Authors

Yue Fan, Lei Ding, Ching-Chen Kuo, Shan Jiang, Yang Zhao, Xinze Guan, Jie Yang, Yi Zhang, Xin Eric Wang

## Citation exports

- BibTeX: https://awesomecua.github.io/resources/arxiv-2406-19263/citation.bib
- CSL JSON: https://awesomecua.github.io/resources/arxiv-2406-19263/citation.csl.json

## Publication record

- Title: Read Anywhere Pointed: Layout-aware GUI Screen Reading with Tree-of-Lens Grounding
- Authors: Yue Fan, Lei Ding, Ching-Chen Kuo, Shan Jiang, Yang Zhao, Xinze Guan, Jie Yang, Yi Zhang, Xin Eric Wang
- Publication date (recorded precision): 2024-11
- Publication source: https://aclanthology.org/2024.emnlp-main.533/
- Publication BibTeX: https://awesomecua.github.io/resources/arxiv-2406-19263/publication-citation.bib
- Publication CSL JSON: https://awesomecua.github.io/resources/arxiv-2406-19263/publication-citation.csl.json
- venue: Proceedings of the 2024 Conference on Empirical Methods in Natural Language Processing
- publisher: Association for Computational Linguistics
- pages: 9503-9522
- doi: 10.18653/v1/2024.emnlp-main.533
- date basis: Explicit publisher citation_publication_date: 2024/11. Preserve supplied precision; do not substitute conference event dates.
- identity basis: Same work: normalized title, ordered author correspondence and the specific contribution in the complete publisher abstract were reviewed against the retained arXiv identity.
- notes: Publisher proceedings edition; retained separately from the arXiv snapshot.
- Publication evidence: https://aclanthology.org/2024.emnlp-main.533/

## Recorded arXiv snapshot

- Version: 2
- Version source: https://arxiv.org/abs/2406.19263v2
- First submission: 2024-06-27T15:34:16Z
- Version updated: 2024-10-25T18:16:21Z
- Abstract SHA-256: 0c86d68b5c3486b099f64792d5d3290d79e25b7c82c6eb2402869b893bb22fea
- Abstract hash normalization: sha256-utf8-whitespace-collapsed-v1

## License declarations

- code: not recorded
- data: not recorded
- document: not recorded
- weights: not recorded

## Primary evidence

- https://export.arxiv.org/api/query?id_list=2406.12814%2C2406.13719%2C2406.14056%2C2406.14250%2C2406.14314%2C2406.18082%2C2406.19263%2C2407.00993%2C2407.01476%2C2407.01489%2C2407.01511%2C2407.01558%2C2407.03037%2C2407.03913%2C2407.04346%2C2407.05291%2C2407.07061%2C2407.08850%2C2407.09018%2C2407.09295%2C2407.10956%2C2407.13032%2C2407.13063%2C2407.15673%2C2407.15711%2C2407.16741%2C2407.17490%2C2407.18901%2C2407.19056%2C2407.20183%2C2408.00203%2C2408.02544%2C2408.06327%2C2408.07199%2C2408.11824%2C2408.15978%2C2409.02453%2C2409.04566%2C2409.07429%2C2409.08264%2C2409.09354%2C2409.10741%2C2409.11295%2C2409.13711%2C2409.14337%2C2409.14818%2C2409.15441%2C2409.15637%2C2409.17140%2C2409.20566&max_results=50 (retrieved 2026-10-16T06:31:32Z; SHA-256 b9a83994d8ac36ccdb606a5219511f8bb35ad3948ade4f9f3e28d386221aedfb)
- https://aclanthology.org/2024.emnlp-main.533/ (retrieved 2026-10-18T04:09:51Z; SHA-256 1d105799f5db853e801090e8ed62273cd72dd3cb9b4060f901747aace8d2406d)
- https://export.arxiv.org/api/query?id_list=2406.12814%2C2406.13719%2C2406.14056%2C2406.14250%2C2406.14314%2C2406.18082%2C2406.19263%2C2407.00993%2C2407.01476%2C2407.01489%2C2407.01511%2C2407.01558%2C2407.03037%2C2407.03913%2C2407.04346%2C2407.05291%2C2407.07061%2C2407.08850%2C2407.09018%2C2407.09295%2C2407.10956%2C2407.13032%2C2407.13063%2C2407.15673%2C2407.15711%2C2407.16741%2C2407.17490%2C2407.18901%2C2407.19056%2C2407.20183%2C2408.00203%2C2408.02544%2C2408.06327%2C2408.07199%2C2408.11824%2C2408.15978%2C2409.02453%2C2409.04566%2C2409.07429%2C2409.08264%2C2409.09354%2C2409.10741%2C2409.11295%2C2409.13711%2C2409.14337%2C2409.14818%2C2409.15441%2C2409.15637%2C2409.17140%2C2409.20566&max_results=50 (retrieved 2026-10-16T06:31:32Z; SHA-256 b9a83994d8ac36ccdb606a5219511f8bb35ad3948ade4f9f3e28d386221aedfb)