# Understanding Mobile GUI: from Pixel-Words to Screen-Sentences

Models mobile screens as atomic text and graphic components whose relationships are learned by a Screen Transformer. The work introduces RICO-PW annotations and evaluates metadata-free screen understanding on relation, clickability, retrieval, and app-classification tasks.

- Catalog URL: https://awesomecua.github.io/resources/arxiv-2105-11941/
- Original source: https://arxiv.org/abs/2105.11941
- Kind: paper
- Review: curated
- Scope: core
- Environments: unspecified
- Topics: screen-understanding, grounding, data-generation
- Evidence checked: 2026-10-17T09:11:47Z
- ID: arxiv:2105.11941

## Limitations

Reviewed against the exact-version official arXiv abstract page; its normalized abstract SHA-256 matches the retained bibliographic snapshot. The original API response body was not locally available for this review. Experimental results and artifact availability have not been independently reproduced.

- Record JSON: https://awesomecua.github.io/resources/arxiv-2105-11941/record.json

## Authors

Jingwen Fu, Xiaoyi Zhang, Yuwang Wang, Wenjun Zeng, Sam Yang, Grayson Hilliard

## Citation exports

- BibTeX: https://awesomecua.github.io/resources/arxiv-2105-11941/citation.bib
- CSL JSON: https://awesomecua.github.io/resources/arxiv-2105-11941/citation.csl.json

## Recorded arXiv snapshot

- Version: 2
- Version source: https://arxiv.org/abs/2105.11941v2
- First submission: 2021-05-25T13:45:54Z
- Version updated: 2022-04-14T11:00:58Z
- Abstract SHA-256: 8788e419d1a901f8ad9dd6089a12fe129baee764b1c285bbeb4ea60d7046afa2
- Abstract hash normalization: sha256-utf8-whitespace-collapsed-v1

## License declarations

- code: not recorded
- data: not recorded
- document: not recorded
- weights: not recorded

## Primary evidence

- https://export.arxiv.org/api/query?id_list=1511.03745%2C1602.02261%2C1809.04860%2C1811.06521%2C1811.07871%2C1812.03426%2C1901.11528%2C1909.00031%2C1912.03098%2C2003.00431%2C2004.14973%2C2006.16241%2C2007.04428%2C2010.09670%2C2010.12083%2C2012.12350%2C2104.08244%2C2105.11941%2C2106.09422%2C2106.13500%2C2107.05911%2C2108.03298%2C2108.05308%2C2110.10661%2C2201.04100%2C2201.11676%2C2203.10897%2C2204.05239%2C2205.10816%2C2206.08853%2C2206.12839%2C2206.15407%2C2208.06032%2C2208.08080%2C2209.14981%2C2210.01379%2C2210.03735%2C2210.08073%2C2210.10275%2C2210.11435%2C2211.07912%2C2211.09527%2C2212.13326%2C2301.10677%2C2301.13779%2C2302.12173%2C2303.00001%2C2303.11366%2C2303.12027%2C2304.03442&max_results=50 (retrieved 2026-10-17T07:57:26Z; SHA-256 3e89566cdb4521a8b3b0d99c514c3401645eda5b5c0de9720eb726b1ffe0a11f)
- https://arxiv.org/abs/2105.11941v2 (retrieved 2026-10-17T09:11:47Z; SHA-256 41c4da71e457ea23725a8070b0da64c234f03613841b95edea3a5f3c95dd8c68)