# ActionBert: Leveraging User Actions for Semantic Understanding of User Interfaces

Pretrains ActionBert on user interaction traces to combine visual, linguistic, and UI-structure features. It evaluates the resulting UI representations on tasks including icon classification and description-based component retrieval.

- Catalog URL: https://awesomecua.github.io/resources/arxiv-2012-12350/
- Original source: https://arxiv.org/abs/2012.12350
- Kind: paper
- Review: curated
- Scope: core
- Environments: unspecified
- Topics: screen-understanding, grounding, action-prediction
- Evidence checked: 2026-10-17T09:11:47Z
- ID: arxiv:2012.12350

## Limitations

Reviewed against the exact-version official arXiv abstract page; its normalized abstract SHA-256 matches the retained bibliographic snapshot. The original API response body was not locally available for this review. Experimental results and artifact availability have not been independently reproduced.

- Record JSON: https://awesomecua.github.io/resources/arxiv-2012-12350/record.json

## Authors

Zecheng He, Srinivas Sunkara, Xiaoxue Zang, Ying Xu, Lijuan Liu, Nevan Wichers, Gabriel Schubiner, Ruby Lee, Jindong Chen, Blaise Agüera y Arcas

## Citation exports

- BibTeX: https://awesomecua.github.io/resources/arxiv-2012-12350/citation.bib
- CSL JSON: https://awesomecua.github.io/resources/arxiv-2012-12350/citation.csl.json

## Recorded arXiv snapshot

- Version: 2
- Version source: https://arxiv.org/abs/2012.12350v2
- First submission: 2020-12-22T20:49:52Z
- Version updated: 2021-01-25T20:37:39Z
- Abstract SHA-256: 51fdddb9a3d8a6090875719a51b22008b1f43bc38c6cfe1713a49a52f7e8434c
- Abstract hash normalization: sha256-utf8-whitespace-collapsed-v1

## License declarations

- code: not recorded
- data: not recorded
- document: not recorded
- weights: not recorded

## Primary evidence

- https://export.arxiv.org/api/query?id_list=1511.03745%2C1602.02261%2C1809.04860%2C1811.06521%2C1811.07871%2C1812.03426%2C1901.11528%2C1909.00031%2C1912.03098%2C2003.00431%2C2004.14973%2C2006.16241%2C2007.04428%2C2010.09670%2C2010.12083%2C2012.12350%2C2104.08244%2C2105.11941%2C2106.09422%2C2106.13500%2C2107.05911%2C2108.03298%2C2108.05308%2C2110.10661%2C2201.04100%2C2201.11676%2C2203.10897%2C2204.05239%2C2205.10816%2C2206.08853%2C2206.12839%2C2206.15407%2C2208.06032%2C2208.08080%2C2209.14981%2C2210.01379%2C2210.03735%2C2210.08073%2C2210.10275%2C2210.11435%2C2211.07912%2C2211.09527%2C2212.13326%2C2301.10677%2C2301.13779%2C2302.12173%2C2303.00001%2C2303.11366%2C2303.12027%2C2304.03442&max_results=50 (retrieved 2026-10-17T07:57:26Z; SHA-256 3e89566cdb4521a8b3b0d99c514c3401645eda5b5c0de9720eb726b1ffe0a11f)
- https://arxiv.org/abs/2012.12350v2 (retrieved 2026-10-17T09:11:47Z; SHA-256 139375513dafe5d7cfd0f5843b2a3465294e2b54051cf1c964a0d0751166a469)