# Widget Captioning: Generating Natural Language Description for Mobile User Interface Elements

Introduces crowdsourced descriptions of mobile UI elements and studies multimodal models that generate useful widget captions.

- Catalog URL: https://awesomecua.github.io/resources/arxiv-2010-04295/
- Original source: https://arxiv.org/abs/2010.04295
- Kind: paper
- Review: curated
- Scope: core
- Environments: unspecified
- Topics: screen-understanding, grounding
- Evidence checked: 2026-10-16T06:42:42Z
- ID: arxiv:2010.04295

## Limitations

The annotation and scope were reviewed against the primary paper abstract. Experimental results and released artifacts have not been independently reproduced.

- Record JSON: https://awesomecua.github.io/resources/arxiv-2010-04295/record.json

## Authors

Yang Li, Gang Li, Luheng He, Jingjie Zheng, Hong Li, Zhiwei Guan

## Citation exports

- BibTeX: https://awesomecua.github.io/resources/arxiv-2010-04295/citation.bib
- CSL JSON: https://awesomecua.github.io/resources/arxiv-2010-04295/citation.csl.json

## Publication record

- Title: Widget Captioning: Generating Natural Language Description for Mobile User Interface Elements
- Authors: Yang Li, Gang Li, Luheng He, Jingjie Zheng, Hong Li, Zhiwei Guan
- Publication date (recorded precision): 2020-11
- Publication source: https://aclanthology.org/2020.emnlp-main.443/
- Publication BibTeX: https://awesomecua.github.io/resources/arxiv-2010-04295/publication-citation.bib
- Publication CSL JSON: https://awesomecua.github.io/resources/arxiv-2010-04295/publication-citation.csl.json
- venue: Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP)
- publisher: Association for Computational Linguistics
- pages: 5495-5510
- doi: 10.18653/v1/2020.emnlp-main.443
- date basis: Explicit publisher metadata: 2020/11; precision=month.
- identity basis: Same work: exact normalized landing title, corresponding author list, and matching specific contribution. Publisher and arXiv versions remain distinct.
- notes: The retained arXiv abstract reports 162,859 phrases; the ACL abstract reports 162,860. This publication proposal does not overwrite the arXiv annotation or assert a unified dataset count. Publisher-hosted proceedings record; distinct from the retained arXiv preprint snapshot.
- Publication evidence: https://aclanthology.org/2020.emnlp-main.443/

## Recorded arXiv snapshot

- Version: 1
- Version source: https://arxiv.org/abs/2010.04295v1
- First submission: 2020-10-08T22:56:03Z
- Version updated: 2020-10-08T22:56:03Z
- Abstract SHA-256: 68af0a0a4b605d0c968e8913232fde16e3eadf6361099e8fc3d015dcdb26302b
- Abstract hash normalization: sha256-utf8-whitespace-collapsed-v1

## License declarations

- code: not recorded
- data: not recorded
- document: not recorded
- weights: not recorded

## Primary evidence

- https://export.arxiv.org/api/query?id_list=2008.05132%2C2010.04295%2C2101.04893%2C2112.05692%2C2306.01242%2C2311.03079%2C2401.05459%2C2402.03746%2C2402.15506%2C2403.20329%2C2404.06395%2C2404.10179%2C2405.10292%2C2406.09187%2C2406.14596%2C2408.03326%2C2409.12089%2C2501.10893%2C2504.09352&max_results=19 (retrieved 2026-10-16T06:42:42Z; SHA-256 d1c1bc3fee56d124427abd2aabbcf876418633a8adf7a03f4ef0317cba6a42ae)
- https://aclanthology.org/2020.emnlp-main.443/ (retrieved 2026-10-17T09:53:48+00:00; SHA-256 750e60b4d28e5b97a46595ea2280b520bf9edcc445b9931bad225e0028935c5e)
- https://export.arxiv.org/api/query?id_list=2008.05132%2C2010.04295%2C2101.04893%2C2112.05692%2C2306.01242%2C2311.03079%2C2401.05459%2C2402.03746%2C2402.15506%2C2403.20329%2C2404.06395%2C2404.10179%2C2405.10292%2C2406.09187%2C2406.14596%2C2408.03326%2C2409.12089%2C2501.10893%2C2504.09352&max_results=19 (retrieved 2026-10-16T06:42:42Z; SHA-256 d1c1bc3fee56d124427abd2aabbcf876418633a8adf7a03f4ef0317cba6a42ae)