# Pix2Struct: Screenshot Parsing as Pretraining for Visual Language Understanding

Pix2Struct pretrains an image-to-text model by reconstructing simplified HTML from masked webpage screenshots. Its variable-resolution visual representation is evaluated across several domains, including user-interface understanding.

- Catalog URL: https://awesomecua.github.io/resources/arxiv-2210-03347/
- Original source: https://arxiv.org/abs/2210.03347
- Kind: paper
- Review: curated
- Scope: adjacent
- Environments: web
- Topics: screen-understanding, data-generation
- Evidence checked: 2026-10-18T04:21:52Z
- ID: arxiv:2210.03347

## Limitations

Reviewed against the complete official arXiv abstract. Experimental results and linked artifact availability have not been independently reproduced. Adjacent visual-language pretraining: webpage screenshot parsing and explicit UI evaluation support interface understanding; the paper does not establish an autonomous GUI policy.

- Record JSON: https://awesomecua.github.io/resources/arxiv-2210-03347/record.json

## Authors

Kenton Lee, Mandar Joshi, Iulia Turc, Hexiang Hu, Fangyu Liu, Julian Eisenschlos, Urvashi Khandelwal, Peter Shaw, Ming-Wei Chang, Kristina Toutanova

## Citation exports

- BibTeX: https://awesomecua.github.io/resources/arxiv-2210-03347/citation.bib
- CSL JSON: https://awesomecua.github.io/resources/arxiv-2210-03347/citation.csl.json

## Recorded arXiv snapshot

- Version: 2
- Version source: https://arxiv.org/abs/2210.03347v2
- First submission: 2022-10-07T06:42:06Z
- Version updated: 2023-06-15T21:34:23Z
- Abstract SHA-256: 7ebb243c4690aba5f1a92b0d1e050399560998a7f304ea483dae2f451cf51ec0
- Abstract hash normalization: sha256-utf8-whitespace-collapsed-v1

## License declarations

- code: not recorded
- data: not recorded
- document: not recorded
- weights: not recorded

## Primary evidence

- https://export.arxiv.org/api/query?id_list=2210.03347%2C2403.04984%2C2405.07090%2C2406.06282%2C2409.00088%2C2409.10811%2C2411.10943%2C2503.09241%2C2503.20279%2C2503.23350%2C2505.05440%2C2505.10593%2C2505.16422%2C2505.21279%2C2505.24878%2C2506.02153%2C2506.02839%2C2506.05079%2C2506.07976%2C2508.02046%2C2511.07885%2C2511.11248%2C2512.21720%2C2603.20351%2C2604.19299&max_results=25 (retrieved 2026-10-18T04:21:52Z; SHA-256 c194d15b7e422aae48b4e05216a1f1d84471b6666718b9d3dd4c9d250c356e77)
- https://export.arxiv.org/api/query?id_list=2210.03347%2C2403.04984%2C2405.07090%2C2406.06282%2C2409.00088%2C2409.10811%2C2411.10943%2C2503.09241%2C2503.20279%2C2503.23350%2C2505.05440%2C2505.10593%2C2505.16422%2C2505.21279%2C2505.24878%2C2506.02153%2C2506.02839%2C2506.05079%2C2506.07976%2C2508.02046%2C2511.07885%2C2511.11248%2C2512.21720%2C2603.20351%2C2604.19299&max_results=25 (retrieved 2026-10-18T04:21:52Z; SHA-256 c194d15b7e422aae48b4e05216a1f1d84471b6666718b9d3dd4c9d250c356e77)