# Scaling GUI Agents with Visual State Transitions

Introduces State Transition Pretraining, which jointly learns to infer actions from GUI changes and predict future states from current screens and actions. Subsequent instruction-trajectory fine-tuning tests whether these representations transfer to desktop and mobile agent tasks.

- Catalog URL: https://awesomecua.github.io/resources/arxiv-2607-24112/
- Original source: https://arxiv.org/abs/2607.24112
- Kind: paper
- Review: curated
- Scope: core
- Environments: desktop, android
- Topics: world-models, action-prediction, data-generation
- Evidence checked: 2026-10-16T06:36:32Z
- ID: arxiv:2607.24112

## Limitations

The annotation and scope were reviewed against the primary paper abstract. Experimental results and released artifacts have not been independently reproduced.

- Record JSON: https://awesomecua.github.io/resources/arxiv-2607-24112/record.json

## Authors

Xiangyan Liu, Kaixin Li, Haonan Wang, Biao Wu, Meng Fang, Longxu Dou, Chao Du, Michael Qizhe Shieh, Tianyu Pang

## Citation exports

- BibTeX: https://awesomecua.github.io/resources/arxiv-2607-24112/citation.bib
- CSL JSON: https://awesomecua.github.io/resources/arxiv-2607-24112/citation.csl.json

## Recorded arXiv snapshot

- Version: 1
- Version source: https://arxiv.org/abs/2607.24112v1
- First submission: 2026-07-27T07:54:08Z
- Version updated: 2026-07-27T07:54:08Z
- Abstract SHA-256: 4651964cbe45e4d740ed8f789753f82ae808e442d82c6598e7b3c997c09453c3
- Abstract hash normalization: sha256-utf8-whitespace-collapsed-v1

## License declarations

- code: not recorded
- data: not recorded
- document: not recorded
- weights: not recorded

## Primary evidence

- https://export.arxiv.org/api/query?search_query=%28ti%3A%22computer+use%22+OR+ti%3A%22computer-use%22+OR+ti%3A%22GUI+agent%22+OR+ti%3A%22GUI+agents%22+OR+ti%3A%22GUI+grounding%22+OR+ti%3A%22web+agents%22+OR+ti%3A%22browser+agent%22%29+AND+submittedDate%3A%5B200001010000+TO+202610160636%5D&start=100&max_results=100&sortBy=submittedDate&sortOrder=descending (retrieved 2026-10-16T06:36:32Z; SHA-256 13d3fb5811d203f2e6b80eb45d30ebc0b3cfefd88e2034ffa3b1b80687bef285)
- https://export.arxiv.org/api/query?search_query=%28ti%3A%22computer+use%22+OR+ti%3A%22computer-use%22+OR+ti%3A%22GUI+agent%22+OR+ti%3A%22GUI+agents%22+OR+ti%3A%22GUI+grounding%22+OR+ti%3A%22web+agents%22+OR+ti%3A%22browser+agent%22%29+AND+submittedDate%3A%5B200001010000+TO+202610160636%5D&start=100&max_results=100&sortBy=submittedDate&sortOrder=descending (retrieved 2026-10-16T06:36:32Z; SHA-256 13d3fb5811d203f2e6b80eb45d30ebc0b3cfefd88e2034ffa3b1b80687bef285)