# ASIL: Replacing Screenshot-and-Click with Structured State and Semantic Actions

ASIL exposes application state as structured JSON and provides code-executable semantic actions. Its application benchmark compares this interface with screenshot control and application-native APIs, and studies supervised and reinforcement learning on the structured interface.

- Catalog URL: https://awesomecua.github.io/resources/arxiv-2608-26991/
- Original source: https://arxiv.org/abs/2608.26991
- Kind: paper
- Review: curated
- Scope: core
- Environments: unspecified
- Topics: automation, tool-integration, evaluation, reinforcement-learning, imitation-learning
- Evidence checked: 2026-10-17T12:31:35Z
- ID: arxiv:2608.26991

## Limitations

Reviewed against the complete retained official abstract. Experimental findings and release availability have not been independently reproduced or verified.

- Record JSON: https://awesomecua.github.io/resources/arxiv-2608-26991/record.json

## Authors

Rui Xie, Lu Chen

## Citation exports

- BibTeX: https://awesomecua.github.io/resources/arxiv-2608-26991/citation.bib
- CSL JSON: https://awesomecua.github.io/resources/arxiv-2608-26991/citation.csl.json

## Recorded arXiv snapshot

- Version: 1
- Version source: https://arxiv.org/abs/2608.26991v1
- First submission: 2026-08-27T11:38:19Z
- Version updated: 2026-08-27T11:38:19Z
- Abstract SHA-256: e1626d5190f145c668c9b7f363b7f84904c12daae485e63b858e440ded5ea6df
- Abstract hash normalization: sha256-utf8-whitespace-collapsed-v1

## License declarations

- code: not recorded
- data: not recorded
- document: not recorded
- weights: not recorded

## Primary evidence

- https://export.arxiv.org/api/query?id_list=2608.16211%2C2608.16391%2C2608.17067%2C2608.17975%2C2608.19741%2C2608.21500%2C2608.22301%2C2608.22510%2C2608.23035%2C2608.23320%2C2608.26238%2C2608.26546%2C2608.26991%2C2608.27260%2C2608.29814%2C2608.31076%2C2609.02292%2C2609.04173%2C2609.04298%2C2609.05776%2C2609.06059%2C2609.07529%2C2609.08149%2C2609.08418%2C2609.09187%2C2609.11561%2C2609.12533%2C2609.15096%2C2609.15544%2C2609.20732%2C2609.21293%2C2609.23986%2C2609.24971%2C2609.24997%2C2609.27455%2C2609.29647%2C2609.29964%2C2609.30798%2C2609.32835%2C2609.34428%2C2609.35002%2C2609.35232%2C2609.35394%2C2609.35581%2C2609.36071%2C2609.36352%2C2609.36777%2C2609.37017%2C2609.38923%2C2609.39222&max_results=50 (retrieved 2026-10-17T12:31:35Z; SHA-256 6499b4f4917eec0765d16d6e26a08d7edc5146a30753b19cf8218517656bb070)
- https://export.arxiv.org/api/query?id_list=2608.16211%2C2608.16391%2C2608.17067%2C2608.17975%2C2608.19741%2C2608.21500%2C2608.22301%2C2608.22510%2C2608.23035%2C2608.23320%2C2608.26238%2C2608.26546%2C2608.26991%2C2608.27260%2C2608.29814%2C2608.31076%2C2609.02292%2C2609.04173%2C2609.04298%2C2609.05776%2C2609.06059%2C2609.07529%2C2609.08149%2C2609.08418%2C2609.09187%2C2609.11561%2C2609.12533%2C2609.15096%2C2609.15544%2C2609.20732%2C2609.21293%2C2609.23986%2C2609.24971%2C2609.24997%2C2609.27455%2C2609.29647%2C2609.29964%2C2609.30798%2C2609.32835%2C2609.34428%2C2609.35002%2C2609.35232%2C2609.35394%2C2609.35581%2C2609.36071%2C2609.36352%2C2609.36777%2C2609.37017%2C2609.38923%2C2609.39222&max_results=50 (retrieved 2026-10-17T12:31:35Z; SHA-256 6499b4f4917eec0765d16d6e26a08d7edc5146a30753b19cf8218517656bb070)