# SUGILITE: Creating Multimodal Smartphone Automation by Demonstration

Introduces SUGILITE, which combines spoken instructions, demonstrated Android actions, and UI hierarchies to generate reusable automation scripts. Its interface includes context checking and repairs for unfamiliar situations and changing app interfaces.

- Catalog URL: https://awesomecua.github.io/resources/url-392a602c660ffa91be46/
- Original source: https://doi.org/10.1145/3025453.3025483
- Kind: paper
- Review: curated
- Scope: core
- Environments: android
- Topics: mobile-navigation, recording, human-in-the-loop, cross-app
- Evidence checked: 2026-10-17T09:54:39Z
- ID: url:392a602c660ffa91be46

## Limitations

The annotation was reviewed against retained primary publication sources. Experiments have not been independently reproduced. Related to later APPINITE and PUMICE work but distinguished here by its 2017 three-author publication and base demonstration-automation contribution.

- Record JSON: https://awesomecua.github.io/resources/url-392a602c660ffa91be46/record.json

## Authors

Toby Jia-Jun Li, Amos Azaria, Brad A. Myers

## Citation exports

- BibTeX: https://awesomecua.github.io/resources/url-392a602c660ffa91be46/citation.bib
- CSL JSON: https://awesomecua.github.io/resources/url-392a602c660ffa91be46/citation.csl.json

## Publication record

- Title: SUGILITE: Creating Multimodal Smartphone Automation by Demonstration
- Authors: Toby Jia-Jun Li, Amos Azaria, Brad A. Myers
- Publication date (recorded precision): 2017
- Publication source: https://doi.org/10.1145/3025453.3025483
- Publication BibTeX: https://awesomecua.github.io/resources/url-392a602c660ffa91be46/publication-citation.bib
- Publication CSL JSON: https://awesomecua.github.io/resources/url-392a602c660ffa91be46/publication-citation.csl.json
- venue: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems
- pages: 6038-6049
- doi: 10.1145/3025453.3025483
- date basis: Supplied year precision: 2017. Publications: SUGILITE, CHI 2017 Only publication year is asserted; event dates are kept separate.
- identity basis: No same-work record located in the full active catalog. Similarity screening was not used as automatic identity approval. The raw archive and exclusions are additionally checked during normalization.
- notes: Related to later APPINITE and PUMICE work but distinguished here by its 2017 three-author publication and base demonstration-automation contribution. Publication month/day are not independently identified by the retained sources.
- Publication evidence: https://www.cs.cmu.edu/~NatProg/sugilite.html, https://toby.li/files/TobyLi-CHI2017-Sugilite.pdf

## License declarations

- code: not recorded
- data: not recorded
- document: not recorded
- weights: not recorded

## Primary evidence

- https://www.cs.cmu.edu/~NatProg/sugilite.html (retrieved 2026-10-17T09:53:22Z; SHA-256 32ee05ff1a91ed4a22d4bc156a462872aa4e28f5c79abfbce5dbf3655f412033)
- https://toby.li/files/TobyLi-CHI2017-Sugilite.pdf (retrieved 2026-10-17T09:54:39Z; SHA-256 cc23c19284f0a558c4b99b8b051abca8ed61e274e9e1f1fed1ae91487f4368ad)
- https://toby.li/files/TobyLi-CHI2017-Sugilite.pdf (retrieved 2026-10-17T09:54:39Z; SHA-256 cc23c19284f0a558c4b99b8b051abca8ed61e274e9e1f1fed1ae91487f4368ad)
- https://www.cs.cmu.edu/~NatProg/sugilite.html (retrieved 2026-10-17T09:53:22Z; SHA-256 32ee05ff1a91ed4a22d4bc156a462872aa4e28f5c79abfbce5dbf3655f412033)