# Sikuli: Using GUI screenshots for search and automation

Introduces screenshot-based GUI search and visual scripting, using image patterns to locate interface targets and direct mouse and keyboard events. It evaluates screenshot queries and demonstrates automation and interactive-help tasks.

- Catalog URL: https://awesomecua.github.io/resources/url-a4cdb2ee0d07d3980651/
- Original source: https://doi.org/10.1145/1622176.1622213
- Kind: paper
- Review: curated
- Scope: core
- Environments: unspecified
- Topics: grounding, automation, screen-understanding
- Evidence checked: 2026-10-17T09:53:21Z
- ID: url:a4cdb2ee0d07d3980651

## Limitations

The annotation was reviewed against retained primary publication sources. Experiments have not been independently reproduced. MIT identifies the retained content as an author final manuscript record. Its PDF download returned HTTP 429; no retry was made. The complete institutional abstract and bibliographic record were read.

- Record JSON: https://awesomecua.github.io/resources/url-a4cdb2ee0d07d3980651/record.json

## Authors

Tom Yeh, Tsung-Hsiang Chang, Robert C. Miller

## Citation exports

- BibTeX: https://awesomecua.github.io/resources/url-a4cdb2ee0d07d3980651/citation.bib
- CSL JSON: https://awesomecua.github.io/resources/url-a4cdb2ee0d07d3980651/citation.csl.json

## Publication record

- Title: Sikuli: Using GUI screenshots for search and automation
- Authors: Tom Yeh, Tsung-Hsiang Chang, Robert C. Miller
- Publication date (recorded precision): 2009-10
- Publication source: https://doi.org/10.1145/1622176.1622213
- Publication BibTeX: https://awesomecua.github.io/resources/url-a4cdb2ee0d07d3980651/publication-citation.bib
- Publication CSL JSON: https://awesomecua.github.io/resources/url-a4cdb2ee0d07d3980651/publication-citation.csl.json
- venue: Proceedings of the 22nd Annual ACM Symposium on User Interface Software and Technology (UIST '09)
- pages: 183-192
- doi: 10.1145/1622176.1622213
- date basis: Supplied month precision: 2009-10. Date Issued: October 2009
- identity basis: No same-work record located in the full active catalog. Similarity screening was not used as automatic identity approval. The raw archive and exclusions are additionally checked during normalization.
- notes: Institutional abstract and published-version citation supply the retained evidence; the institution identifies an author final manuscript record. Exact publication day is not supplied. Operating-system-specific experiments are not asserted.
- Publication evidence: https://dspace.mit.edu/entities/publication/604a3f79-451d-4186-a94e-d12690ffe40a

## License declarations

- code: not recorded
- data: not recorded
- document: not recorded
- weights: not recorded

## Primary evidence

- https://dspace.mit.edu/entities/publication/604a3f79-451d-4186-a94e-d12690ffe40a (retrieved 2026-10-17T09:53:21Z; SHA-256 dbaaf21982522b34857be014b39d6c43fedcda20f808efe40d04958c77243a5a)
- https://dspace.mit.edu/entities/publication/604a3f79-451d-4186-a94e-d12690ffe40a (retrieved 2026-10-17T09:53:21Z; SHA-256 dbaaf21982522b34857be014b39d6c43fedcda20f808efe40d04958c77243a5a)