# SPA-Bench

340 single-app and cross-app tasks over 66 Chinese and English Android apps, with plug-in support for 11 smartphone agents and automatic coarse-to-fine success detection.

- Catalog URL: https://awesomecua.github.io/resources/github-ai-agents-2030-spa-bench/
- Original source: https://github.com/ai-agents-2030/SPA-Bench
- Kind: benchmark
- Review: curated
- Scope: core
- Environments: android
- Topics: evaluation, mobile-navigation
- Evidence checked: 2026-10-16T06:35:58Z
- ID: github:ai-agents-2030/spa-bench

## Limitations

Inclusion does not establish reproducibility or endorsement.

- Record JSON: https://awesomecua.github.io/resources/github-ai-agents-2030-spa-bench/record.json

## Structured profile

- actions: agent-specific Android actions
- evaluation: A multimodal evaluator and annotated key-component matching assess completion; the framework also reports resource-use metrics.
- focus: mobile task completion, resource use, cross-app workflows
- observations: screenshot trajectories
- setting: interactive

### Count observations

- 340 tasks — complete README task collection (reported; source: https://raw.githubusercontent.com/ai-agents-2030/SPA-Bench/HEAD/README.md)
- 300 tasks — single-app subset (reported; source: https://raw.githubusercontent.com/ai-agents-2030/SPA-Bench/HEAD/README.md)
- 40 tasks — cross-app subset (reported; source: https://raw.githubusercontent.com/ai-agents-2030/SPA-Bench/HEAD/README.md)
- 66 applications — README total including third-party, Google, and system apps (reported; source: https://raw.githubusercontent.com/ai-agents-2030/SPA-Bench/HEAD/README.md)

The two task subsets partition the 340 tasks. The integrated agent and evaluator modes must accompany results. The preconfigured emulator snapshot is still marked coming soon in the inspected documentation.

## License declarations

- code: MIT — https://raw.githubusercontent.com/ai-agents-2030/SPA-Bench/HEAD/LICENSE
- data: not recorded
- document: not recorded
- weights: not recorded

## Primary evidence

- https://raw.githubusercontent.com/ai-agents-2030/SPA-Bench/HEAD/README.md (retrieved 2026-10-16T06:35:58Z; SHA-256 5294a67728072a3658cb2073bdf9a084fe3c00d87a848923dff68adb35213072)
- https://raw.githubusercontent.com/ai-agents-2030/SPA-Bench/HEAD/README.md (retrieved 2026-10-16T06:35:58Z; SHA-256 5294a67728072a3658cb2073bdf9a084fe3c00d87a848923dff68adb35213072)
- https://raw.githubusercontent.com/ai-agents-2030/SPA-Bench/HEAD/Documentation.md (retrieved 2026-10-17T02:03:23Z; SHA-256 a55290d62f8823097c73d74bac4eb08853a3caff724382e60a6a9364e7ced987)
- https://raw.githubusercontent.com/ai-agents-2030/SPA-Bench/HEAD/LICENSE (retrieved 2026-10-16T06:41:58Z; SHA-256 6962bfc8ee16c6e985e4fb680b8fe46d5b34d30f7cb8a1e61fd4c55c93120ddf)