AppWorld
Simulated applications, user data, and state-based task checks for evaluating agents that interact through APIs and code.
AI-assisted editorial review against a primary source. This is a catalog annotation, not an independent reproduction or endorsement.
What is this resource?
- Resource type
- Benchmarks
- Environment
- unspecified
- Topics
- evaluation, tool-integration, cross-app
- Record publication date
- Not recorded
- Evidence retrieved
- 2026-10-16T10:45:03Z
- Resource ID
- github:stonybrooknlp/appworld
What licenses are recorded?
Code, training data, weights, and documents can have different terms. A blank declaration means unknown, not unrestricted use.
- Code
- Apache-2.0 (repository LICENSE header; not a weights/data license)
- Data
- Not recorded
- Document
- Not recorded
- Weights
- Not recorded
Evaluation protocol
- Focus
- interactive coding, cross-app tool use
- Setting
- interactive
- Observations
- API documentation, execution feedback
- Actions / outputs
- Python code, application API calls
- Evaluation
- State-based unit tests check task outcomes and unintended changes.
Scale and counting units
- 750 tasks — original AppWorld benchmark (2024 paper) source
- 9 applications — original AppWorld environment (2024 paper) source
This is an adjacent API/coding benchmark, not a screenshot-only task suite. Preserve the named release; task counts here refer to the original paper.
Compare source-reviewed profiles
Recorded specifications
- leaderboard url
- https://appworld.dev/appworld/leaderboard
Explore related topics
Scope and limitations
Primarily API and coding interaction; included as an adjacent comparison for GUI and hybrid agents.
Related resources
- documented by: AppWorld: A Controllable World of Apps and People for Benchmarking Interactive Coding Agents
Primary evidence and verification
Source retrieval is separate from experiment reproduction. Use the original publication or publisher page when citing claims about the resource.
- editorial primary review
Retrieved 2026-10-16T10:45:03Z. Supports: name, kind, summary, environments, topics, scope, relations, details.benchmark_protocol, details.leaderboard_url.
Content fingerprint
f4049141b6e520217395b7ccd9f27048c8e0f90331690dcbef300d2bc4ff7bdd - editorial primary review
Retrieved 2026-10-16T06:31:32Z. Supports: details.benchmark_protocol.
Content fingerprint
b9a83994d8ac36ccdb606a5219511f8bb35ad3948ade4f9f3e28d386221aedfb - license header
Retrieved 2026-10-16T10:48:42Z. Supports: licenses.code.
Content fingerprint
c71d239df91726fc519c6eb72d318ec65820627232b2f796219e87dcf35d0ab4
Suggest a correction
Open a source-backed correction. Include this record ID and the original source. Read our editorial policy and coverage boundaries.