MacAgentBench
676 macOS tasks across 25 applications, each run in its own Docker container and scored by deterministic rule-based checks with multiple checkpoints.
AI-assisted editorial review against a primary source. This is a catalog annotation, not an independent reproduction or endorsement.
What is this resource?
- Resource type
- Benchmarks
- Environment
- desktop, macos
- Topics
- evaluation
- Record publication date
- Not recorded
- Evidence retrieved
- 2026-10-16T06:35:59Z
- Resource ID
- github:jetastra/macagentbench
What licenses are recorded?
Code, training data, weights, and documents can have different terms. A blank declaration means unknown, not unrestricted use.
- Code
- MIT (repository LICENSE header; not a weights/data license)
- Data
- Not recorded
- Document
- Not recorded
- Weights
- Not recorded
Evaluation protocol
- Focus
- macOS task completion, cross-application workflows
- Setting
- interactive
- Observations
- Not recorded
- Actions / outputs
- GUI interactions, command-line operations
- Evaluation
- Deterministic rule-based evaluators with intermediate checkpoints.
Scale and counting units
- 676 tasks — documented MacAgentBench suite source
- 25 applications — documented application coverage source
The benchmark evaluates multiple agent frameworks and GUI/CLI workflows. Framework skills and interfaces affect the task setting; the inspected README does not define one universal observation representation.
Compare source-reviewed profiles
Recorded specifications
- leaderboard url
- https://jetastra.github.io/MacAgentBench/
Explore related topics
Related resources
Primary evidence and verification
Source retrieval is separate from experiment reproduction. Use the original publication or publisher page when citing claims about the resource.
- editorial primary review
Retrieved 2026-10-16T06:35:59Z. Supports: name, kind, summary, environments, topics, scope, relations, details.benchmark_protocol, details.leaderboard_url.
Content fingerprint
f1f8c0cd5d984e4026abea858b54f0f1427c018cad7e75322e744f1e070225a9 - editorial primary review
Retrieved 2026-10-16T06:32:03Z. Supports: details.benchmark_protocol, relations.
Content fingerprint
2adad07a9ec7189aea25e48c76fa28206eda97f8974d5745cadda9529103bb1a - license header
Retrieved 2026-10-16T06:41:59Z. Supports: licenses.code.
Content fingerprint
adf69997028988ba1eec34666afa119e007771dfc8e830c5dc50ad650896e897
Suggest a correction
Open a source-backed correction. Include this record ID and the original source. Read our editorial policy and coverage boundaries.