# MacAgentBench

676 macOS tasks across 25 applications, each run in its own Docker container and scored by deterministic rule-based checks with multiple checkpoints.

- Catalog URL: https://awesomecua.github.io/resources/github-jetastra-macagentbench/
- Original source: https://github.com/JetAstra/MacAgentBench
- Kind: benchmark
- Review: curated
- Scope: core
- Environments: desktop, macos
- Topics: evaluation
- Evidence checked: 2026-10-16T06:35:59Z
- ID: github:jetastra/macagentbench

## Limitations

Inclusion does not establish reproducibility or endorsement.

- Record JSON: https://awesomecua.github.io/resources/github-jetastra-macagentbench/record.json

## Structured profile

- actions: GUI interactions, command-line operations
- evaluation: Deterministic rule-based evaluators with intermediate checkpoints.
- focus: macOS task completion, cross-application workflows
- observations: not recorded
- setting: interactive

### Count observations

- 676 tasks — documented MacAgentBench suite (reported; source: https://raw.githubusercontent.com/JetAstra/MacAgentBench/HEAD/README.md)
- 25 applications — documented application coverage (reported; source: https://raw.githubusercontent.com/JetAstra/MacAgentBench/HEAD/README.md)

The benchmark evaluates multiple agent frameworks and GUI/CLI workflows. Framework skills and interfaces affect the task setting; the inspected README does not define one universal observation representation.

## License declarations

- code: MIT — https://raw.githubusercontent.com/JetAstra/MacAgentBench/HEAD/LICENSE
- data: not recorded
- document: not recorded
- weights: not recorded

## Primary evidence

- https://raw.githubusercontent.com/JetAstra/MacAgentBench/HEAD/README.md (retrieved 2026-10-16T06:35:59Z; SHA-256 f1f8c0cd5d984e4026abea858b54f0f1427c018cad7e75322e744f1e070225a9)
- https://export.arxiv.org/api/query?id_list=2605.06761%2C2605.07110%2C2605.07134%2C2605.09497%2C2605.11212%2C2605.12481%2C2605.12501%2C2605.12549%2C2605.13631%2C2605.14290%2C2605.14747%2C2605.15030%2C2605.15542%2C2605.15777%2C2605.16024%2C2605.16116%2C2605.16402%2C2605.17320%2C2605.18048%2C2605.18652%2C2605.18758%2C2605.19260%2C2605.19769%2C2605.21470%2C2605.25160%2C2605.25624%2C2605.25707%2C2605.26114%2C2605.27761%2C2605.28116%2C2605.28534%2C2605.28646%2C2605.28775%2C2605.29119%2C2605.29324%2C2605.29447%2C2605.29486%2C2605.29534%2C2605.29615%2C2606.06322%2C2606.09426%2C2606.11042%2C2606.22557%2C2606.23189%2C2606.24551%2C2608.17659%2C2609.22000%2C2609.34139&max_results=48 (retrieved 2026-10-16T06:32:03Z; SHA-256 2adad07a9ec7189aea25e48c76fa28206eda97f8974d5745cadda9529103bb1a)
- https://raw.githubusercontent.com/JetAstra/MacAgentBench/HEAD/LICENSE (retrieved 2026-10-16T06:41:59Z; SHA-256 adf69997028988ba1eec34666afa119e007771dfc8e830c5dc50ad650896e897)