Windows Agent Arena

Windows 11 benchmark for multimodal desktop agents in a reproducible VM that runs locally or in parallel on Azure ML, with a harder mode where agents must set tasks up themselves.

AI-assisted editorial review against a primary source. This is a catalog annotation, not an independent reproduction or endorsement.

What is this resource?

Resource type

Benchmarks

Environment

windows, desktop

Topics

evaluation

Record publication date

Not recorded

Evidence retrieved

2026-10-16T06:35:57Z

Resource ID

github:microsoft/windowsagentarena

What licenses are recorded?

Code, training data, weights, and documents can have different terms. A blank declaration means unknown, not unrestricted use.

Code

MIT (repository LICENSE header; not a weights/data license)

Data

Not recorded

Document

Not recorded

Weights

Not recorded

Evaluation protocol

Focus

Windows task completion, desktop workflows

Setting

interactive

Observations

screenshots, accessibility information, screen-element detections

Actions / outputs

agent-defined Windows interactions

Evaluation

Task configuration selects an evaluator and expected application or system outcome; evaluators return a reward for the resulting state.

Scale and counting units

• 154 task definitions — evaluation_examples_windows/test_all.json (computed) source

Counted unique identifiers in the inspected manifest. Normal and hard modes differ in how much task setup the agent must perform. Record the difficulty and observation/backend configuration with each result; no VM was launched.

Compare source-reviewed profiles

Explore related topics

Related resources

• documented by: Windows Agent Arena: Evaluating Multi-Modal OS Agents at Scale

Primary evidence and verification

Source retrieval is separate from experiment reproduction. Use the original publication or publisher page when citing claims about the resource.

1. editorial primary review

   Retrieved 2026-10-16T06:35:57Z. Supports: name, kind, summary, environments, topics, scope, relations, details.

   Content fingerprint

   e481abd422a4cc8b819ad84b32684e4af641e18a7c0d277dd608cfd5d3c5e6ad
2. editorial primary review

   Retrieved 2026-10-16T06:35:57Z. Supports: details.benchmark_protocol.

   Content fingerprint

   e481abd422a4cc8b819ad84b32684e4af641e18a7c0d277dd608cfd5d3c5e6ad
3. editorial primary review

   Retrieved 2026-10-17T02:06:30Z. Supports: details.benchmark_protocol.

   Content fingerprint

   c565dc02ab0049bf9710b87e2760dda8d7c92268ea78f0354952695ab59c0de3
4. editorial primary review

   Retrieved 2026-10-17T02:06:30Z. Supports: details.benchmark_protocol.

   Content fingerprint

   902fb24cf0a9e146bf6c29288e74a7254fe40294e49dd47363e0b15d65608fef
5. license header

   Retrieved 2026-10-16T06:41:57Z. Supports: licenses.code.

   Content fingerprint

   9906940f61b1f0b533fa7d99baf55178b2808fbe113ea51dfbfad8572ccd5f2b

Suggest a correction

Open a source-backed correction. Include this record ID and the original source. Read our editorial policy and coverage boundaries.