Benchmarks

OSWorld

Execution-based evaluation of agents performing tasks in real desktop environments.

AI-assisted editorial review against a primary source. This is a catalog annotation, not an independent reproduction or endorsement.

Original source ↗Read as Markdown Record JSON

What is this resource?

Resource type: Benchmarks
Environment: desktop, linux, windows, macos
Topics: evaluation, cross-app
Record publication date: Not recorded
Evidence retrieved: 2026-10-16T06:35:57Z
Resource ID: github:xlang-ai/osworld

What licenses are recorded?

Code, training data, weights, and documents can have different terms. A blank declaration means unknown, not unrestricted use.

Code: Apache-2.0 (repository LICENSE header; not a weights/data license)
Data: Not recorded
Document: Not recorded
Weights: Not recorded

Evaluation protocol

Focus: task completion, cross-application workflows
Setting: interactive
Observations: screenshots
Actions / outputs: mouse and keyboard via PyAutoGUI
Evaluation: Task-specific execution-based evaluators inspect application or system state.

Scale and counting units

369 task definitions — test_all.json (computed) source
361 task definitions — test_nogdrive.json (computed) source

Counts were computed from unique task identifiers in the two inspected manifests. The subsets differ; record the manifest, repository revision, agent interface, and step budget with a score.

Compare source-reviewed profiles

Recorded specifications

leaderboard url: https://os-world.github.io/

Explore related topics

Evaluation Cross-app

Related resources

documented by: OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments

Connected assets

These links come from reviewed resource records. A documentation link does not imply a training-data or evaluation dependency.

uses: GUI-ReWalk · Frameworks · relationship evidence
uses: OS-Map · Benchmarks · relationship evidence

Primary evidence and verification

Source retrieval is separate from experiment reproduction. Use the original publication or publisher page when citing claims about the resource.

editorial primary review
Retrieved 2026-10-16T06:35:57Z. Supports: name, kind, summary, environments, topics, scope, relations, details.benchmark_protocol, details.leaderboard_url.
Content fingerprint
83b8684c2f4ac4f31996b8b4752195fcb6cc5429f89d92398177b9b721d787fc
editorial primary review
Retrieved 2026-10-16T06:31:28Z. Supports: details.benchmark_protocol.
Content fingerprint
be244bfae7f89ef9449b69d85aac44295eb7b5f7e5e1bbedd76df312696676a7
editorial primary review
Retrieved 2026-10-16T11:34:59Z. Supports: details.benchmark_protocol.
Content fingerprint
9ebc5187cbd727ef26c24626820076b102fff812863c640a67c467fea9542ab5
editorial primary review
Retrieved 2026-10-16T11:34:59Z. Supports: details.benchmark_protocol.
Content fingerprint
fcb9497e93a8986407345d3012c872c9c2fed253420730fbea64ccfcace67dbb
license header
Retrieved 2026-10-16T06:41:57Z. Supports: licenses.code.
Content fingerprint
a8d1d88a0f9cce3cdbff73b03a80dc5e5d69e3ba2443946c268815a550c9e0ec

Suggest a correction

Open a source-backed correction. Include this record ID and the original source. Read our editorial policy and coverage boundaries.