# Spider2-V

Visual agent evaluation on data-science and engineering workflows.

- Catalog URL: https://awesomecua.github.io/resources/github-xlang-ai-spider2-v/
- Original source: https://github.com/xlang-ai/Spider2-V
- Kind: benchmark
- Review: curated
- Scope: core
- Environments: web, desktop
- Topics: evaluation, workflow
- Evidence checked: 2026-10-16T06:35:58Z
- ID: github:xlang-ai/spider2-v

## Limitations

Inclusion does not establish reproducibility or endorsement.

- Record JSON: https://awesomecua.github.io/resources/github-xlang-ai-spider2-v/record.json

## Structured profile

- actions: PyAutoGUI mouse and keyboard actions
- evaluation: Task-specific evaluation functions are invoked after agent execution.
- focus: data-science workflows, multi-application task completion
- observations: screenshots, accessibility trees, set-of-marks views
- setting: interactive

### Count observations

- 494 task definitions — evaluation_examples/test_all.json (computed) (computed; source: https://raw.githubusercontent.com/xlang-ai/Spider2-V/HEAD/evaluation_examples/test_all.json)
- 20 applications — README task-suite coverage (reported; source: https://raw.githubusercontent.com/xlang-ai/Spider2-V/HEAD/README.md)

Counted unique task IDs in the full manifest. Verbose versus abstract instructions and account versus non-account tasks form different subsets. The Ubuntu VM and external-account requirements are part of the protocol; no accounts or environments were provisioned.

## License declarations

- code: Apache-2.0 — https://raw.githubusercontent.com/xlang-ai/Spider2-V/HEAD/LICENSE
- data: not recorded
- document: not recorded
- weights: not recorded

## Primary evidence

- https://raw.githubusercontent.com/xlang-ai/Spider2-V/HEAD/README.md (retrieved 2026-10-16T06:35:58Z; SHA-256 e3edf30ab45da16382b46d8c542607c1bbbfcf077b6df57a2decf938fc84ff0f)
- https://raw.githubusercontent.com/xlang-ai/Spider2-V/HEAD/README.md (retrieved 2026-10-16T06:35:58Z; SHA-256 e3edf30ab45da16382b46d8c542607c1bbbfcf077b6df57a2decf938fc84ff0f)
- https://raw.githubusercontent.com/xlang-ai/Spider2-V/HEAD/evaluation_examples/test_all.json (retrieved 2026-10-17T02:06:30Z; SHA-256 338dd38ba7047b0cf6dbc68f3f7b9374f962708fa2dadcc8d126c232dfc31746)
- https://raw.githubusercontent.com/xlang-ai/Spider2-V/HEAD/LICENSE (retrieved 2026-10-16T06:41:59Z; SHA-256 a8d1d88a0f9cce3cdbff73b03a80dc5e5d69e3ba2443946c268815a550c9e0ec)