Spider2-V
Visual agent evaluation on data-science and engineering workflows.
AI-assisted editorial review against a primary source. This is a catalog annotation, not an independent reproduction or endorsement.
What is this resource?
- Resource type
- Benchmarks
- Environment
- web, desktop
- Topics
- evaluation, workflow
- Record publication date
- Not recorded
- Evidence retrieved
- 2026-10-16T06:35:58Z
- Resource ID
- github:xlang-ai/spider2-v
What licenses are recorded?
Code, training data, weights, and documents can have different terms. A blank declaration means unknown, not unrestricted use.
- Code
- Apache-2.0 (repository LICENSE header; not a weights/data license)
- Data
- Not recorded
- Document
- Not recorded
- Weights
- Not recorded
Evaluation protocol
- Focus
- data-science workflows, multi-application task completion
- Setting
- interactive
- Observations
- screenshots, accessibility trees, set-of-marks views
- Actions / outputs
- PyAutoGUI mouse and keyboard actions
- Evaluation
- Task-specific evaluation functions are invoked after agent execution.
Scale and counting units
- 494 task definitions — evaluation_examples/test_all.json (computed) source
- 20 applications — README task-suite coverage source
Counted unique task IDs in the full manifest. Verbose versus abstract instructions and account versus non-account tasks form different subsets. The Ubuntu VM and external-account requirements are part of the protocol; no accounts or environments were provisioned.
Compare source-reviewed profiles
Explore related topics
Related resources
- documented by: Spider2-V: How Far Are Multimodal Agents From Automating Data Science and Engineering Workflows?
Primary evidence and verification
Source retrieval is separate from experiment reproduction. Use the original publication or publisher page when citing claims about the resource.
- editorial primary review
Retrieved 2026-10-16T06:35:58Z. Supports: name, kind, summary, environments, topics, scope, relations, details.
Content fingerprint
e3edf30ab45da16382b46d8c542607c1bbbfcf077b6df57a2decf938fc84ff0f - editorial primary review
Retrieved 2026-10-16T06:35:58Z. Supports: details.benchmark_protocol.
Content fingerprint
e3edf30ab45da16382b46d8c542607c1bbbfcf077b6df57a2decf938fc84ff0f - editorial primary review
Retrieved 2026-10-17T02:06:30Z. Supports: details.benchmark_protocol.
Content fingerprint
338dd38ba7047b0cf6dbc68f3f7b9374f962708fa2dadcc8d126c232dfc31746 - license header
Retrieved 2026-10-16T06:41:59Z. Supports: licenses.code.
Content fingerprint
a8d1d88a0f9cce3cdbff73b03a80dc5e5d69e3ba2443946c268815a550c9e0ec
Suggest a correction
Open a source-backed correction. Include this record ID and the original source. Read our editorial policy and coverage boundaries.