ScienceBoard
Scientific-workflow tasks for multimodal agents in desktop software such as ChimeraX, KAlgebra, Celestia, Grass GIS and Lean 4, with apps modified to expose state for evaluation.
AI-assisted editorial review against a primary source. This is a catalog annotation, not an independent reproduction or endorsement.
What is this resource?
- Resource type
- Benchmarks
- Environment
- desktop
- Topics
- evaluation, science
- Record publication date
- Not recorded
- Evidence retrieved
- 2026-10-16T06:35:58Z
- Resource ID
- github:os-copilot/scienceboard
What licenses are recorded?
Code, training data, weights, and documents can have different terms. A blank declaration means unknown, not unrestricted use.
- Code
- MIT (repository LICENSE header; not a weights/data license)
- Data
- Not recorded
- Document
- Not recorded
- Weights
- Not recorded
Evaluation protocol
- Focus
- scientific-software desktop workflows, GUI and code-assisted scientific tasks
- Setting
- interactive
- Observations
- desktop screenshots, accessibility-tree observations, set-of-marks with accessibility tree
- Actions / outputs
- PyAutoGUI desktop actions, code/tool commands and answer/stop actions according to configured agent mode
- Evaluation
- Application-specific verifiers inspect state, files, database values, equations, compilation or answer/stop conditions. The official audit records 242 verifier entries across its VM task suite. This profile covers the released VM task definitions; Docker smoke results and Raw/Dual alternatives are separate populations.
Scale and counting units
- 169 VM task definitions — tasks/VM/**/*.json at tree 21b5eb0f; excludes tasks/Raw and tasks/Dual (computed) source
- 6 application families — VM suite: Celestia, ChimeraX, GrassGIS, KAlgebra, Lean, TeXstudio source
Manifest count 169 matches the official verifier audit. Other Raw/Dual definitions can represent alternative execution modes of the same problems and are not added. Official Docker migration documents a 20-task acceptance subset, not a full-suite performance result; legacy VMware remains. External service sign-in/rate limits, amd64 binaries and ChimeraX redistribution terms limit portability. Only code MIT is established; VM, scientific data and bundled apps need their own terms.
Compare source-reviewed profiles
Explore related topics
Related resources
- documented by: ScienceBoard: Evaluating Multimodal Autonomous Agents in Realistic Scientific Workflows
Primary evidence and verification
Source retrieval is separate from experiment reproduction. Use the original publication or publisher page when citing claims about the resource.
- editorial primary review
Retrieved 2026-10-16T06:35:58Z. Supports: name, kind, summary, environments, topics, scope, relations, details.benchmark_protocol.
Content fingerprint
4c3fe93aa740dbf4a279a246ddd836f2590a2a36f39b820abe166e38cb484577 - editorial primary review
Retrieved 2026-10-17T14:21:10Z. Supports: details.benchmark_protocol.
Content fingerprint
d276e11127abc5826bd1b2aa2caf543fcd45a8a60477774e8c8b1054067997ec - editorial primary review
Retrieved 2026-10-17T14:21:10Z. Supports: details.benchmark_protocol.
Content fingerprint
4f90270617c0c1d58f523c0a7f2e4603fa67a49560d68e2f88cfa080d764313b - editorial primary review
Retrieved 2026-10-17T14:14:07Z. Supports: details.benchmark_protocol.
Content fingerprint
3398670a517a5d1d71f420dd866493d3f8a77853e3933fd4928103f9a8be1e4f - editorial primary review
Retrieved 2026-10-17T14:35:24Z. Supports: details.benchmark_protocol.
Content fingerprint
d6a1481b453c074e34739fd29def56efa35a1dc10e856418f557f067e9ff08c5 - license header
Retrieved 2026-10-16T06:41:59Z. Supports: licenses.code.
Content fingerprint
cbb1d28b515a39c294963d4bfc24077e669a483fc5ebba9b84a727afd4187426
Suggest a correction
Open a source-backed correction. Include this record ID and the original source. Read our editorial policy and coverage boundaries.