CRAB
Cross-environment benchmark framework for multimodal agents.
AI-assisted editorial review against a primary source. This is a catalog annotation, not an independent reproduction or endorsement.
What is this resource?
- Resource type
- Benchmarks
- Environment
- desktop, android
- Topics
- evaluation, cross-app
- Record publication date
- Not recorded
- Evidence retrieved
- 2026-10-16T06:35:58Z
- Resource ID
- github:camel-ai/crab
What licenses are recorded?
Code, training data, weights, and documents can have different terms. A blank declaration means unknown, not unrestricted use.
- Code
- Not recorded
- Data
- Not recorded
- Document
- Not recorded
- Weights
- Not recorded
Evaluation protocol
- Focus
- cross-environment tasks
- Setting
- interactive
- Observations
- Not recorded
- Actions / outputs
- Python-defined environment actions
- Evaluation
- Graph-based evaluators measure progress through task requirements.
Scale and counting units
- 120 tasks — CRAB Benchmark-v0 (paper) source
CRAB is also an extensible framework. The 120-task count belongs to Benchmark-v0, not every environment that can be created with the framework.
Compare source-reviewed profiles
Explore related topics
Related resources
Primary evidence and verification
Source retrieval is separate from experiment reproduction. Use the original publication or publisher page when citing claims about the resource.
- editorial primary review
Retrieved 2026-10-16T06:35:58Z. Supports: name, kind, summary, environments, topics, scope, relations, details.benchmark_protocol.
Content fingerprint
7b894c5474ba78294427cfd4ef0eea8cb22d76dfa0c2e8c7f01e31ad927064e9 - editorial primary review
Retrieved 2026-10-16T06:31:32Z. Supports: details.benchmark_protocol.
Content fingerprint
b9a83994d8ac36ccdb606a5219511f8bb35ad3948ade4f9f3e28d386221aedfb
Suggest a correction
Open a source-backed correction. Include this record ID and the original source. Read our editorial policy and coverage boundaries.