Benchmarks

WebTestBench

An end-to-end web-testing benchmark where agents derive test checklists from development instructions, interact with deployed applications and report functional or constraint-related defects.

AI-assisted editorial review against a primary source. This is a catalog annotation, not an independent reproduction or endorsement.

Original source ↗Read as Markdown Record JSON

What is this resource?

Resource type: Benchmarks
Environment: web
Topics: evaluation, web-navigation, tool-integration
Record publication date: Not recorded
Evidence retrieved: 2026-10-17T11:18:18Z
Resource ID: github:friedrichor/webtestbench

What licenses are recorded?

Code, training data, weights, and documents can have different terms. A blank declaration means unknown, not unrestricted use.

Code: Apache-2.0 (Repository source LICENSE; external tools and services retain their own licenses.)
Data: apache-2.0 (Publisher Hub-card license declaration for the released benchmark dataset.)
Document: Not recorded
Weights: Not recorded

License review notes

Code: Repository source LICENSE; external tools and services retain their own licenses. Source 1
Data: Publisher Hub-card license declaration for the released benchmark dataset. Source 1

Evaluation protocol

Focus: web functionality testing, latent permission and business constraints
Setting: interactive
Observations: web application interface, development instruction
Actions / outputs: browser interaction, test checklist generation, defect reports
Evaluation: The supplied WebTester pipeline separates checklist generation from interactive defect detection and provides a scoring workflow.

Scale and counting units

7 application categories — publisher benchmark overview source
100 web-application archives — Hub file manifest (computed) source

The 100 archive files are not counted as 100 independent test tasks or defects. Backend/tool versions and deployment settings affect reproducibility.

Compare source-reviewed profiles

Recorded specifications

availability: Public evaluation harness and ungated web-application archive manifest.
repository revision: 04873065bdb0938ed53d809559d4ab473c7ccd31

Explore related topics

Evaluation Web navigation Tool integration

Scope and limitations

Public evaluation/deployment scripts accompany an ungated Hub release with JSON/JSONL metadata and 100 application ZIP files. The ZIP count is an artifact inventory, not an asserted test-case or defect count. The documented baseline uses Claude Agent SDK and a pinned Playwright MCP version with external model credentials; these were not installed or run. Code and data Apache-2.0 declarations are retained separately.

Related resources

documented by: WebTestBench: Evaluating Computer-Use Agents towards End-to-End Automated Web Testing

Primary evidence and verification

Source retrieval is separate from experiment reproduction. Use the original publication or publisher page when citing claims about the resource.

editorial primary review
Retrieved 2026-10-17T11:18:18Z. Supports: name, kind, summary, environments, topics, scope, notes, relations, details.benchmark_protocol.
Content fingerprint
b41472026cd7d4f81a01d58bf39193dc1fabc7875c61e370090a20cc77500cdd
editorial primary review
Retrieved 2026-10-17T11:18:18Z. Supports: identity.
Content fingerprint
7203ecc2aae5f1697eeeb804ca8f21d47b9f22a8b8fa43a4064c4e1c49f5e72c
editorial primary review
Retrieved 2026-10-17T11:22:55Z. Supports: release_status, details.repository_revision.
Content fingerprint
bd1f09149b00d59a0078d414216bd2fb78e09ce7212d15bbda84ddec7052c8af
editorial primary review
Retrieved 2026-10-17T11:23:08Z. Supports: identity, release_status, details.benchmark_protocol.
Content fingerprint
db1e97f40107038ae03835dc20f17b13b37e74b5c8587f8c39b441271618f017
editorial primary review
Retrieved 2026-10-17T11:23:08Z. Supports: licenses.data, details.benchmark_protocol.
Content fingerprint
6f2f94a25ed66a14779f6b4e6686044d0510eb32a0187645cc473978d93f7c5b
editorial primary review
Retrieved 2026-10-17T11:22:55Z. Supports: licenses.code.
Content fingerprint
c71d239df91726fc519c6eb72d318ec65820627232b2f796219e87dcf35d0ab4

Suggest a correction

Open a source-backed correction. Include this record ID and the original source. Read our editorial policy and coverage boundaries.