Benchmarks

GUIHalluBench

Measures hallucination in UI-element parsing by matching the boxes and descriptions a model lists for 1,000 English and 1,000 Chinese screenshots against human annotations.

AI-assisted editorial review against a primary source. This is a catalog annotation, not an independent reproduction or endorsement.

Original source ↗Read as Markdown Record JSON

What is this resource?

Resource type: Benchmarks
Environment: cross-platform
Topics: evaluation, robustness
Record publication date: Not recorded
Evidence retrieved: 2026-10-16T06:40:51Z
Resource ID: github:ant-research/guihallubench

What licenses are recorded?

Code, training data, weights, and documents can have different terms. A blank declaration means unknown, not unrestricted use.

Code: Not recorded
Data: Not recorded
Document: Not recorded
Weights: Not recorded

Evaluation protocol

Focus: GUI page-element detection, element-description consistency
Setting: offline
Observations: page screenshot
Actions / outputs: list of UI element boxes [x1,y1,x2,y2] and text descriptions
Evaluation: Metrics use greedy one-to-one matching in descending IoU order, then micro precision/recall/F1 and lowercased difflib.SequenceMatcher description similarity on matched pairs. Actual metrics.py accepts IoU >= 0.5 at its default threshold; README prose says > 0.5. Predictions align to annotations by image_path; callers must supply complete prediction coverage because missing predictions are not independently enumerated by evaluate_dataset.

Scale and counting units

1,000 page-annotation entries — English annotation subset reported by README source
1,000 page-annotation entries — Chinese annotation subset reported by README source

Each annotation entry represents a page, not one UI element. README’s illustrative num_samples=4796 output is not adopted as a corpus count. Public tree contains both annotation JSON files and an images.tar.gz Git-LFS pointer; payload availability/integrity was not tested by downloading images. LEGAL.md only governs comment-language precedence, not a code/data license grant; both slots remain unknown. No benchmark run was performed.

Compare source-reviewed profiles

Explore related topics

Evaluation Robustness

Primary evidence and verification

Source retrieval is separate from experiment reproduction. Use the original publication or publisher page when citing claims about the resource.

editorial primary review
Retrieved 2026-10-16T06:40:51Z. Supports: name, kind, summary, environments, topics, scope, relations.
Content fingerprint
51df9dbcf52f46e4774207f2e6ad84dc8a6d117cdab6f77a38e9897ded1d0963
editorial primary review
Retrieved 2026-10-16T06:35:59Z. Supports: details.benchmark_protocol.
Content fingerprint
d9c476d6960a4ef99e20d4c3b5d35a63f3ae5e61fc20e9dea00b11e64b74c9e9
editorial primary review
Retrieved 2026-10-17T14:21:09Z. Supports: details.benchmark_protocol.
Content fingerprint
f36707dfcd3e83fc9d7f57b50d0c27007801e5029cc1ac255e000560ecf778fc
editorial primary review
Retrieved 2026-10-17T14:14:07Z. Supports: details.benchmark_protocol.
Content fingerprint
c0896bc90d515b7bd8fe86b062aeb199bd47570e6eca8474986f5b1ef1d98dbd
editorial primary review
Retrieved 2026-10-17T14:21:09Z. Supports: details.benchmark_protocol.
Content fingerprint
adba50d9794b9ef3f7ec8cbc680f7f1fa3fbf9df0ac8d1f9b9ccab6d941bc11b
editorial primary review
Retrieved 2026-10-17T14:34:56Z. Supports: details.benchmark_protocol.
Content fingerprint
8f35fa75be42a7a53e4f96de1b62be84962d2cf1beb8b23dfcfaecfaea4a6b90

Suggest a correction

Open a source-backed correction. Include this record ID and the original source. Read our editorial policy and coverage boundaries.