MMA-SafetyBench: A Benchmark for Multimodal Agent Safety Evaluation

Builds trajectory-guided visual-injection tests for web, desktop and mobile observations, separating target-matching action predictions from broader semantic compromise.

AI-assisted editorial review against a primary source. This is a catalog annotation, not an independent reproduction or endorsement.

What is this resource?

Resource type

Research papers

Environment

web, desktop

Topics

evaluation, safety-security, grounding

Record publication date

2026-10-14

Evidence retrieved

2026-10-20T19:02:16.329923+00:00

Resource ID

arxiv:2610.19384

Recorded paper version

arXiv v1

Version updated: 2026-10-14T01:30:03Z.

This identifies the retained arXiv metadata snapshot; newly observed changes require separate review.

Authors

Yuke Wang, Benlei Cui, Shen Pang, Xuemei Dong, Longtao Huang, Hui Xue, Yuwen Zhai, Junjie Li, Jingqun Tang, Haiwen Hong

Cite this paper

Download BibTeX · Download CSL JSON

Citations preserve the selected source edition, its authors and recorded date precision. Unrecorded venues and identifiers are omitted.

What licenses are recorded?

Code, training data, weights, and documents can have different terms. A blank declaration means unknown, not unrestricted use.

Code

Not recorded

Data

Not recorded

Document

CC-BY-4.0 (The retained exact-v1 arXiv HTML infobox explicitly declares License: CC BY 4.0 for this document.)

Weights

Not recorded

License review notes

• Document: Paper-specific document declaration only; no code, data or weights permissions are inferred. Source 1

Explore related topics

Scope and limitations

ASR measures predicted coordinates inside an annotated target or compromised generated content; actual clicks, environment changes and downstream harm are not verified. SCR additionally counts adopted adversarial intent or task/schema disruption. Clean-task and attacked-trajectory denominators differ, including 42 clean desktop tasks versus 209 adversarial desktop trajectories. Payloads and injection points are fixed, and neutral prompts change both compliance cues and instruction-priority guidance. GPT-5 judging has a 300-trajectory human-label check, with borderline disagreements. Android-style mobile dialogs do not establish an Android runtime. Code/data links were identified but not inspected; no independent reproduction.

Primary evidence and verification

Source retrieval is separate from experiment reproduction. Use the original publication or publisher page when citing claims about the resource.

1. arxiv api

   Retrieved 2026-10-20T05:52:01Z. Supports: name, authors, published_at, updated_at, identifiers.arxiv, details.arxiv_snapshot.

   Content fingerprint

   63e37cd26bfe69455a3fa835f3d6ba4cfcc5bb7dc91f90505920d37b1e08f0b6
2. editorial primary review

   Retrieved 2026-10-20T19:02:16.329923+00:00. Supports: name, kind, summary, environments, topics, scope, notes, licenses.document.

   Content fingerprint

   ca995ec330f62a43d3d4fbbf28dfe83c5fa8b05f5c1e446929493409760268d1

Suggest a correction

Open a source-backed correction. Include this record ID and the original source. Read our editorial policy and coverage boundaries.