Safety & security

MobileSafetyBench: Evaluating Safety of Autonomous Agents in Mobile Device Control

Juyong Lee, Dongyoon Hahm et al.

Evaluates mobile agents on misuse, side effects, and indirect prompt-injection risks in an Android emulator environment.

AdvAgent: Controllable Blackbox Red-teaming on Web Agents

Chejian Xu, Mintong Kang et al.

Introduces a black-box web-agent red-teaming framework that learns adversarial prompts from agent feedback to study security weaknesses.

Refusal-Trained LLMs Are Easily Jailbroken As Browser Agents

Priyanshu Kumar, Elaine Lau et al.

Introduces BrowserART and studies whether chat-model refusal behavior transfers to browser agents carrying out potentially harmful instructions.

From Interaction to Impact: Towards Safer AI Agents Through Understanding and Evaluating Mobile UI Operation Impacts

Zhuohao Jerry Zhang, Eldon Schoop et al.

Develops a taxonomy of mobile-action consequences and annotated interaction data to evaluate whether models recognize impactful or risky UI operations.

ST-WebAgentBench: A Benchmark for Evaluating Safety and Trustworthiness in Web Agents

Ido Levy, Ben Wiesel et al.

Evaluates web-task completion under explicit enterprise policies, measuring policy-compliant completion and safety violations separately.

EIA: Environmental Injection Attack on Generalist Web Agents for Privacy Leakage

Zeyi Liao, Lingbo Mo et al.

Evaluates privacy-directed webpage instruction injection against generalist web agents using PII-related interaction steps from Mind2Web, including attempts to expose specific user information or entire requests.

Caution for the Environment: Multimodal LLM Agents are Susceptible to Environmental Distractions

Xinbei Ma, Yiting Wang et al.

Measures how irrelevant GUI content affects multimodal agents using a simulated dataset and three perception patterns. The study also examines adversarial environment injection and possible ways to improve instruction faithfulness.

Systematic Categorization, Construction and Evaluation of New Attacks against Multi-modal Mobile GUI Agents

Yulong Yang, Xinshan Yang et al.

Develops a threat taxonomy and evaluation framework for attacks against multimodal mobile GUI agents, with case studies and dataset experiments.

Dissecting Adversarial Robustness of Multimodal LM Agents

Chen Henry Wu, Rishi Shah et al.

Introduces adversarial tasks on VisualWebArena and an evaluation framework that tracks how manipulated information passes between agent components. The analysis examines vulnerabilities introduced by reflection and tree-search evaluators as well as direct action generation.

GuardAgent: Safeguard LLM Agents by a Guard Agent via Knowledge-Enabled Reasoning

Zhen Xiang, Linzhi Zheng et al.

Introduces GuardAgent, which translates safety requests into executable guardrail code using plans and retrieved demonstrations. Mind2Web-SC supplies a web-agent safety-policy evaluation alongside a separate healthcare access-control benchmark.

WIPI: A New Web Threat for LLM-Driven Web Agents

Fangzhou Wu, Shutong Wu et al.

Studies WIPI, a black-box indirect-instruction threat in which webpage content redirects LLM-driven web agents. Evaluation spans plugin-based agents, Web GPTs, and open-source web-agent implementations.

A Trembling House of Cards? Mapping Adversarial Attacks against Language Agents

Lingbo Mo, Zeyi Liao et al.

Presents a conceptual perception–reasoning–action framework and 12 potential attack scenarios for language agents, including hypothetical web-navigation attacks.

Responsible Task Automation: Empowering Large Language Models as Responsible Task Automators

Zhizheng Zhang, Xiaoyi Zhang et al.

Presents ResponsibleTA, a coordinator-executor framework that checks command feasibility and task completion while using local memory for privacy-related controls. The paper compares prompted and learned implementations and evaluates them on UI task automation.

Adversarial Web Agents

Evaluation resources for adversarial attacks on multimodal web agents.

AgentDAM

Web-agent benchmark for measuring unnecessary use of sensitive personal information during end-to-end task execution.

BrowserART

Red-teaming benchmark and test suite from Scale AI evaluating whether refusal-trained language models can be jailbroken when deployed as browser agents.

EIA

Evaluation suite for Environmental Injection Attacks that embed invisible malicious HTML elements into benign websites to test privacy leakage in generalist web agents.

MLA-Trust

Unified evaluation framework measuring truthfulness, controllability, safety, and privacy of multimodal LLM agents across interactive web and mobile GUI tasks.

MobileSafetyBench

Android emulator testbed that scores mobile device-control agents on both safety and helpfulness, using ADB and Appium for observations, actions and success checks.

OS-Harm

Benchmark for safety of computer-use agents across misuse, prompt injection, and unintended model behavior.

PopupAttack

Resources for evaluating GUI agents under malicious popup instructions.

SafeArena

Web-agent benchmark pairing benign and harmful tasks to study deliberate misuse risks.

ST-WebAgentBench

BrowserGym-based benchmark for evaluating the safety and policy compliance of enterprise web agents across six trustworthiness risk dimensions.

WebGuard

Codebase, annotation toolkit, and action-level risk dataset for training and evaluating guardrail models that flag unsafe browser-agent actions before execution.

WebTailBench

Live-web task definitions and rubrics for evaluating computer-use agents, with preserved original tasks, a refreshed V2 task suite and a separate refusal set.

AdvAgent (formerly AdvWeb)

Research code for constructing and evaluating adversarial inputs to web agents, with a black-box red-team pipeline using Mind2Web data and SeeAct.

Agent Browser Shield

Browser extension for filtering potentially unsafe agent input.

ORBIT framework

Inspect-based multi-agent evaluation framework with role, scheduling and communication controls, BrowserART accessibility-tree actions, and desktop mouse, keyboard and screenshot tools.

OS-Kairos

Android GUI-agent research implementation that uses confidence-guided interaction to choose between autonomous actions and human assistance.

OS-Sentinel

Safety-checking environment and benchmark for mobile agents on AndroidWorld that judges proposed actions with rule-based and VLM-based checks, evaluated on the MobileRisk trajectory dataset.

RedTeamCUA

A controlled testing framework that combines desktop virtual machines and replicated websites to evaluate computer-use agents against indirect prompt injection.

VeriOS

Research code for proactive human–GUI-agent verification, including offline interaction-loop and dual-agent evaluation scripts plus LLaMA-Factory fine-tuning configurations.

VeriSafe Agent

Runtime verification framework for Android GUI agents that checks proposed actions against logical rules derived from user instructions and developer-defined app states.