Web navigation

WebCanvas: Benchmarking Web Agents in Online Environments

Yichen Pan, Dehan Kong et al.

Introduces WebCanvas and Mind2Web-Live, evaluating online task progress through meaningful intermediate states on changing websites.

CAAP: Context-Aware Action Planning Prompting to Solve Computer Tasks with Front-End UI Only

Junhee Cho, Jihoon Kim et al.

CAAP converts screenshots into textual observations and plans keyboard-and-mouse actions using a prompt that examines task context from multiple perspectives. The agent is evaluated on MiniWoB++ and WebShop using only front-end visual observations.

WebSuite: Systematically Evaluating Why Web Agents Fail

Eric Li, Jim Waldo

WebSuite organizes browser operations into an action taxonomy and tests both isolated controls and complete workflows. Its diagnostic tasks attribute failures to specific actions, enabling comparison of text-based and multimodal web agents.

Large Language Models Can Self-Improve At Web Agent Tasks

Ajay Patel, Markus Hofmarcher et al.

Examines self-improvement by fine-tuning web agents on three mixtures of model-generated training data. WebArena experiments and trajectory-level evaluation measures study task performance, robustness, and behavior beyond aggregate success rates.

Navigating WebAI: Training Agents to Complete Web Tasks with Large Language Models and Reinforcement Learning

Lucas-Andreï Thil, Mirela Popa et al.

Combines supervised and reinforcement learning for MiniWoB web tasks and investigates whether agents understand HTML structure or memorize target elements.

Grounded Language Agent for Product Search via Intelligent Web Interactions

Moghis Fereidouni, Adib Mosharrof et al.

Introduces GLAINTEL, a Flan-T5-based product-search web agent studied under unsupervised learning, demonstration-based training, and unsupervised domain adaptation. It compares ways to combine demonstrations with reinforcement learning.

WILBUR: Adaptive In-Context Learning for Robust and Accurate Web Agents

Michael Lutz, Arth Bohra et al.

WILBUR learns to rank prior task demonstrations for a web agent’s prompt and synthesizes instructions from past runs. A generated task curriculum and a backtracking mechanism support adaptation and recovery, evaluated on WebVoyager.

AutoWebGLM: A Large Language Model-based Web Navigating Agent

Hanyu Lai, Xiao Liu et al.

Builds a web agent through simplified HTML observations, human-and-AI browsing data, curriculum training, and reinforcement-learning refinement.

Tur[k]ingBench: A Challenge Benchmark for Web Agents

Kevin Xu, Yeganeh Kordi et al.

Evaluates web agents on multimodal annotation tasks adapted from HTML interfaces originally built for crowdsourcing workers.

WIPI: A New Web Threat for LLM-Driven Web Agents

Fangzhou Wu, Shutong Wu et al.

Studies WIPI, a black-box indirect-instruction threat in which webpage content redirects LLM-driven web agents. Evaluation spans plugin-based agents, Web GPTs, and open-source web-agent implementations.

On the Multi-turn Instruction Following for Conversational Web Agents

Yang Deng, Xuan Zhang et al.

MT-Mind2Web evaluates web navigation through sequences of conversational user instructions. Self-MAP combines memory-augmented planning with reflection to preserve context and dependencies across those interaction turns.

WebLINX: Real-World Website Navigation with Multi-Turn Dialogue

Xing Han Lù, Zdeněk Kasner et al.

Collects expert demonstrations of conversational web navigation and studies retrieval-based element selection for processing large pages.

Dual-View Visual Contextualization for Web Navigation

Jihyung Kil, Chan Hee Song et al.

Uses screenshot geometry and visual content to enrich HTML element representations for web navigation. Nearby elements contribute textual and visual context, with evaluation on cross-task, cross-website, and cross-domain Mind2Web splits.

WebVoyager: Building an End-to-End Web Agent with Large Multimodal Models

Hongliang He, Wenlin Yao et al.

Combines multimodal web interaction with a real-website task collection and an automated evaluation protocol.

VisualWebArena: Evaluating Multimodal Agents on Realistic Visual Web Tasks

Jing Yu Koh, Robert Lo et al.

Introduces visually grounded website tasks for evaluating multimodal agents that must interpret both images and text.

AgentBoard: An Analytical Evaluation Board of Multi-turn LLM Agents

Chang Ma, Junlei Zhang et al.

Introduces AgentBoard, a multiturn agent evaluation framework with intermediate progress metrics and analysis tools. Its broader task suite includes WebShop and WebArena interactions with stepwise progress measurements.

GPT-4V(ision) is a Generalist Web Agent, if Grounded

Boyuan Zheng, Boyu Gou et al.

Introduces SeeAct and live-website evaluation, examining how visual planning and element grounding affect web-agent task completion.

WebVLN: Vision-and-Language Navigation on Websites

Qi Chen, Dileepa Pitawela et al.

WebVLN trains website navigation from question-based instructions using both rendered visual information and underlying HTML. It introduces WebVLN-v1 and a website-aware navigation network to study this digital variant of vision-and-language navigation.

Exposing Limitations of Language Model Agents in Sequential-Task Compositions on the Web

Hiroki Furuta, Yutaka Matsuo et al.

CompWoB adds 50 composed web-automation tasks to study generalization from simpler interactions. The paper compares prompted and fine-tuned agents, balances task training data, and probes sensitivity to changes in instruction order.

WebWISE: Web Interface Control and Sequential Exploration with Large Language Models

Heyi Tao, Sethuraman T et al.

WebWISE filters DOM elements into observations and generates short programs for click, scroll, and text-entry steps. It evaluates in-context learning from either one supplied example or an example collected during a successful zero-shot MiniWoB++ trial.

AllTogether: Investigating the Efficacy of Spliced Prompt for Web Navigation using Large Language Models

Jiarun Liu, Wentao Hu et al.

AllTogether standardizes prompts for HTML-based web navigation and compares prompting with instruction fine-tuning. Its analysis examines how HTML context, interaction history, and current environmental feedback affect task execution.

OpenAgents: An Open Platform for Language Agents in the Wild

Tianbao Xie, Fan Zhou et al.

OpenAgents hosts three language agents, for data analysis, API plugins and autonomous web browsing, behind a web interface for non-expert users, with local deployment for developers.

A Zero-Shot Language Agent for Computer Control with Structured Reflection

Tao Li, Gang Li et al.

Studies a zero-shot computer-control agent that plans executable actions and organizes its own reflections after mistakes. MiniWoB++ experiments examine learning to complete tasks without supplied expert trajectories.

SteP: Stacked LLM Policies for Web Actions

Paloma Sodhi, S. R. K. Branavan et al.

SteP dynamically composes specialized web-action policies through a stack that records the active chain of policy calls. It studies this control structure across WebArena, MiniWoB++, and CRM workflows.

LASER: LLM Agent with State-Space Exploration for Web Navigation

Kaixin Ma, Hongming Zhang et al.

LASER represents web navigation as exploration over a defined set of states, allowing an agent to revisit earlier states after mistakes. The study evaluates this backtracking approach on WebShop and Amazon tasks.

WebArena: A Realistic Web Environment for Building Autonomous Agents

Shuyan Zhou, Frank F. Xu et al.

Introduces reproducible web environments with task-completion checks for evaluating autonomous web agents.

A Real-World WebAgent with Planning, Long Context Understanding, and Program Synthesis

Izzeddin Gur, Hiroki Furuta et al.

Combines task decomposition, HTML summarization, and generated Python programs to carry out instructions on websites.

Synapse: Trajectory-as-Exemplar Prompting with Memory for Computer Control

Longtao Zheng, Rundong Wang et al.

Synapse filters task-irrelevant page state and places complete abstracted state-action trajectories into the prompt. An embedding-based exemplar memory retrieves relevant demonstrations for new tasks, evaluated on MiniWoB++ and Mind2Web.

Mind2Web: Towards a Generalist Agent for the Web

Xiang Deng, Yu Gu et al.

Introduces a web-task dataset spanning diverse websites and evaluates generalization to unseen sites and domains.

From Pixels to UI Actions: Learning to Follow Instructions via Graphical User Interfaces

Peter Shaw, Mandar Joshi et al.

Studies instruction-following agents that observe screenshots and act through a generic mouse-and-keyboard action space, evaluating pixel-based pretraining for GUI tasks on MiniWoB++.

Hierarchical Prompting Assists Large Language Model on Web Navigation

Abishek Sridhar, Robert Lo et al.

Separates web navigation prompting into a SUMMARIZER that extracts action-relevant observations and an ACTOR that chooses the next action from the condensed context.

Multimodal Web Navigation with Instruction-Finetuned Foundation Models

Hiroki Furuta, Kuang-Huei Lee et al.

Trains WebGUM offline from demonstrations by jointly tuning an instruction-following language model and a vision encoder over webpage screenshots and HTML. The agent predicts browser actions and is evaluated on MiniWoB, WebShop, and Mind2Web.

Language Models can Solve Computer Tasks

Geunwoo Kim, Pierre Baldi et al.

Introduces recursive criticism and improvement prompting for language-model agents performing computer tasks, evaluated on MiniWoB++ with a small number of demonstrations per task.

Understanding HTML with Large Language Models

Izzeddin Gur, Ofir Nachum et al.

Studies language models for HTML element classification, page description and autonomous web navigation. Fine-tuned models are evaluated on MiniWoB, and a distilled, automatically labeled CommonCrawl HTML corpus supports the training approach.

WebShop: Towards Scalable Real-World Web Interaction with Grounded Language Agents

Shunyu Yao, Howard Chen et al.

Provides a simulated shopping environment where language agents search, configure, and select products from natural-language requirements.

Grounding Open-Domain Instructions to Automate Web Support Tasks

Nancy Xu, Sam Masling et al.

Introduces RUSS, which translates support instructions into a domain-specific language and grounds them in actionable web elements.

Rousillon: Scraping Distributed Hierarchical Web Data

Sarah E. Chasins, Maria Mueller et al.

Introduces Rousillon, which generalizes a demonstration of collecting one hierarchical data row into a browser-automation program that traverses related pages. The paper develops relation-selection and generalization methods and evaluates users building such scripts.

World of Bits: An Open-Domain Platform for Web-Based Agents

Tianlin Shi, Andrej Karpathy et al.

Introduces World of Bits, a web-task platform combining human demonstrations, keyboard and mouse actions, and cached HTTP traffic for repeatable task environments. It evaluates agents trained through behavioral cloning and reinforcement learning.

Ringer: Web Automation by Demonstration

Shaon Barman, Sarah Chasins et al.

Introduces Ringer, which converts demonstrated webpage interactions into scripts combining actions, target selectors, and waiting conditions. It evaluates replay robustness on tasks recorded from real websites.

End-to-End Goal-Driven Web Navigation

Rodrigo Nogueira, Kyunghyun Cho

Introduces WebNav for converting websites into goal-directed hyperlink navigation tasks and WikiNav for evaluating language-conditioned navigation and question answering on Wikipedia.

Koala: Capture, Share, Automate, Personalize Business Processes on the Web

Greg Little, Tessa A. Lau et al.

Introduces Koala, a collaborative web automation system that records browser interactions as editable pseudo-natural-language scripts. Scripts are shared through a wiki and instantiated with personal or corporate data for reuse by other users.

PLOW: A Collaborative Task Learning Agent

James Allen, Nathanael Chambers et al.

Describes PLOW, which learns executable web-task models from a session combining demonstrations, natural-language explanations, and dialogue. Browser instrumentation connects user actions with DOM structure to support task generalization.

AgentRewardBench

A benchmark and evaluation toolkit for comparing automatic web-agent judges against expert judgments of task success, side effects and repetition.

BrowserART

Red-teaming benchmark and test suite from Scale AI evaluating whether refusal-trained language models can be jailbroken when deployed as browser agents.

CUAVerifierBench

Human-annotated browser-agent trajectories for comparing automated task-success and process-quality verifiers against reviewer judgments.

EIA

Evaluation suite for Environmental Injection Attacks that embed invisible malicious HTML elements into benign websites to test privacy leakage in generalist web agents.

Mind2Web

Human demonstrations and tasks for generalist web navigation research.

Mind2Web 2

Long-horizon agentic web search benchmark whose Agent-as-a-Judge framework scores time-varying, citation-backed answers, with evaluation scripts for its dev and test sets.

Online Mind2Web

300 tasks on 136 live websites for evaluating web agents online, with outdated tasks replaced over time, human-reviewed submissions and the WebJudge automatic evaluator.

ST-WebAgentBench

BrowserGym-based benchmark for evaluating the safety and policy compliance of enterprise web agents across six trustworthiness risk dimensions.

WebArena

Self-hosted web environments and tasks for evaluating autonomous web agents.

WebClick

A web-interface localization benchmark with screenshot instructions and target boxes, including agent browsing, human browsing and calendar cases.

WebGuard

Codebase, annotation toolkit, and action-level risk dataset for training and evaluating guardrail models that flag unsafe browser-agent actions before execution.

WebLINX

Demonstrations and evaluation for conversational web navigation.

WebRewardBench

An offline preference-evaluation set for web process reward models, pairing page and trajectory context with preferred and rejected candidate decisions.

WebShop

Simulated shopping environment for instruction-following web agents.

WebTailBench

Live-web task definitions and rubrics for evaluating computer-use agents, with preserved original tasks, a refreshed V2 task suite and a separate refusal set.

WebTestBench

An end-to-end web-testing benchmark where agents derive test checklists from development instructions, interact with deployed applications and report functional or constraint-related defects.

WebVoyager

Multimodal web navigation agent and evaluation tasks on real websites.

Go-Browse-WA processed data

Flattened web-interaction training records with accessibility-tree observations, assistant completions, trajectory rewards and step-link metadata.