AwesomeCUA research reading list

AwesomeCUA research reading listhttps://awesomecua.github.io/papers/2026-10-22T20:07:57ZCUA-Lite: An Open Platform for Computer Use Agentshttps://awesomecua.github.io/resources/arxiv-2610-26071/2026-10-20T08:27:26.236797+00:00

CUA-Lite unifies environment interfaces, supervised data formats and per-model harnesses for evaluation, reinforcement learning and supervised fine-tuning across desktop, browser and mobile tasks.

AwesomeCUAFocused Omni-Modal GUI Agentshttps://awesomecua.github.io/resources/arxiv-2610-26203/2026-10-20T08:27:26.236797+00:00

FocusGUI learns step-specific weights over screenshot, audio, video and interaction-context channels, injecting a compact residual into a frozen decoder without pruning its inputs.

AwesomeCUAVeriCUA: Learning Proactive Verification for Reliable GUI Agent Rewardshttps://awesomecua.github.io/resources/arxiv-2610-26469/2026-10-20T08:27:26.236797+00:00

VeriCUA trains a Judge to select verification checks and stopping decisions while a frozen Probe inspects the post-task GUI. Single-attempt reinforcement learning balances verdict correctness and Probe use.

AwesomeCUAScopeMem: Latent Memory That Learns Where to Revise Frozen GUI Agentshttps://awesomecua.github.io/resources/arxiv-2610-26503/2026-10-20T08:27:26.236797+00:00

ScopeMem conditions latent-memory prefixes on a frozen GUI agent's initial action and trains them to correct errors while preserving correct decisions. Its evaluation separates action repairs from harmful revisions.

AwesomeCUATandem: Serving Hybrid Linear-Attention GUI Agents on Mobile NPUshttps://awesomecua.github.io/resources/arxiv-2610-26660/2026-10-20T08:27:26.236797+00:00

Tandem serves hybrid GUI-agent models on Android NPUs using divergence-point state checkpoints, rollback-aware speculative decoding and idle power release between request bursts.

AwesomeCUAComparing Computer Use Behavior Across Multiple Levels of Abstractionhttps://awesomecua.github.io/resources/arxiv-2610-26703/2026-10-20T08:27:26.236797+00:00

MAP annotates computer-use traces at linked operation, function/consequence, method and goal levels, then compares human and agent workflows through sequence, pattern and effort metrics.

AwesomeCUAAgent World Model in the Wild: Computer-Use Agent Environment Synthesishttps://awesomecua.github.io/resources/arxiv-2610-26828/2026-10-20T08:27:26.236797+00:00

WildAWM synthesizes resettable GUI/MCP/CLI environments and cross-interface tasks, using execution-based checks of solvability, difficulty and verifier consistency before supervised and reinforcement learning.

AwesomeCUABeyond Next-Screen Prediction: The Decision Value of GUI World Modelshttps://awesomecua.github.io/resources/arxiv-2610-24372/2026-10-20T13:12:51.579373Z

Separates next-screen prediction from action evaluation in GUI lookahead and trains Direct Action Ranking on reference actions, comparing prediction-augmented scoring under matched offline supervision.

AwesomeCUAPersonaCUABench: Benchmarking Computer-Use Agents Serving Diverse Personashttps://awesomecua.github.io/resources/arxiv-2610-22990/2026-10-20T08:27:26.236797+00:00

PersonaCUABench varies simulated user requests and clarification dialogue over fixed web tasks, comparing task completion with requirement handling and persona-agent satisfaction ratings.

AwesomeCUAAdaptation of Computer Use Agents to New Software with User Manualshttps://awesomecua.github.io/resources/arxiv-2610-23012/2026-10-20T08:27:26.236797+00:00

MAGE retrieves software-manual sections into fixed execution plans and generates setup/main task-plan pairs whose GUI rollouts support application-specific policy adaptation with round-based GRPO.

AwesomeCUAMobileJailBench: Red Queen Jailbreak Benchmarking for Mobile Agentshttps://awesomecua.github.io/resources/arxiv-2610-23303/2026-10-20T08:27:26.236797+00:00

MobileJailBench evaluates mobile-agent jailbreaks in a browser-based Android-like simulator, combining feedback-driven attack evolution, separately admitted task expansions and evidence-grounded outcome adjudication.

AwesomeCUAMobileInject: Evaluating Reasoning-to-Action Security of MLLM-Based Mobile Agents under Cross-Modal Consistency Injectionhttps://awesomecua.github.io/resources/arxiv-2610-23307/2026-10-20T08:27:26.236797+00:00

MobileInject evaluates visual/textual observation injection in stateful Android workflows using resettable emulators, benign/adversarial state oracles and trace-based diagnosis of emitted rationales and actions.

AwesomeCUASPIDES: Information-Flow Control for Browser Agentshttps://awesomecua.github.io/resources/arxiv-2610-24277/2026-10-20T08:27:26.236797+00:00

SPIDES propagates HTML integrity and confidentiality labels into agent context and mediates browser actions with information-flow policies and approval gates, evaluated in the labeled MicroNet ecosystem.

AwesomeCUAOpenGCUA: Toward a Fully Open Stack for Generalist Computer Usehttps://awesomecua.github.io/resources/arxiv-2610-22200/2026-10-16T06:36:29Z

Introduces an open generalist computer-use stack combining visual and programmatic interaction, with model-controlled observation and task-state decisions.

AwesomeCUAOS-Helix: Unifying GUI Interaction and Code Execution in Native Computer-Using Agentshttps://awesomecua.github.io/resources/arxiv-2610-22234/2026-10-16T06:36:29Z

Trains native computer-use models to coordinate GUI actions and code execution using hybrid demonstrations and online reinforcement learning.

AwesomeCUALEASH: Uncertainty-Aware Weak-to-Strong Escalation and Oversight for Computer-Use Agentshttps://awesomecua.github.io/resources/arxiv-2610-22280/2026-10-16T06:36:29Z

Uses weak-agent uncertainty to decide when to escalate to a stronger model and monitor its subsequent task execution.

AwesomeCUADon't Retrain, Remix: Adaptation of Web Agents using Mixture Hypernetworkshttps://awesomecua.github.io/resources/arxiv-2610-22539/2026-10-20T08:27:26.236797+00:00

WebMix uses a scout-produced interface manual to route mixtures of source-trained LoRA adapters for web agents, adapting to target interfaces without target-time gradient updates.

AwesomeCUANot All Graphs Are Maps: A Task-Agnostic Exploration Framework for Mobile GUI Navigationhttps://awesomecua.github.io/resources/arxiv-2610-22672/2026-10-20T12:56:56.464343+00:00

TAG-Nav supplies Android GUI agents with destination and route memory distilled from offline app exploration. DestBench measures arrival at a reference pre-action screen, separately from downstream task execution.

AwesomeCUAWhat It Is, Not What It Says: Securing Agentic Browsing Interactions with Web Semanticshttps://awesomecua.github.io/resources/arxiv-2610-22697/2026-10-20T13:38:46.444001Z

Intact records webpage element provenance and capabilities, isolates planning per element, and uses a browser harness to constrain actions and request consent for third-party interactions.

AwesomeCUASeeing, Maintaining, and Learning: The Evolution of Multimodal Agent Memoryhttps://awesomecua.github.io/resources/arxiv-2610-19366/2026-10-20T19:02:22.864948+00:00

Surveys multimodal memory using representation and management axes, distinguishing fixed-policy updates from policy evolution and comparing GUI memory evaluation with other modalities.

AwesomeCUAMMA-SafetyBench: A Benchmark for Multimodal Agent Safety Evaluationhttps://awesomecua.github.io/resources/arxiv-2610-19384/2026-10-20T19:02:16.329923+00:00

Builds trajectory-guided visual-injection tests for web, desktop and mobile observations, separating target-matching action predictions from broader semantic compromise.

AwesomeCUATeaching Agents in their Own Style: Blackbox Teacher Distillation for Web Agentshttps://awesomecua.github.io/resources/arxiv-2610-19834/2026-10-16T06:36:29Z

Introduces WebKD, which repairs failed student web-agent trajectories with minimal teacher edits and trains on corrections that preserve the student's reasoning style.

AwesomeCUAHow GUI Agents Fail: GraceBench for Execution Stability Diagnosishttps://awesomecua.github.io/resources/arxiv-2610-19900/2026-10-16T06:36:29Z

Introduces GraceBench, using application transition graphs and recorded interaction evidence to diagnose deviations, loops, and recovery that final task-success scores can hide.

AwesomeCUATask-State Governance: A Virtual Machine for Verifiable and Recoverable GUI Agent Executionhttps://awesomecua.github.io/resources/arxiv-2610-20100/2026-10-16T06:36:29Z

Presents TaskVM, an interface for specifying desired and preserved application state, checking agent actions, and exposing whether changes can be reversed or compensated.

AwesomeCUAWhy Generate a Click? Plug-in Decision Heads for Web Agentshttps://awesomecua.github.io/resources/arxiv-2610-20107/2026-10-16T06:36:29Z

Adds decision heads to a frozen vision-language generator for selecting web actions and targets, with a confidence gate that falls back to text generation.

AwesomeCUASTP-GUI: Spatially-Aware Token Pruning for Efficient Web-GUI Agentshttps://awesomecua.github.io/resources/arxiv-2610-20390/2026-10-16T06:36:29Z

Introduces STP-GUI, which selects spatially coherent screenshot windows conditioned on the instruction to reduce visual tokens while retaining useful interface structure.

AwesomeCUAPlayer-1: Learning to Play from Synthetic Experiencehttps://awesomecua.github.io/resources/arxiv-2610-20584/2026-10-16T06:33:24Z

Extends computer-use policies to game control through synthetic training environments and richer keyboard and pointer actions. Its evaluation concerns games rather than ordinary productivity workflows.

AwesomeCUAMAI-Gym: An Open All-in-One Platform for GUI Agent Evaluation and RL Training across Mobile and Computer Environmentshttps://awesomecua.github.io/resources/arxiv-2610-20939/2026-10-16T06:36:29Z

Unifies GUI evaluation, trajectory collection, environment synthesis, and online reinforcement learning across mobile, computer, and web tasks.

AwesomeCUAECHOES: Seeing Beyond the Current Image via Recursive History Compression and Privileged On-Policy Distillationhttps://awesomecua.github.io/resources/arxiv-2610-22408/2026-10-20T13:12:54.814500Z

ECHOES learns recursive textual task memory through privileged screenshot-history distillation and GRPO, then predicts GUI actions from the current screenshot and memory.

AwesomeCUABlenderWorld: Benchmarking Visual Agents for 3D Creation from Real-World Referenceshttps://awesomecua.github.io/resources/arxiv-2610-23986/2026-10-20T15:09:58.148629Z

Benchmarks agents creating editable Blender projects through shell, Python and MCP tools, judging delivered scenes with multi-view renders, task-specific rubrics and visual-quality ratings.

AwesomeCUAAdaptive-Group Process Rewards for GUI Agentshttps://awesomecua.github.io/resources/arxiv-2610-17710/2026-10-16T06:36:29Z

Groups GUI trajectories by subgoal and combines action-level and group-level assessments into intermediate rewards for training agents with sparse task outcomes.

AwesomeCUAMHVF: Training-Free GUI Grounding via Verification-Guided Multi-Hypothesis Searchhttps://awesomecua.github.io/resources/arxiv-2610-17792/2026-10-16T06:36:29Z

Introduces MHVF, a training-free grounding method that explores several candidate screen regions, verifies them in local views, and uses those checks to guide further search.

AwesomeCUAInteraction-as-a-Judgehttps://awesomecua.github.io/resources/arxiv-2610-17895/2026-10-16T06:33:24Z

Uses computer-use agents to interact with generated applications, studying whether interaction improves preference judging, requirement checks, and feedback for application generation.

AwesomeCUALearning How to Remember: Memory Modality Selection for GUI Agentshttps://awesomecua.github.io/resources/arxiv-2610-17928/2026-10-16T06:36:29Z

Introduces SwitchMem, which chooses a textual or visual representation of retrieved experience at each GUI interaction step instead of fixing one memory modality for a task.

AwesomeCUAWorkRover: Scaling Long-Horizon Cowork Agent Training Via Recursive Environment Evolutionhttps://awesomecua.github.io/resources/arxiv-2610-18067/2026-10-20T19:02:20Z

Synthesizes resettable workspaces with files, application observations and skills; repairs environment incompatibilities and trains long-task agents using verifier-linked turn credit.

AwesomeCUATowards Safe Autonomy for Computer Usehttps://awesomecua.github.io/resources/arxiv-2610-18508/2026-10-16T06:36:29Z

Presents SA-CUA, separating trusted and untrusted visual content using accessibility-derived masking rules and a controlled channel for verifying information before endorsement.

AwesomeCUAReading Right, Acting Wrong: Diagnosing Decision Failures of System One Models in Agent Loopshttps://awesomecua.github.io/resources/arxiv-2610-18579/2026-10-20T15:08:55Z

Pairs factual probes with next-action choices on frozen agent traces, including Mind2Web, to diagnose reading-decision gaps without closed-loop execution.

AwesomeCUAPack Light, CarryOn: Building Object-Oriented Context for Long-Horizon Agentshttps://awesomecua.github.io/resources/arxiv-2610-18784/2026-10-16T06:33:24Z

Introduces CarryOn, which maintains evidence-linked task objects and remaining-work context across history compression, with evaluations on desktop tasks and web research.

AwesomeCUAGroundGUI: GUI Grounding by High-Quality Data and Distillation-to-Reinforcement Learninghttps://awesomecua.github.io/resources/arxiv-2610-19100/2026-10-16T06:36:29Z

Introduces ScreenRef screenshot grounding data and a training sequence from supervised learning through teacher-guided distillation to reinforcement learning for GroundGUI-8B.

AwesomeCUACLASHarness: Benchmarking Multi-Agent Conflict in Computer Use via a Controllable and Verifiable Generation Pipelinehttps://awesomecua.github.io/resources/arxiv-2610-19128/2026-10-16T06:36:29Z

Introduces CLASHarness, generating verifiable computer-use tasks with conflicting agent reports, shared-state constraints, and coordination requirements to evaluate conflict detection and resolution.

AwesomeCUASeg2Click: Learning Pixel-Level Action Masks for GUI Visual Groundinghttps://awesomecua.github.io/resources/arxiv-2610-16234/2026-10-16T06:33:24Z

Recasts GUI grounding as predicting a mask of valid click locations from bounding-box supervision, then derives a click point with a deterministic selection rule.

AwesomeCUAHyWe: Unifying Search and Operation for Real-World Hybrid Web Taskshttps://awesomecua.github.io/resources/arxiv-2610-16319/2026-10-16T06:33:24Z

Introduces HyWe, which switches between information search and browser operation within one trajectory, alongside a demonstration pipeline and a benchmark of hybrid web tasks.

AwesomeCUALearn from Move: the Next Step for GUI Agentshttps://awesomecua.github.io/resources/arxiv-2610-16720/2026-10-16T06:36:29Z

Introduces LatentGUIWorld to test learning interface semantics and dynamics through interaction, and studies finer movement actions with intermediate visual feedback and exploration demonstrations.

AwesomeCUAPrivacyGuard: Source-Linked Sanitization and Locally Resolvable PII References for Cloud GUI Agentshttps://awesomecua.github.io/resources/arxiv-2610-16778/2026-10-16T06:36:29Z

Introduces PrivacyGuard, which sanitizes source-linked personal information and maintains locally resolvable references across mobile-agent turns to reduce exposure to remote planners.

AwesomeCUABrowseSafe++: Benchmarking Prompt Injection Detection Across Web Contexts, Languages, and Browser Viewshttps://awesomecua.github.io/resources/arxiv-2610-17457/2026-10-16T06:33:24Z

Extends BrowseSafe with paired attacked and clean webpages in several representations, and studies text and screenshot detectors plus feedback-adapted prompt injections.

AwesomeCUAFrom Observations to Authorized Actions: Measuring Source Information in Android Agentshttps://awesomecua.github.io/resources/arxiv-2610-15502/2026-10-20T13:12:58.069698Z

Measures source-attribution limits in Android accessibility and separately tests application-history and selected-WebView-object checks for authorizing AppFunctions actions.

AwesomeCUAVisible but Not Avoided: State-Dependent Re-Proposal of Rejected Candidates in Catalog-Based LLM Optimization Loopshttps://awesomecua.github.io/resources/arxiv-2610-15738/2026-10-20T15:31:34.620763Z

Studies repeated rejected edits while optimizing a screenshot-only GUI agent, contrasting static probes with reached loop states and testing how rejection wording changes subsequent edit proposals.

AwesomeCUAThe Agent Clicks Approve: Injected Page Text Makes Browser Agents Authorize Access the Task Never Namedhttps://awesomecua.github.io/resources/arxiv-2610-15138/2026-10-16T06:36:29Z

Studies prompt-injected account-access approvals in browser agents and proposes a service-side authority envelope that checks authorization requests against the user task before allowing them to complete.

AwesomeCUAWhen Charts Deceive Web Agents: A Paired Benchmark for Misleading Visualization-Grounded Taskshttps://awesomecua.github.io/resources/arxiv-2610-12823/2026-10-16T06:36:29Z

Introduces MisVisAgentBench, a paired executable benchmark that changes chart evidence while holding web tasks fixed, to trace how misleading visualizations affect agent decisions and downstream actions.

AwesomeCUACaptchaAgent: Verifiable Visual Reasoning for CAPTCHA Solvinghttps://awesomecua.github.io/resources/arxiv-2610-12866/2026-10-16T06:33:24Z

Presents CaptchaAgent, which builds an explicit visual representation of objects and relations, refines ambiguous regions, and checks counterfactual consistency before acting on interactive CAPTCHAs.

AwesomeCUA