{"additional_discovery_provenance":{"method_papers_selected_from_root_search":["arxiv:2606.20622","arxiv:2605.05765","arxiv:2601.19306","arxiv:2510.06042","arxiv:2509.02444","arxiv:2508.18040"],"root_raw_abstracts_remain_cache_only":true,"root_title_search":"research/survey/evidence/title-search.json","user_requested_primary_paper":"arxiv:2609.30186"},"assembled_at":"2026-10-18T10:54:15.682120+00:00","catalog_baseline":{"catalog_sha256":"7f71a1683ab013e587a20086c3e357956981f517a24ae8bc8e81791108c66e4f","source_commit":"87166fe8ad9c8e832f7a90ee84b2e93c6e0e40db"},"claim_to_source_ledger":[{"citation_keys":["arxiv_2306_06070","arxiv_2410_05243","arxiv_2501_12326"],"claim":"The central methodological question is how an agent's observation, action interface, training data, and learning objective determine what transfers to a new task. Generalization has several meanings here: locating unfamiliar controls, composing familiar operations into a new workflow, recovering from an unexpected state, or moving between operating systems. A method can improve one without establishing the others. We therefore compare systems by the information and control they receive, the experience they reuse, and the feedback that changes their behavior. This separates architectural mechanisms from reported benchmark outcomes, which often use different histories, tools, evaluators, and interaction budgets.","claim_id":"methods-c01","claim_type":"primary_method_description_with_explicit_survey_synthesis","evidence_boundary":"Mechanisms are source descriptions. Cross-paper implications and illustrative failure conditions are survey analysis, not new experiments, reproduction, or causal effect estimates.","paragraph_purpose":"Define multiple generalization targets and the comparison axes.","section_anchor":"methods-c01","sources":[{"body_sha256":"75d8af953769a53bf75913908e2bb6185c6b338c8d89b7304e7fd8cfb79934e8","citation_key":"arxiv_2306_06070","pdf_pages_read":[1,5,6,7,9],"primary_url":"https://arxiv.org/pdf/2306.06070v3","sections_read":["3.1 Candidate Generation","3.2 Action Prediction","4.1 Experimental Setup","4.2 Evaluation","6 Limitations"],"version_read":3},{"body_sha256":"b166b8e09f604e64067d12ded4689e7582df106988b28f018651835843b04e68","citation_key":"arxiv_2410_05243","pdf_pages_read":[1,3,4,5],"primary_url":"https://arxiv.org/pdf/2410.05243v3","sections_read":["2.1 Overview","2.2 Data Construction","2.3 Model Design"],"version_read":3},{"body_sha256":"999d01bdd86eb39df1c322d7ab2a48f9f10d8f8a6cb2af913591612712b3964e","citation_key":"arxiv_2501_12326","pdf_pages_read":[1,2,13,14,15,16,17,18,19,20,21,22],"primary_url":"https://arxiv.org/pdf/2501.12326v1","sections_read":["Front matter and abstract","4.1 Architecture Overview","4.2 Enhancing GUI Perception","4.3 Unified Action Modeling and Grounding","4.4 Infusing System-2 Reasoning","4.5 Learning from Prior Experience in Long-term Memory","4.6 Training"],"version_read":1}]},{"citation_keys":["arxiv_2306_06070","arxiv_2401_01614"],"claim":"Structured interface representations make some decisions easier by exposing labels, hierarchy, and actionable objects directly. Mind2Web's baseline first ranks DOM elements, then selects an element and operation from reduced candidate groups. This makes large pages manageable, but the pruning stage creates a recall boundary: downstream reasoning cannot select a target that was removed. SeeAct separates screenshot-based action description from grounding that description to an executable element. Its experiments distinguish textual choices, visual marks, and human oracle grounding. That separation is diagnostically useful: knowing what should happen and identifying where to act are different capabilities, and oracle grounding does not demonstrate autonomous execution.","claim_id":"methods-c02","claim_type":"primary_method_description_with_explicit_survey_synthesis","evidence_boundary":"Mechanisms are source descriptions. Cross-paper implications and illustrative failure conditions are survey analysis, not new experiments, reproduction, or causal effect estimates.","paragraph_purpose":"Separate planning from candidate selection and its recall ceiling.","section_anchor":"methods-c02","sources":[{"body_sha256":"75d8af953769a53bf75913908e2bb6185c6b338c8d89b7304e7fd8cfb79934e8","citation_key":"arxiv_2306_06070","pdf_pages_read":[1,5,6,7,9],"primary_url":"https://arxiv.org/pdf/2306.06070v3","sections_read":["3.1 Candidate Generation","3.2 Action Prediction","4.1 Experimental Setup","4.2 Evaluation","6 Limitations"],"version_read":3},{"body_sha256":"ac281a3c5cdbda428e14605e9d96035fd35af82073ce46117c27c34792eea041","citation_key":"arxiv_2401_01614","pdf_pages_read":[1,2,3,4,7],"primary_url":"https://arxiv.org/pdf/2401.01614v2","sections_read":["2.1 Formulation","2.2 Action Generation","2.3 Action Grounding","3.2 Methods","4.3 Analysis"],"version_read":2}]},{"citation_keys":["arxiv_2401_13919","arxiv_2312_13771","arxiv_2401_16158"],"claim":"The label “visual agent” does not uniquely specify the observation contract. WebVoyager uses screenshots annotated with interactive elements obtained through JavaScript, together with auxiliary element text. AppAgent's Android implementation uses XML-derived identifiers and numbered screenshot overlays. Mobile-Agent instead combines optical character recognition (OCR), icon detection, and visual matching to localize actions without XML. These choices exchange structural coverage for perceptual estimation: a missing accessibility node and a misrecognized icon fail differently. Mobile-Agent's experiments are on Android even though the interface motivates broader portability. Cross-platform intent should therefore be distinguished from platforms actually evaluated.","claim_id":"methods-c03","claim_type":"primary_method_description_with_explicit_survey_synthesis","evidence_boundary":"Mechanisms are source descriptions. Cross-paper implications and illustrative failure conditions are survey analysis, not new experiments, reproduction, or causal effect estimates.","paragraph_purpose":"Distinguish actual observation contracts and evaluated platforms.","section_anchor":"methods-c03","sources":[{"body_sha256":"476ed4a064311f93414f5d5d17d90f10d07d2922bc05e09ee89cea26b617a2cb","citation_key":"arxiv_2401_13919","pdf_pages_read":[1,3,4],"primary_url":"https://arxiv.org/pdf/2401.13919v4","sections_read":["3.1 Browsing Environment","3.2 Interaction Formulation","3.3 Observation Space","3.4 Action Space","4.1 Website Selection"],"version_read":4},{"body_sha256":"800ec0437da9183b77fb6fe3db7b4f64e2ff23b88efc887f6809cd00db0af4c8","citation_key":"arxiv_2312_13771","pdf_pages_read":[1,4,5],"primary_url":"https://arxiv.org/pdf/2312.13771v3","sections_read":["3.1 Environment and Action Space","3.2 Exploration","3.3 Deployment"],"version_read":3},{"body_sha256":"6b511d598fa2f9bea3692d3acf028ebfe46b36b83181baab60a0152955ebec0d","citation_key":"arxiv_2401_16158","pdf_pages_read":[1,2,4],"primary_url":"https://arxiv.org/pdf/2401.16158v2","sections_read":["2.1 Visual Perception","2.2 Instruction Execution","3 Experiments"],"version_read":2}]},{"citation_keys":["arxiv_2401_10935","arxiv_2312_08914","arxiv_2410_05243","arxiv_2412_16256"],"claim":"Coordinate-based grounding models predict locations directly, but they still make representation choices. SeeClick trains coordinate-text prediction using GUI grounding and inverse OCR tasks. CogAgent supplements a conventional visual path with a higher-resolution cross-attention module, addressing small text and controls without placing all high-resolution features in one self-attention sequence. UGround separates a screenshot planner from a specialized visual grounder and adapts image resolution for interface localization. These approaches make coordinate prediction trainable, while retaining a practical trade-off between spatial detail and processing cost. Better localization alone does not resolve an incorrect instruction from the planner; Aria-UI explicitly identifies that dependency as a limitation.","claim_id":"methods-c04","claim_type":"primary_method_description_with_explicit_survey_synthesis","evidence_boundary":"Mechanisms are source descriptions. Cross-paper implications and illustrative failure conditions are survey analysis, not new experiments, reproduction, or causal effect estimates.","paragraph_purpose":"Compare visual grounding architecture, resolution and planner dependence.","section_anchor":"methods-c04","sources":[{"body_sha256":"b62bd796d8e6d45e13b41ed41a55d021a9df3329bf500d14efed129e8a1e04e2","citation_key":"arxiv_2401_10935","pdf_pages_read":[1,3,4,9],"primary_url":"https://arxiv.org/pdf/2401.10935v2","sections_read":["3.1 GUI Grounding","3.2 Data Construction","3.3 Training Details","Limitations"],"version_read":2},{"body_sha256":"d9cb086d826eaee9f919e0cbf142193b77f476b0aef6260b34408abe142fe13b","citation_key":"arxiv_2312_08914","pdf_pages_read":[1,3,4,5],"primary_url":"https://arxiv.org/pdf/2312.08914v3","sections_read":["2.1 Architecture","2.2 High-Resolution Cross-Module","2.3 Pre-training","2.4 Multi-task Fine-tuning"],"version_read":3},{"body_sha256":"b166b8e09f604e64067d12ded4689e7582df106988b28f018651835843b04e68","citation_key":"arxiv_2410_05243","pdf_pages_read":[1,3,4,5],"primary_url":"https://arxiv.org/pdf/2410.05243v3","sections_read":["2.1 Overview","2.2 Data Construction","2.3 Model Design"],"version_read":3},{"body_sha256":"169bc7694b456530ce8a6f0336b7efc33a32a21a731e1656caf4934e83f59334","citation_key":"arxiv_2412_16256","pdf_pages_read":[1,3,4,9],"primary_url":"https://arxiv.org/pdf/2412.16256v2","sections_read":["Front matter and abstract","2.1 Large-scale Diverse GUI Data Synthesizing","2.1.2 Context-aware Data Extension from Trajectories","6 Limitations"],"version_read":2}]},{"citation_keys":["arxiv_2312_08914","arxiv_2401_10935","arxiv_2410_05243","arxiv_2410_23218"],"claim":"Visual deployment also should not be confused with structure-free training. CogAgent, SeeClick, and UGround use structured web information to construct visual supervision. OS-ATLAS extends collection through platform-specific accessibility APIs and exploration, then separates grounding pretraining from action fine-tuning. Thus, structure can serve as a training instrument even when it is absent at inference. The transfer question becomes whether the resulting visual representations cover unfamiliar controls and referring expressions, rather than whether HTML was ever used. Conversely, a shared action schema is an interface convention: standardizing click, tap, and scroll labels does not make their platform-specific effects identical.","claim_id":"methods-c05","claim_type":"primary_method_description_with_explicit_survey_synthesis","evidence_boundary":"Mechanisms are source descriptions. Cross-paper implications and illustrative failure conditions are survey analysis, not new experiments, reproduction, or causal effect estimates.","paragraph_purpose":"Separate inference inputs from structure used during training-data collection.","section_anchor":"methods-c05","sources":[{"body_sha256":"d9cb086d826eaee9f919e0cbf142193b77f476b0aef6260b34408abe142fe13b","citation_key":"arxiv_2312_08914","pdf_pages_read":[1,3,4,5],"primary_url":"https://arxiv.org/pdf/2312.08914v3","sections_read":["2.1 Architecture","2.2 High-Resolution Cross-Module","2.3 Pre-training","2.4 Multi-task Fine-tuning"],"version_read":3},{"body_sha256":"b62bd796d8e6d45e13b41ed41a55d021a9df3329bf500d14efed129e8a1e04e2","citation_key":"arxiv_2401_10935","pdf_pages_read":[1,3,4,9],"primary_url":"https://arxiv.org/pdf/2401.10935v2","sections_read":["3.1 GUI Grounding","3.2 Data Construction","3.3 Training Details","Limitations"],"version_read":2},{"body_sha256":"b166b8e09f604e64067d12ded4689e7582df106988b28f018651835843b04e68","citation_key":"arxiv_2410_05243","pdf_pages_read":[1,3,4,5],"primary_url":"https://arxiv.org/pdf/2410.05243v3","sections_read":["2.1 Overview","2.2 Data Construction","2.3 Model Design"],"version_read":3},{"body_sha256":"ed7ba8eb36d50e36b14e480a3cb89ac9f8fe825d28f763acd8210dcb7ade8bd0","citation_key":"arxiv_2410_23218","pdf_pages_read":[1,3,4,5,6],"primary_url":"https://arxiv.org/pdf/2410.23218v1","sections_read":["3.1 Task Formulation and Training","3.2 Grounding Data Collection","3.3 Unified Action Space","4.1 Evaluation Details"],"version_read":1}]},{"citation_keys":["arxiv_2411_17465","arxiv_2412_16256"],"claim":"ShowUI treats visual efficiency as part of grounding. It groups neighboring patches with similar pixel values and skips selected redundant tokens while preserving the positions of retained tokens. It also interleaves screenshots and actions for navigation, and packs multiple grounding queries against one screenshot during training. Aria-UI adds task and action history to grounding examples, including interleaved visual context. Together, these designs show why neither resolution nor context length is a sufficient description of perception: the agent must retain the small spatial cues and earlier states that matter to the current decision. Compression quality should be evaluated against that decision, not only token savings.","claim_id":"methods-c06","claim_type":"primary_method_description_with_explicit_survey_synthesis","evidence_boundary":"Mechanisms are source descriptions. Cross-paper implications and illustrative failure conditions are survey analysis, not new experiments, reproduction, or causal effect estimates.","paragraph_purpose":"Connect token compression and history encoding to retained decision evidence.","section_anchor":"methods-c06","sources":[{"body_sha256":"8bc023be285e71dab992fb5aeffc2bcbcefe155a67b6ce5e6e7e3a3499bb5db5","citation_key":"arxiv_2411_17465","pdf_pages_read":[1,3,4,5,6],"primary_url":"https://arxiv.org/pdf/2411.17465v1","sections_read":["2.1 UI-Guided Visual Token Selection","2.2 Interleaved VLA Streaming","2.3 GUI Instructional Tuning"],"version_read":1},{"body_sha256":"169bc7694b456530ce8a6f0336b7efc33a32a21a731e1656caf4934e83f59334","citation_key":"arxiv_2412_16256","pdf_pages_read":[1,3,4,9],"primary_url":"https://arxiv.org/pdf/2412.16256v2","sections_read":["Front matter and abstract","2.1 Large-scale Diverse GUI Data Synthesizing","2.1.2 Context-aware Data Extension from Trajectories","6 Limitations"],"version_read":2}]},{"citation_keys":["arxiv_2402_07939","arxiv_2410_08164"],"claim":"Modular systems allocate different responsibilities to different components. UFO's HostAgent selects applications and coordinates work, while an AppAgent operates application controls through Windows automation. Agent S separates a manager's task decomposition from workers' grounded execution and reflection. Its interface augments an accessibility tree with OCR and restricts execution to discrete actions with feedback after each step. These decompositions provide explicit places to diagnose planning, grounding, and execution failures. They also introduce dependencies: UFO cannot operate controls unsupported by its backend, and Agent S's original interface requires more than pixels. Hierarchy organizes those dependencies; it does not eliminate them.","claim_id":"methods-c07","claim_type":"primary_method_description_with_explicit_survey_synthesis","evidence_boundary":"Mechanisms are source descriptions. Cross-paper implications and illustrative failure conditions are survey analysis, not new experiments, reproduction, or causal effect estimates.","paragraph_purpose":"Explain modular control and its backend dependencies.","section_anchor":"methods-c07","sources":[{"body_sha256":"55680fbdd9e98455e4b8c93474e3f367f76d86d170b722433ea2e298fd0a8acf","citation_key":"arxiv_2402_07939","pdf_pages_read":[1,4,5,6,7,14],"primary_url":"https://arxiv.org/pdf/2402.07939v5","sections_read":["3.1 Agent Framework","3.4 Action Execution","5 Limitations"],"version_read":5},{"body_sha256":"ee8c9b38e20b5f747bd78828d96beab0b266eb99baaafe44c3936ff0d89cb92b","citation_key":"arxiv_2410_08164","pdf_pages_read":[1,4,5,6,7],"primary_url":"https://arxiv.org/pdf/2410.08164v1","sections_read":["3.1 Experience-Augmented Hierarchical Planning","3.2 Self-Supervised Memory Construction","3.3 Agent-Computer Interface","4.1 Experimental Setup"],"version_read":1}]},{"citation_keys":["arxiv_2412_04454","arxiv_2501_12326"],"claim":"Native policies move more of this coordination into a trained model. Aguvis trains grounding before planning and reasoning, representing actions through PyAutoGUI-style commands plus platform-specific extensions. UI-TARS predicts reasoning and actions from screenshots and interaction history, with shared and device-specific actions and an explicit request for user intervention. This reduces the need to hand-specify every intermediate module, but execution still depends on an external action interpreter and environment. A unified output format also cannot guarantee semantic compatibility: typing into a focused field, selecting an object, and terminating a task require different state conditions even when their commands share one syntax.","claim_id":"methods-c08","claim_type":"primary_method_description_with_explicit_survey_synthesis","evidence_boundary":"Mechanisms are source descriptions. Cross-paper implications and illustrative failure conditions are survey analysis, not new experiments, reproduction, or causal effect estimates.","paragraph_purpose":"Explain native policy training without claiming removal of execution dependencies.","section_anchor":"methods-c08","sources":[{"body_sha256":"d9f0c31bdae82ae2be8ce1921ba5b23b3898cb61f14ad7755ff8290e7d877a6c","citation_key":"arxiv_2412_04454","pdf_pages_read":[1,3,4,5,6],"primary_url":"https://arxiv.org/pdf/2412.04454v2","sections_read":["2.2 Agent Framework (continued)","2.3 AGUVIS DATA COLLECTION","2.4 Model Architecture","2.5 Training Paradigm","3.2 Offline GUI Agent Evaluation","3.3 Online GUI Agent Evaluation","4.2 Role of Inner Monologue"],"version_read":2},{"body_sha256":"999d01bdd86eb39df1c322d7ab2a48f9f10d8f8a6cb2af913591612712b3964e","citation_key":"arxiv_2501_12326","pdf_pages_read":[1,2,13,14,15,16,17,18,19,20,21,22],"primary_url":"https://arxiv.org/pdf/2501.12326v1","sections_read":["Front matter and abstract","4.1 Architecture Overview","4.2 Enhancing GUI Perception","4.3 Unified Action Modeling and Grounding","4.4 Infusing System-2 Reasoning","4.5 Learning from Prior Experience in Long-term Memory","4.6 Training"],"version_read":1}]},{"citation_keys":["arxiv_2504_00906","arxiv_2504_14603","arxiv_2509_02544"],"claim":"The practical boundary is increasingly hybrid. Agent S2 routes actions to visual, OCR-based text-span, or structural specialists; its spreadsheet specialist can update cells programmatically. UFO2 combines Windows UI Automation and visual detections, then prefers registered native application APIs with GUI fallback. UI-TARS-2 extends GUI control with terminal and other tool operations in shared stateful environments. These are substantive changes to the available action space, not interchangeable implementations of a pointer-only agent. They can shorten workflows by expressing structured operations directly, while introducing application bindings, tool permissions, and backend-specific failure modes. Comparisons should state which routes were available and which actually executed.","claim_id":"methods-c09","claim_type":"primary_method_description_with_explicit_survey_synthesis","evidence_boundary":"Mechanisms are source descriptions. Cross-paper implications and illustrative failure conditions are survey analysis, not new experiments, reproduction, or causal effect estimates.","paragraph_purpose":"Identify hybrid execution routes and their comparison implications.","section_anchor":"methods-c09","sources":[{"body_sha256":"b87310f7b40407c9413d99bbd6cd027bac381cae77ef4d116dc3070012aa3ae5","citation_key":"arxiv_2504_00906","pdf_pages_read":[1,3,4,5],"primary_url":"https://arxiv.org/pdf/2504.00906v1","sections_read":["3 Agent S2","3.1 Mixture of Grounding","3.2 Proactive Hierarchical Planning"],"version_read":1},{"body_sha256":"ceecfb496791b8b101275520066eb69c3289c7c727a9e69cc844219598acece5","citation_key":"arxiv_2504_14603","pdf_pages_read":[1,3,4,5,6,7,8],"primary_url":"https://arxiv.org/pdf/2504.14603v2","sections_read":["3.1 UFO2 as a System Substrate","3.2 HostAgent","3.3 AppAgent","3.4 Hybrid Control Detection","3.5 Unified GUI–API Action Orchestrator"],"version_read":2},{"body_sha256":"6a2df5206955e0eab044e1bc4e0ed9239439e0377f79d179044da41a0bd5c603","citation_key":"arxiv_2509_02544","pdf_pages_read":[1,2,4,5,7,9,10,11,12,13],"primary_url":"https://arxiv.org/pdf/2509.02544v2","sections_read":["2.1 Formulation","2.2 All-in-One GUI Sandbox","2.3 Data Flywheel","2.4.2 Interactive Annotation","2.5 Multi-turn Reinforcement Learning","2.6 Parameter Interpolation"],"version_read":2}]},{"citation_keys":["arxiv_2609_30186"],"claim":"Jev-Mobile examines a different division of labor: a vision-language model (VLM) sets a local goal and exact input text, while a typed decision service chooses from actions enumerated from the current Android accessibility tree. Candidates are rebuilt after each action, so delegated execution retains feedback rather than replaying an unobserved batch. Typed output constrains the available choices but does not establish that a chosen action is correct. Tree-missing visual targets remain inaccessible, and “local” denotes decision scope rather than on-device inference. Its comparison evaluates complete systems with different executors; it does not isolate a causal advantage of the decision model over a small VLM in the same controller.","claim_id":"methods-c29","claim_type":"primary_method_description_with_explicit_survey_synthesis","evidence_boundary":"Mechanisms are source descriptions. Cross-paper implications and illustrative failure conditions are survey analysis, not new experiments, reproduction, or causal effect estimates.","paragraph_purpose":"Explain typed decision execution and the limited comparison in Jev-Mobile.","section_anchor":"methods-c29","sources":[{"body_sha256":"ddc55d413582d0a17eec43bb37ffaeb348c4228b434694cc653853de2b32d33a","citation_key":"arxiv_2609_30186","pdf_pages_read":[],"primary_url":"https://arxiv.org/html/2609.30186v1","sections_read":["2 Problem Setup","3.1 Delegation State","3.2 Current Accessibility-Tree Candidates","3.3 Typed Local Decision","3.4 Budgets and Boundary","5 Experimental Design","8 Limitations"],"version_read":1}]},{"citation_keys":["arxiv_2312_13771","arxiv_2409_07429","arxiv_2410_08164"],"claim":"A history is not yet a useful state representation. AppAgent turns exploration or demonstrations into documents about application behavior. Agent Workflow Memory extracts reusable subroutines and replaces example-specific values with variables; its offline and online variants differ in when those workflows are acquired. Agent S retrieves high-level narrative experience for planning and more concrete episodic experience for workers. These mechanisms reuse experience without necessarily updating model weights. Their evaluation must specify what memory exists initially, whether it grows across test tasks, and how experiences are admitted. In the online variants, model-based success judgments can propagate mistakes into later retrieval.","claim_id":"methods-c10","claim_type":"primary_method_description_with_explicit_survey_synthesis","evidence_boundary":"Mechanisms are source descriptions. Cross-paper implications and illustrative failure conditions are survey analysis, not new experiments, reproduction, or causal effect estimates.","paragraph_purpose":"Distinguish retrieved experience from weight updates and define memory evaluation conditions.","section_anchor":"methods-c10","sources":[{"body_sha256":"800ec0437da9183b77fb6fe3db7b4f64e2ff23b88efc887f6809cd00db0af4c8","citation_key":"arxiv_2312_13771","pdf_pages_read":[1,4,5],"primary_url":"https://arxiv.org/pdf/2312.13771v3","sections_read":["3.1 Environment and Action Space","3.2 Exploration","3.3 Deployment"],"version_read":3},{"body_sha256":"ddc6f6fedec48c88ae743b43a464f7eb4aecaab05acd18c29cfb657ab445ebe4","citation_key":"arxiv_2409_07429","pdf_pages_read":[1,3,4,5],"primary_url":"https://arxiv.org/pdf/2409.07429v1","sections_read":["2.2 Workflow Representation","2.3 Inducing and Using Workflows","3.1 WebArena"],"version_read":1},{"body_sha256":"ee8c9b38e20b5f747bd78828d96beab0b266eb99baaafe44c3936ff0d89cb92b","citation_key":"arxiv_2410_08164","pdf_pages_read":[1,4,5,6,7],"primary_url":"https://arxiv.org/pdf/2410.08164v1","sections_read":["3.1 Experience-Augmented Hierarchical Planning","3.2 Self-Supervised Memory Construction","3.3 Agent-Computer Interface","4.1 Experimental Setup"],"version_read":1}]},{"citation_keys":["arxiv_2605_18652"],"claim":"Memory can also be learned as a control policy. MementoGUI trains operators for selecting salient events, compressing working memory, writing episodes, and filtering retrieval results. It preserves selected image regions alongside textual summaries and augments a frozen GUI action model. This distinguishes learning what to remember from fine-tuning how to act. However, its reported evaluations use offline GUI benchmarks and model-judged action or memory measures; they should not be read as proof of live task completion. The broader design issue is evidence preservation: compression can discard a transient selection or dialog state that is unavailable in the current screenshot.","claim_id":"methods-c11","claim_type":"primary_method_description_with_explicit_survey_synthesis","evidence_boundary":"Mechanisms are source descriptions. Cross-paper implications and illustrative failure conditions are survey analysis, not new experiments, reproduction, or causal effect estimates.","paragraph_purpose":"Explain learned multimodal memory with the offline evaluation boundary.","section_anchor":"methods-c11","sources":[{"body_sha256":"bf6e672269ac55c35870ab2b93d652b472c30fd6ad9c98797b00127d58132305","citation_key":"arxiv_2605_18652","pdf_pages_read":[1,3,4,5,6,7],"primary_url":"https://arxiv.org/pdf/2605.18652v1","sections_read":["3 Data Curation","4.1 Framework","4.2 Memory System Design","4.3 Training MementoCore","4.4 Benchmarking","5.1 Experimental Setup"],"version_read":1}]},{"citation_keys":["arxiv_2510_06042","arxiv_2602_20502"],"claim":"Structured planning introduces stronger internal constraints, with correspondingly stronger modeling assumptions. Agent+P builds a UI transition graph using static program analysis and dynamic refinement, translates navigation into a symbolic planning problem, and sends the plan to a UI executor. Its formal guarantees apply to the represented graph and chosen target, not automatically to the real application or the interpretation of user intent. ActionEngine instead learns application states and reusable operations through offline exploration, then compiles a generated program into browser actions and data operations. Both shift work from repeated local deliberation to reusable application knowledge; incomplete graphs, changed controls, and mistaken state identification remain operational concerns.","claim_id":"methods-c12","claim_type":"primary_method_description_with_explicit_survey_synthesis","evidence_boundary":"Mechanisms are source descriptions. Cross-paper implications and illustrative failure conditions are survey analysis, not new experiments, reproduction, or causal effect estimates.","paragraph_purpose":"Compare graph-based planning and program compilation under model assumptions.","section_anchor":"methods-c12","sources":[{"body_sha256":"5e2cb2f053983140904ec15c097d060ff18315c6c3c956bbcacbec26921709c5","citation_key":"arxiv_2510_06042","pdf_pages_read":[1,4,5,6],"primary_url":"https://arxiv.org/pdf/2510.06042v3","sections_read":["3.3 UI Automation to Classical Planning","4.1 UTG Builder","4.2 Node Selector","4.3 Plan Generator","4.4 UI Explorer","5.1 Datasets"],"version_read":3},{"body_sha256":"ec8339a07ee3c83d4f416a27cf9e8bf254efa26968b7bc9fdf3e07ff09e38ab6","citation_key":"arxiv_2602_20502","pdf_pages_read":[1,3,4,5,6],"primary_url":"https://arxiv.org/pdf/2602.20502v2","sections_read":["3 State-Machine Memory","3.1 Application State Representation","3.2 Automated SMG Construction","4.1 Planner","4.2 Executor","4.3 Patcher (opening)"],"version_read":2}]},{"citation_keys":["arxiv_2401_16158","arxiv_2410_08164","arxiv_2504_00906","arxiv_2412_09605"],"claim":"Verification operates at several levels that should remain separate. Mobile-Agent reflects on unchanged or unexpected screens. Agent S can return failure to its manager, and Agent S2 revises the remaining plan after subgoals. Such checks can guide recovery without independently proving that the user's goal was achieved. A visible draft is not necessarily a saved document, and a model's completion token is not an external state check. Likewise, success judgments used to filter training trajectories need their own validation. The methodological distinction is between detecting a suspicious transition, choosing a corrective action, and establishing the final postcondition.","claim_id":"methods-c13","claim_type":"primary_method_description_with_explicit_survey_synthesis","evidence_boundary":"Mechanisms are source descriptions. Cross-paper implications and illustrative failure conditions are survey analysis, not new experiments, reproduction, or causal effect estimates.","paragraph_purpose":"Separate transition checking, recovery and final-goal verification.","section_anchor":"methods-c13","sources":[{"body_sha256":"6b511d598fa2f9bea3692d3acf028ebfe46b36b83181baab60a0152955ebec0d","citation_key":"arxiv_2401_16158","pdf_pages_read":[1,2,4],"primary_url":"https://arxiv.org/pdf/2401.16158v2","sections_read":["2.1 Visual Perception","2.2 Instruction Execution","3 Experiments"],"version_read":2},{"body_sha256":"ee8c9b38e20b5f747bd78828d96beab0b266eb99baaafe44c3936ff0d89cb92b","citation_key":"arxiv_2410_08164","pdf_pages_read":[1,4,5,6,7],"primary_url":"https://arxiv.org/pdf/2410.08164v1","sections_read":["3.1 Experience-Augmented Hierarchical Planning","3.2 Self-Supervised Memory Construction","3.3 Agent-Computer Interface","4.1 Experimental Setup"],"version_read":1},{"body_sha256":"b87310f7b40407c9413d99bbd6cd027bac381cae77ef4d116dc3070012aa3ae5","citation_key":"arxiv_2504_00906","pdf_pages_read":[1,3,4,5],"primary_url":"https://arxiv.org/pdf/2504.00906v1","sections_read":["3 Agent S2","3.1 Mixture of Grounding","3.2 Proactive Hierarchical Planning"],"version_read":1},{"body_sha256":"7bdb3828c245ef1ad1cf60ead2140e1bd5f6ff01f8a189f50914cb07f49ef603","citation_key":"arxiv_2412_09605","pdf_pages_read":[1,4,5,6],"primary_url":"https://arxiv.org/pdf/2412.09605v2","sections_read":["2.1 Tutorial Collection and Standardization","2.2.1 Trajectory Data Schema","2.2.2 Guided Replay with Tutorials","2.2.3 Evaluation of Trajectory"],"version_read":2}]},{"citation_keys":["arxiv_2501_12326","arxiv_2508_09123"],"claim":"Recovery becomes more concrete when training includes states reached after mistakes. UI-TARS annotates both a corrected action at the error point and a subsequent correction after the wrong action has already changed the environment. OpenCUA allows imperfect human demonstrations, uses a reflector to identify incorrect or redundant steps, and constructs reasoning around state transitions while excluding unsuitable action targets from training. These recipes address a gap in clean demonstrations: an agent deployed on its own predictions visits states that an expert path may never contain. Reflection text is useful supervision, but its presence does not establish that the generated explanation faithfully causes the action.","claim_id":"methods-c14","claim_type":"primary_method_description_with_explicit_survey_synthesis","evidence_boundary":"Mechanisms are source descriptions. Cross-paper implications and illustrative failure conditions are survey analysis, not new experiments, reproduction, or causal effect estimates.","paragraph_purpose":"Explain post-error state supervision and limits of generated reflection.","section_anchor":"methods-c14","sources":[{"body_sha256":"999d01bdd86eb39df1c322d7ab2a48f9f10d8f8a6cb2af913591612712b3964e","citation_key":"arxiv_2501_12326","pdf_pages_read":[1,2,13,14,15,16,17,18,19,20,21,22],"primary_url":"https://arxiv.org/pdf/2501.12326v1","sections_read":["Front matter and abstract","4.1 Architecture Overview","4.2 Enhancing GUI Perception","4.3 Unified Action Modeling and Grounding","4.4 Infusing System-2 Reasoning","4.5 Learning from Prior Experience in Long-term Memory","4.6 Training"],"version_read":1},{"body_sha256":"fd1c1e7b4d467acf63fc743c3925c7ceb015e5b1625f62c22295c0b9e2d82a70","citation_key":"arxiv_2508_09123","pdf_pages_read":[1,3,4,5,6,7],"primary_url":"https://arxiv.org/pdf/2508.09123v3","sections_read":["2 AgentNet Collection","2.2 AgentNet Tool","3.1 Synthesizing Reflective Long CoT Reasoning","3.2 Context Encoding and Test-Time Reasoning","3.3 Training Data Mixtures"],"version_read":3}]},{"citation_keys":["arxiv_2410_23218","arxiv_2412_04454","arxiv_2410_05243","arxiv_2412_16256"],"claim":"Grounding data and trajectory data supervise different problems. Element descriptions paired with coordinates teach localization; a sequence conditioned on a user goal teaches when and why an action is useful. OS-ATLAS and Aguvis make this distinction explicit through staged training. The referring-expression distribution matters too: functional descriptions, spatial relationships, and visual appearance ask different questions of the same control. UGround and Aria-UI synthesize such variation instead of relying only on literal element text. Scaling annotation count can therefore leave important gaps if most examples repeat easy labels, shallow pages, or one interaction style.","claim_id":"methods-c15","claim_type":"primary_method_description_with_explicit_survey_synthesis","evidence_boundary":"Mechanisms are source descriptions. Cross-paper implications and illustrative failure conditions are survey analysis, not new experiments, reproduction, or causal effect estimates.","paragraph_purpose":"Distinguish localization data from sequential decision data.","section_anchor":"methods-c15","sources":[{"body_sha256":"ed7ba8eb36d50e36b14e480a3cb89ac9f8fe825d28f763acd8210dcb7ade8bd0","citation_key":"arxiv_2410_23218","pdf_pages_read":[1,3,4,5,6],"primary_url":"https://arxiv.org/pdf/2410.23218v1","sections_read":["3.1 Task Formulation and Training","3.2 Grounding Data Collection","3.3 Unified Action Space","4.1 Evaluation Details"],"version_read":1},{"body_sha256":"d9f0c31bdae82ae2be8ce1921ba5b23b3898cb61f14ad7755ff8290e7d877a6c","citation_key":"arxiv_2412_04454","pdf_pages_read":[1,3,4,5,6],"primary_url":"https://arxiv.org/pdf/2412.04454v2","sections_read":["2.2 Agent Framework (continued)","2.3 AGUVIS DATA COLLECTION","2.4 Model Architecture","2.5 Training Paradigm","3.2 Offline GUI Agent Evaluation","3.3 Online GUI Agent Evaluation","4.2 Role of Inner Monologue"],"version_read":2},{"body_sha256":"b166b8e09f604e64067d12ded4689e7582df106988b28f018651835843b04e68","citation_key":"arxiv_2410_05243","pdf_pages_read":[1,3,4,5],"primary_url":"https://arxiv.org/pdf/2410.05243v3","sections_read":["2.1 Overview","2.2 Data Construction","2.3 Model Design"],"version_read":3},{"body_sha256":"169bc7694b456530ce8a6f0336b7efc33a32a21a731e1656caf4934e83f59334","citation_key":"arxiv_2412_16256","pdf_pages_read":[1,3,4,9],"primary_url":"https://arxiv.org/pdf/2412.16256v2","sections_read":["Front matter and abstract","2.1 Large-scale Diverse GUI Data Synthesizing","2.1.2 Context-aware Data Extension from Trajectories","6 Limitations"],"version_read":2}]},{"citation_keys":["arxiv_2508_09123"],"claim":"Human demonstration pipelines must reconstruct decision-time information carefully. OpenCUA records screen video and input signals, compresses low-level events into meaningful actions, and aligns actions to prior screen states. For clicks, it searches before cursor movement to reduce leakage from a pointer already indicating the target. This illustrates a general issue in offline training: the recorded observation must contain information available before the decision. Action reduction also chooses the temporal unit of learning. A merged scroll or text insertion is easier to model than thousands of device events, but exposes different opportunities for intermediate feedback.","claim_id":"methods-c16","claim_type":"primary_method_description_with_explicit_survey_synthesis","evidence_boundary":"Mechanisms are source descriptions. Cross-paper implications and illustrative failure conditions are survey analysis, not new experiments, reproduction, or causal effect estimates.","paragraph_purpose":"Describe decision-time alignment and action discretization in human demonstrations.","section_anchor":"methods-c16","sources":[{"body_sha256":"fd1c1e7b4d467acf63fc743c3925c7ceb015e5b1625f62c22295c0b9e2d82a70","citation_key":"arxiv_2508_09123","pdf_pages_read":[1,3,4,5,6,7],"primary_url":"https://arxiv.org/pdf/2508.09123v3","sections_read":["2 AgentNet Collection","2.2 AgentNet Tool","3.1 Synthesizing Reflective Long CoT Reasoning","3.2 Context Encoding and Test-Time Reasoning","3.3 Training Data Mixtures"],"version_read":3}]},{"citation_keys":["arxiv_2412_09605","arxiv_2502_11357","arxiv_2412_19723"],"claim":"Synthetic trajectories differ mainly in where their goals come from. AgentTrek filters and standardizes tutorials, then uses a browser agent to replay their instructions and a model to judge the resulting trajectories. Explorer refines a task while navigating, allowing the eventual instruction to reflect what is actually available. OS-Genesis starts from observed action transitions, infers plausible low- and high-level tasks, and subsequently executes those tasks. These approaches reduce dependence on manually recorded demonstrations, but select different experience distributions: published procedures, goals consistent with an exploration path, or functions discovered through traversal. None establishes exhaustive coverage of realistic user intentions.","claim_id":"methods-c17","claim_type":"primary_method_description_with_explicit_survey_synthesis","evidence_boundary":"Mechanisms are source descriptions. Cross-paper implications and illustrative failure conditions are survey analysis, not new experiments, reproduction, or causal effect estimates.","paragraph_purpose":"Compare the sources of synthetic goals and resulting coverage limits.","section_anchor":"methods-c17","sources":[{"body_sha256":"7bdb3828c245ef1ad1cf60ead2140e1bd5f6ff01f8a189f50914cb07f49ef603","citation_key":"arxiv_2412_09605","pdf_pages_read":[1,4,5,6],"primary_url":"https://arxiv.org/pdf/2412.09605v2","sections_read":["2.1 Tutorial Collection and Standardization","2.2.1 Trajectory Data Schema","2.2.2 Guided Replay with Tutorials","2.2.3 Evaluation of Trajectory"],"version_read":2},{"body_sha256":"038a14d81f9587ac869667d77f93ce33eac65d18e8037402680c091204e8cac8","citation_key":"arxiv_2502_11357","pdf_pages_read":[1,3,4],"primary_url":"https://arxiv.org/pdf/2502.11357v4","sections_read":["3 Data Recipe","3.1 Website Selection","3.2 Data Generation Pipeline"],"version_read":4},{"body_sha256":"65bb449a7e9d433f29b088e0ed5d52f22833572b7a67533b48cf07e88f14292d","citation_key":"arxiv_2412_19723","pdf_pages_read":[1,3,4,5],"primary_url":"https://arxiv.org/pdf/2412.19723v3","sections_read":["3.1 Interaction-Driven Functional Discovery","3.2 Reverse Task Synthesis","3.3 Trajectory Reward Model","4.1 Experimental Settings"],"version_read":3}]},{"citation_keys":["arxiv_2502_11357","arxiv_2412_09605","arxiv_2412_19723"],"claim":"Filtering determines what the model is allowed to learn from. Explorer halts collection at login, CAPTCHA, and payment boundaries, which limits the resulting workflows. AgentTrek's replay uses structured browser observations even when the resulting data later trains a visual policy. OS-Genesis uses graded trajectory scores to sample some incomplete but informative executions rather than keeping only binary successes. These choices should be reported alongside dataset size. Our synthesis is that diversity has at least three dimensions: interface coverage, goal coverage, and state coverage after errors. A large corpus can be broad in one dimension while remaining narrow in the others.","claim_id":"methods-c18","claim_type":"primary_method_description_with_explicit_survey_synthesis","evidence_boundary":"Mechanisms are source descriptions. Cross-paper implications and illustrative failure conditions are survey analysis, not new experiments, reproduction, or causal effect estimates.","paragraph_purpose":"Explain how collection and filtering select interface, goal and error-state coverage.","section_anchor":"methods-c18","sources":[{"body_sha256":"038a14d81f9587ac869667d77f93ce33eac65d18e8037402680c091204e8cac8","citation_key":"arxiv_2502_11357","pdf_pages_read":[1,3,4],"primary_url":"https://arxiv.org/pdf/2502.11357v4","sections_read":["3 Data Recipe","3.1 Website Selection","3.2 Data Generation Pipeline"],"version_read":4},{"body_sha256":"7bdb3828c245ef1ad1cf60ead2140e1bd5f6ff01f8a189f50914cb07f49ef603","citation_key":"arxiv_2412_09605","pdf_pages_read":[1,4,5,6],"primary_url":"https://arxiv.org/pdf/2412.09605v2","sections_read":["2.1 Tutorial Collection and Standardization","2.2.1 Trajectory Data Schema","2.2.2 Guided Replay with Tutorials","2.2.3 Evaluation of Trajectory"],"version_read":2},{"body_sha256":"65bb449a7e9d433f29b088e0ed5d52f22833572b7a67533b48cf07e88f14292d","citation_key":"arxiv_2412_19723","pdf_pages_read":[1,3,4,5],"primary_url":"https://arxiv.org/pdf/2412.19723v3","sections_read":["3.1 Interaction-Driven Functional Discovery","3.2 Reverse Task Synthesis","3.3 Trajectory Reward Model","4.1 Experimental Settings"],"version_read":3}]},{"citation_keys":["arxiv_2412_04454","arxiv_2501_12326","arxiv_2508_09123"],"claim":"Generated reasoning adds another supervisory layer. Aguvis constructs inner monologue using the goal, observation, history, and known action. UI-TARS explicitly recognizes the risk of explanations that merely fit a supplied answer and also samples thought–action pairs before selecting those that match the target. OpenCUA adds reflection, memory, and planning information through a multi-stage synthesis pipeline. These are methods for constructing useful training targets, not direct recordings of human reasoning or demonstrations of causal faithfulness. Their value should be assessed through controlled ablations and behavior on unfamiliar states, with the generator and filtering policy treated as part of the data recipe.","claim_id":"methods-c19","claim_type":"primary_method_description_with_explicit_survey_synthesis","evidence_boundary":"Mechanisms are source descriptions. Cross-paper implications and illustrative failure conditions are survey analysis, not new experiments, reproduction, or causal effect estimates.","paragraph_purpose":"Distinguish generated reasoning supervision from causal-faithfulness evidence.","section_anchor":"methods-c19","sources":[{"body_sha256":"d9f0c31bdae82ae2be8ce1921ba5b23b3898cb61f14ad7755ff8290e7d877a6c","citation_key":"arxiv_2412_04454","pdf_pages_read":[1,3,4,5,6],"primary_url":"https://arxiv.org/pdf/2412.04454v2","sections_read":["2.2 Agent Framework (continued)","2.3 AGUVIS DATA COLLECTION","2.4 Model Architecture","2.5 Training Paradigm","3.2 Offline GUI Agent Evaluation","3.3 Online GUI Agent Evaluation","4.2 Role of Inner Monologue"],"version_read":2},{"body_sha256":"999d01bdd86eb39df1c322d7ab2a48f9f10d8f8a6cb2af913591612712b3964e","citation_key":"arxiv_2501_12326","pdf_pages_read":[1,2,13,14,15,16,17,18,19,20,21,22],"primary_url":"https://arxiv.org/pdf/2501.12326v1","sections_read":["Front matter and abstract","4.1 Architecture Overview","4.2 Enhancing GUI Perception","4.3 Unified Action Modeling and Grounding","4.4 Infusing System-2 Reasoning","4.5 Learning from Prior Experience in Long-term Memory","4.6 Training"],"version_read":1},{"body_sha256":"fd1c1e7b4d467acf63fc743c3925c7ceb015e5b1625f62c22295c0b9e2d82a70","citation_key":"arxiv_2508_09123","pdf_pages_read":[1,3,4,5,6,7],"primary_url":"https://arxiv.org/pdf/2508.09123v3","sections_read":["2 AgentNet Collection","2.2 AgentNet Tool","3.1 Synthesizing Reflective Long CoT Reasoning","3.2 Context Encoding and Test-Time Reasoning","3.3 Training Data Mixtures"],"version_read":3}]},{"citation_keys":["arxiv_2501_12326","arxiv_2504_10458"],"claim":"Supervised fine-tuning, preference optimization, and reinforcement learning should not be grouped solely by whether a paper uses the word “reasoning.” Supervised learning increases the likelihood of demonstrated outputs. UI-TARS's preference stage additionally contrasts corrected and erroneous actions. GUI-R1 applies group-relative optimization to candidate responses using format, action-type, point-in-box, and input-text rewards. These rewards verify an annotated local prediction; they do not by themselves measure whether a freely executed multi-step task succeeds. Local action accuracy can improve while delayed consequences and recovery remain outside the training signal.","claim_id":"methods-c20","claim_type":"primary_method_description_with_explicit_survey_synthesis","evidence_boundary":"Mechanisms are source descriptions. Cross-paper implications and illustrative failure conditions are survey analysis, not new experiments, reproduction, or causal effect estimates.","paragraph_purpose":"Contrast SFT, preferences and local verifiable-action optimization.","section_anchor":"methods-c20","sources":[{"body_sha256":"999d01bdd86eb39df1c322d7ab2a48f9f10d8f8a6cb2af913591612712b3964e","citation_key":"arxiv_2501_12326","pdf_pages_read":[1,2,13,14,15,16,17,18,19,20,21,22],"primary_url":"https://arxiv.org/pdf/2501.12326v1","sections_read":["Front matter and abstract","4.1 Architecture Overview","4.2 Enhancing GUI Perception","4.3 Unified Action Modeling and Grounding","4.4 Infusing System-2 Reasoning","4.5 Learning from Prior Experience in Long-term Memory","4.6 Training"],"version_read":1},{"body_sha256":"6e9d838b0c87ac3e21e266748924be44869abe8f1e818b7e1bb066f746d9bc9f","citation_key":"arxiv_2504_10458","pdf_pages_read":[1,3,4,5],"primary_url":"https://arxiv.org/pdf/2504.10458v4","sections_read":["3 GUI-R1 Framework","3.1 Preliminaries","3.2 Verifiable Rewards in Unified Action Space","3.3 Training Data Curation"],"version_read":4}]},{"citation_keys":["arxiv_2504_10458","arxiv_2506_07976","arxiv_2501_12326"],"claim":"Reward granularity determines which mistakes the learner can distinguish. A point inside the annotated box establishes a localization match, but does not check whether the application accepted the operation or retained the intended change. Conversely, a final success label does not identify which intermediate actions were necessary. In successful-rollout filtering, actions along a retained path receive imitation supervision even when some were redundant. Preference pairs can target a known mistake more precisely, but their coverage is limited to the states and alternatives collected. These are complementary sources of information rather than a universal ordering of objectives. A transfer claim is stronger when evaluation tests both unfamiliar successful paths and the error states that the chosen supervision leaves weakly constrained.","claim_id":"methods-c31","claim_type":"primary_method_description_with_explicit_survey_synthesis","evidence_boundary":"Mechanisms are source descriptions. Cross-paper implications and illustrative failure conditions are survey analysis, not new experiments, reproduction, or causal effect estimates.","paragraph_purpose":"Explain which errors local rewards, outcome filtering and preferences can distinguish.","section_anchor":"methods-c31","sources":[{"body_sha256":"6e9d838b0c87ac3e21e266748924be44869abe8f1e818b7e1bb066f746d9bc9f","citation_key":"arxiv_2504_10458","pdf_pages_read":[1,3,4,5],"primary_url":"https://arxiv.org/pdf/2504.10458v4","sections_read":["3 GUI-R1 Framework","3.1 Preliminaries","3.2 Verifiable Rewards in Unified Action Space","3.3 Training Data Curation"],"version_read":4},{"body_sha256":"be76d2989976e3d59e20189821c3dfccc7b6008e2e29f1a2741b567fd5b01c65","citation_key":"arxiv_2506_07976","pdf_pages_read":[1,3,4,5,7,8],"primary_url":"https://arxiv.org/pdf/2506.07976v2","sections_read":["3 Problem Setup","4.1 Scaling Interaction by Acting Longer","4.2 Interaction vs Per-Step Compute","5.1 Fixed Long Horizons","5.2 Curriculum Over Interaction Horizon"],"version_read":2},{"body_sha256":"999d01bdd86eb39df1c322d7ab2a48f9f10d8f8a6cb2af913591612712b3964e","citation_key":"arxiv_2501_12326","pdf_pages_read":[1,2,13,14,15,16,17,18,19,20,21,22],"primary_url":"https://arxiv.org/pdf/2501.12326v1","sections_read":["Front matter and abstract","4.1 Architecture Overview","4.2 Enhancing GUI Perception","4.3 Unified Action Modeling and Grounding","4.4 Infusing System-2 Reasoning","4.5 Learning from Prior Experience in Long-term Memory","4.6 Training"],"version_read":1}]},{"citation_keys":["arxiv_2411_02337","arxiv_2509_02544"],"claim":"Environment-interacting RL exposes those consequences but must solve reward and credit-assignment problems. WebRL generates curriculum tasks from failed attempts, evaluates outcomes with a learned reward model, and constrains policy updates while retaining replay experience. Its policy observes HTML and history, and its outcome model uses the final page plus actions. UI-TARS-2 trains multi-turn policies with stateful rollouts, using different reward mechanisms for different domains: executable checks, answer judges, or generative outcome scoring. A result described as “verifiable” must therefore specify what is checked and by whom. A learned judge is a useful source of supervision, not an automatic guarantee of goal correctness.","claim_id":"methods-c21","claim_type":"primary_method_description_with_explicit_survey_synthesis","evidence_boundary":"Mechanisms are source descriptions. Cross-paper implications and illustrative failure conditions are survey analysis, not new experiments, reproduction, or causal effect estimates.","paragraph_purpose":"Explain rollout reward sources and their assumptions.","section_anchor":"methods-c21","sources":[{"body_sha256":"79a93b02cca5586d9e3e2742c5af53f683c632dc0eaed3b7bf87f53daaa2d462","citation_key":"arxiv_2411_02337","pdf_pages_read":[1,3,4,5],"primary_url":"https://arxiv.org/pdf/2411.02337v3","sections_read":["2 Self-Evolving Online Curriculum RL","2.1 Self-evolving New Instruction","2.2 Reinforcement Learning for LLMs in Online Web Environments"],"version_read":3},{"body_sha256":"6a2df5206955e0eab044e1bc4e0ed9239439e0377f79d179044da41a0bd5c603","citation_key":"arxiv_2509_02544","pdf_pages_read":[1,2,4,5,7,9,10,11,12,13],"primary_url":"https://arxiv.org/pdf/2509.02544v2","sections_read":["2.1 Formulation","2.2 All-in-One GUI Sandbox","2.3 Data Flywheel","2.4.2 Interactive Annotation","2.5 Multi-turn Reinforcement Learning","2.6 Parameter Interpolation"],"version_read":2}]},{"citation_keys":["arxiv_2506_07976"],"claim":"The interaction horizon changes both learning difficulty and the behaviors an agent can express. Test-Time Interaction (TTI) studies a curriculum that progressively permits longer rollouts, using successful trajectories for updates. Short budgets favor tasks solved quickly; longer budgets permit exploration and recovery but make outcome credit harder to assign. Its prompting experiments also show that repeated rechecking can replace a correct answer with an incorrect one. The useful question is consequently not whether longer thinking or more actions always wins, but which additional computation supplies relevant evidence under a fixed task, observation contract, and budget.","claim_id":"methods-c22","claim_type":"primary_method_description_with_explicit_survey_synthesis","evidence_boundary":"Mechanisms are source descriptions. Cross-paper implications and illustrative failure conditions are survey analysis, not new experiments, reproduction, or causal effect estimates.","paragraph_purpose":"Explain interaction-horizon curricula and the limits of repeated rechecking.","section_anchor":"methods-c22","sources":[{"body_sha256":"be76d2989976e3d59e20189821c3dfccc7b6008e2e29f1a2741b567fd5b01c65","citation_key":"arxiv_2506_07976","pdf_pages_read":[1,3,4,5,7,8],"primary_url":"https://arxiv.org/pdf/2506.07976v2","sections_read":["3 Problem Setup","4.1 Scaling Interaction by Acting Longer","4.2 Interaction vs Per-Step Compute","5.1 Fixed Long Horizons","5.2 Curriculum Over Interaction Horizon"],"version_read":2}]},{"citation_keys":["arxiv_2606_05597"],"claim":"Systems choices can change the effective learning objective. AsyncWebRL overlaps rollout, training, and policy refresh through persistent workers and lightweight screenshot references. It separately corrects for stale rollout policies and studies how per-trajectory length normalization underweights tokens in long failed episodes. Replacing that normalization changes the relative influence of those failures. The reported diagnosis is tied to its WebGym setup, memory prompt, and model variants; it should not be generalized into a universal claim about all policy optimization. It nevertheless shows why efficiency analysis must inspect both wall-clock scheduling and the weighting of experience consumed by the learner.","claim_id":"methods-c23","claim_type":"primary_method_description_with_explicit_survey_synthesis","evidence_boundary":"Mechanisms are source descriptions. Cross-paper implications and illustrative failure conditions are survey analysis, not new experiments, reproduction, or causal effect estimates.","paragraph_purpose":"Connect asynchronous scheduling and experience weighting without universal causal claims.","section_anchor":"methods-c23","sources":[{"body_sha256":"423c18c48b337d91ecd7fe548cd67a910ccf4cc373a4b400674e9f0b2bc4cba4","citation_key":"arxiv_2606_05597","pdf_pages_read":[1,3,4,7],"primary_url":"https://arxiv.org/pdf/2606.05597v3","sections_read":["3.1 System","3.2 Algorithm","4.1 Setup","5.2 Ablations"],"version_read":3}]},{"citation_keys":["arxiv_2602_20502","arxiv_2607_28227"],"claim":"Fewer model calls can arise through different mechanisms. ActionEngine amortizes exploration into reusable operations and compiles a task program; its design still allows state filtering, repair, and explicit semantic helper calls. Qwen-UI-Agent can emit ordered action batches and combine GUI operations with CLI or service calls. Batching reduces observation and inference boundaries, whereas compilation reuses application knowledge. Both require care when later actions depend on an intermediate result. Reporting only action counts can obscure that difference: one model turn may contain several operations, and a programmatic action may perform work that would require many pointer events.","claim_id":"methods-c24","claim_type":"primary_method_description_with_explicit_survey_synthesis","evidence_boundary":"Mechanisms are source descriptions. Cross-paper implications and illustrative failure conditions are survey analysis, not new experiments, reproduction, or causal effect estimates.","paragraph_purpose":"Separate amortized planning, action batching and backend-expanded operations.","section_anchor":"methods-c24","sources":[{"body_sha256":"ec8339a07ee3c83d4f416a27cf9e8bf254efa26968b7bc9fdf3e07ff09e38ab6","citation_key":"arxiv_2602_20502","pdf_pages_read":[1,3,4,5,6],"primary_url":"https://arxiv.org/pdf/2602.20502v2","sections_read":["3 State-Machine Memory","3.1 Application State Representation","3.2 Automated SMG Construction","4.1 Planner","4.2 Executor","4.3 Patcher (opening)"],"version_read":2},{"body_sha256":"705de6c54fa4eec0e5f81792cd323695e1d0274333886973cbad82d3c436fdb0","citation_key":"arxiv_2607_28227","pdf_pages_read":[1,2,3,4,6,9,10,13,14,15,44],"primary_url":"https://arxiv.org/pdf/2607.28227v1","sections_read":["2.1 System Overview and Action Space","2.2.2 Real-Device Runtime","2.2.3 Hybrid GUI+CLI","2.4.1 SFT","2.4.2 Action RL","2.4.3 Online RL","7 Limitations"],"version_read":1}]},{"citation_keys":["arxiv_2605_17320"],"claim":"Branching adds another axis of computation: trying alternatives from the same workspace state. TClone provides local workspace versions using containers, copy-on-write sharing, and asynchronous checkpointing, including GUI state rather than only files. This supports rollback and speculative branches with an explicit commit boundary. Its network design also makes the limit clear: external connections must be re-established, and effects on remote services generally cannot be undone by restoring local state. Thus, cheaper search is not equivalent to harmless exploration. Any use of branching for computer control must distinguish reversible local operations from actions that affect other systems.","claim_id":"methods-c25","claim_type":"primary_method_description_with_explicit_survey_synthesis","evidence_boundary":"Mechanisms are source descriptions. Cross-paper implications and illustrative failure conditions are survey analysis, not new experiments, reproduction, or causal effect estimates.","paragraph_purpose":"Explain local workspace branching and remote-effect boundaries.","section_anchor":"methods-c25","sources":[{"body_sha256":"cd6d543feb3ec804b35f91ae9dd897a563038f9230067abd5eaafd88976e46af","citation_key":"arxiv_2605_17320","pdf_pages_read":[1,4,5,8],"primary_url":"https://arxiv.org/pdf/2605.17320v1","sections_read":["3 TClone Overview","4 TClone Design","4.4 Network Subsystem","4.5 GUI Subsystem"],"version_read":1}]},{"citation_keys":["arxiv_2606_20622","arxiv_2607_28227"],"claim":"Mobile execution brings reset and failure accounting into the training design. Darwin Mobile Agent uses asynchronous cloud Android instances and a setup–execution–teardown lifecycle; minimal cleanup preserves some state between episodes, while infrastructure failures are distinguished from task termination. Its implemented system uses manually selected tasks, an external judge, and a configured history window. Internally learned curricula, verification, and memory remain roadmap goals. Qwen-UI-Agent similarly describes device-health scheduling and separate review of model and environment failures. Treating an unavailable service as an agent error, or silently removing such episodes, would change the learning and evaluation distribution.","claim_id":"methods-c26","claim_type":"primary_method_description_with_explicit_survey_synthesis","evidence_boundary":"Mechanisms are source descriptions. Cross-paper implications and illustrative failure conditions are survey analysis, not new experiments, reproduction, or causal effect estimates.","paragraph_purpose":"Separate mobile state persistence, environment failures and self-evolution roadmap.","section_anchor":"methods-c26","sources":[{"body_sha256":"e4711466886372760daa83384cb4d129d54fdcfe320be6e91fb7b4971d1a61de","citation_key":"arxiv_2606_20622","pdf_pages_read":[1,3,4,5,6,7,8],"primary_url":"https://arxiv.org/pdf/2606.20622v1","sections_read":["2 Agent-Environment Interface and Policy Optimisation","3 Scalable Asynchronous Mobile Environment","3.2 Persistent State Management","3.3 Task Lifecycle","4.2 Rollout and Optimisation","4.3 Anatomy of the Self-Evolving Agent"],"version_read":1},{"body_sha256":"705de6c54fa4eec0e5f81792cd323695e1d0274333886973cbad82d3c436fdb0","citation_key":"arxiv_2607_28227","pdf_pages_read":[1,2,3,4,6,9,10,13,14,15,44],"primary_url":"https://arxiv.org/pdf/2607.28227v1","sections_read":["2.1 System Overview and Action Space","2.2.2 Real-Device Runtime","2.2.3 Hybrid GUI+CLI","2.4.1 SFT","2.4.2 Action RL","2.4.3 Online RL","7 Limitations"],"version_read":1}]},{"citation_keys":["arxiv_2601_19306","arxiv_2605_05765"],"claim":"Recent mobile work asks when an agent should acquire application knowledge rather than repeatedly improvise. Curiosity Driven Knowledge Retrieval compares predicted and observed interface descriptions, triggering retrieval into structured AppCards when its uncertainty signal rises. The cards summarize functions, parameters, interface mappings, and interaction patterns from documentation, code, and earlier trajectories. X-OmniClaw takes a complementary route: it combines Android XML, OCR, visual grounding, and multimodal context, then captures user behavior as reusable skill cards and activity or deeplink entry routes. Its “behavior cloning” describes skill capture at runtime, not demonstrated policy-weight training, and literal trace replay is presented as a future extension.","claim_id":"methods-c27","claim_type":"primary_method_description_with_explicit_survey_synthesis","evidence_boundary":"Mechanisms are source descriptions. Cross-paper implications and illustrative failure conditions are survey analysis, not new experiments, reproduction, or causal effect estimates.","paragraph_purpose":"Compare targeted knowledge acquisition and captured mobile skills.","section_anchor":"methods-c27","sources":[{"body_sha256":"ef3bbc8adcc147239ea910e0c59904d7a26ee1610a7a4d1613be2cb845098536","citation_key":"arxiv_2601_19306","pdf_pages_read":[1,3,4,5],"primary_url":"https://arxiv.org/pdf/2601.19306v1","sections_read":["3.1 Problem Formulation","3.2 Curiosity-Driven Information Gain Estimation","3.3 Retrieving and Embedding External Knowledge"],"version_read":1},{"body_sha256":"80375cd810dcda2e9ccca2c6fbee44730de8449972036c548bde312566b20673","citation_key":"arxiv_2605_05765","pdf_pages_read":[1,3,4,5,6,7],"primary_url":"https://arxiv.org/pdf/2605.05765v2","sections_read":["2 Framework","3 Omni Perception","4 Omni Memory","5.1 Omni Action in App Ecosystem","5.2 Trajectory Cloned Execution"],"version_read":2}]},{"citation_keys":["arxiv_2508_18040","arxiv_2509_02444"],"claim":"Adaptation also concerns the user and the stability of repeated workflows. PerPilot identifies missing personal information, retrieves it from accumulated memory, or explores a likely application before requesting clarification if information remains unavailable. This makes instruction interpretation part of control rather than assuming every goal is fully specified. AppCopilot describes a different reuse mechanism: semantic matching selects successful historical logs for direct replay under stable layouts and task structures. Its coordinate calibration moves out-of-region clicks toward detected widgets while remembering ineffective corrections. These mechanisms depend on the validity of stored information and detected regions. Reusing a preference, replaying a path, and correcting a coordinate therefore require different checks; none follows automatically from a high retrieval match.","claim_id":"methods-c30","claim_type":"primary_method_description_with_explicit_survey_synthesis","evidence_boundary":"Mechanisms are source descriptions. Cross-paper implications and illustrative failure conditions are survey analysis, not new experiments, reproduction, or causal effect estimates.","paragraph_purpose":"Distinguish personal information, stable-path replay and coordinate calibration.","section_anchor":"methods-c30","sources":[{"body_sha256":"3c3474b771af7bdd500e6aaa91b83d39ec0d455e59bb83df906e7a94700ad2b7","citation_key":"arxiv_2508_18040","pdf_pages_read":[1,3,4],"primary_url":"https://arxiv.org/pdf/2508.18040v1","sections_read":["The Framework of PerPilot","Personalization Perception","Personalization Completion","PerInstruct Data Construction"],"version_read":1},{"body_sha256":"6c2591dad8893f87cffdea211584a3733dc2137dafa9416c1885d1b0126fd540","citation_key":"arxiv_2509_02444","pdf_pages_read":[1,5,6,34,37,38,42,48,49,50],"primary_url":"https://arxiv.org/pdf/2509.02444v2","sections_read":["4.2 OCR/OR Region Localization","4.2.2 Coordinate Calibration","4.3 Multi-Agent Voting","5.2 Task Decomposition","6.2 Experience Reuse","6.3 Function Calling"],"version_read":2}]},{"citation_keys":["arxiv_2306_06070","arxiv_2412_16256","arxiv_2605_18652","arxiv_2607_28227"],"claim":"Across these methods, generalization depends on what is retained and what can be checked. Visual grounding broadens interface access; application knowledge reduces repeated discovery; demonstrations and synthetic rollouts expand experience; outcome learning changes behavior under feedback. Each can also introduce a new dependency: perceptual errors, stale memory, collection bias, or an imperfect verifier. The strongest methodological reports expose those dependencies and distinguish local prediction, offline trajectory scoring, live completion, and infrastructure reliability. This makes progress interpretable without assuming that a larger model, a longer reasoning trace, or a higher score under a different protocol resolves the entire computer-use problem.","claim_id":"methods-c28","claim_type":"primary_method_description_with_explicit_survey_synthesis","evidence_boundary":"Mechanisms are source descriptions. Cross-paper implications and illustrative failure conditions are survey analysis, not new experiments, reproduction, or causal effect estimates.","paragraph_purpose":"Synthesize trade-offs across data, interfaces, memory and feedback.","section_anchor":"methods-c28","sources":[{"body_sha256":"75d8af953769a53bf75913908e2bb6185c6b338c8d89b7304e7fd8cfb79934e8","citation_key":"arxiv_2306_06070","pdf_pages_read":[1,5,6,7,9],"primary_url":"https://arxiv.org/pdf/2306.06070v3","sections_read":["3.1 Candidate Generation","3.2 Action Prediction","4.1 Experimental Setup","4.2 Evaluation","6 Limitations"],"version_read":3},{"body_sha256":"169bc7694b456530ce8a6f0336b7efc33a32a21a731e1656caf4934e83f59334","citation_key":"arxiv_2412_16256","pdf_pages_read":[1,3,4,9],"primary_url":"https://arxiv.org/pdf/2412.16256v2","sections_read":["Front matter and abstract","2.1 Large-scale Diverse GUI Data Synthesizing","2.1.2 Context-aware Data Extension from Trajectories","6 Limitations"],"version_read":2},{"body_sha256":"bf6e672269ac55c35870ab2b93d652b472c30fd6ad9c98797b00127d58132305","citation_key":"arxiv_2605_18652","pdf_pages_read":[1,3,4,5,6,7],"primary_url":"https://arxiv.org/pdf/2605.18652v1","sections_read":["3 Data Curation","4.1 Framework","4.2 Memory System Design","4.3 Training MementoCore","4.4 Benchmarking","5.1 Experimental Setup"],"version_read":1},{"body_sha256":"705de6c54fa4eec0e5f81792cd323695e1d0274333886973cbad82d3c436fdb0","citation_key":"arxiv_2607_28227","pdf_pages_read":[1,2,3,4,6,9,10,13,14,15,44],"primary_url":"https://arxiv.org/pdf/2607.28227v1","sections_read":["2.1 System Overview and Action Space","2.2.2 Real-Device Runtime","2.2.3 Hybrid GUI+CLI","2.4.1 SFT","2.4.2 Action RL","2.4.3 Online RL","7 Limitations"],"version_read":1}]},{"citation_keys":["arxiv_2306_06070","arxiv_2410_05243","arxiv_2412_16256","arxiv_2602_20502","arxiv_2504_14603","arxiv_2607_28227","arxiv_2605_17320"],"claim":"Design choices create distinct useful capabilities and failure boundaries; rows are not measured rankings.","claim_id":"methods-design-table","claim_type":"qualitative_survey_synthesis","sources":[{"body_sha256":"75d8af953769a53bf75913908e2bb6185c6b338c8d89b7304e7fd8cfb79934e8","citation_key":"arxiv_2306_06070","primary_url":"https://arxiv.org/pdf/2306.06070v3"},{"body_sha256":"b166b8e09f604e64067d12ded4689e7582df106988b28f018651835843b04e68","citation_key":"arxiv_2410_05243","primary_url":"https://arxiv.org/pdf/2410.05243v3"},{"body_sha256":"169bc7694b456530ce8a6f0336b7efc33a32a21a731e1656caf4934e83f59334","citation_key":"arxiv_2412_16256","primary_url":"https://arxiv.org/pdf/2412.16256v2"},{"body_sha256":"ec8339a07ee3c83d4f416a27cf9e8bf254efa26968b7bc9fdf3e07ff09e38ab6","citation_key":"arxiv_2602_20502","primary_url":"https://arxiv.org/pdf/2602.20502v2"},{"body_sha256":"ceecfb496791b8b101275520066eb69c3289c7c727a9e69cc844219598acece5","citation_key":"arxiv_2504_14603","primary_url":"https://arxiv.org/pdf/2504.14603v2"},{"body_sha256":"705de6c54fa4eec0e5f81792cd323695e1d0274333886973cbad82d3c436fdb0","citation_key":"arxiv_2607_28227","primary_url":"https://arxiv.org/pdf/2607.28227v1"},{"body_sha256":"cd6d543feb3ec804b35f91ae9dd897a563038f9230067abd5eaafd88976e46af","citation_key":"arxiv_2605_17320","primary_url":"https://arxiv.org/pdf/2605.17320v1"}]}],"format":"semantic_html_fragment","fresh_searches":[{"queried_at":"2026-10-18 09:58:26 UTC","queries":[{"domains":["arxiv.org"],"q":"SeeClick CogAgent ShowUI GUI grounding screenshot model paper"},{"domains":["arxiv.org"],"q":"Agent S UFO AppAgent Mobile-Agent computer use planning memory paper"},{"domains":["arxiv.org"],"q":"UI-TARS OpenCUA Aguvis native GUI agent training trajectories paper"},{"domains":["arxiv.org"],"q":"WebRL Test-Time Interaction UI-TARS-2 web agent reinforcement learning"}],"raw_search_response_sha256":"e9391fec1e4d34f0b0cbe7d849b67ad9d6f73c03ab6372623ee169ddb827ac6a","results":[{"canonical_identity":"arxiv:2501.12326","decision":"selected and full-text sections read","selected_for_methods":true,"title":"UI-TARS: Pioneering Automated GUI Interaction with Native Agents","url":"https://arxiv.org/abs/2501.12326"},{"canonical_identity":"arxiv:2412.04454","decision":"selected and full-text sections read","selected_for_methods":true,"title":"Aguvis: Unified Pure Vision Agents for Autonomous GUI Interaction","url":"https://arxiv.org/abs/2412.04454"},{"canonical_identity":"arxiv:2401.10935","decision":"selected and full-text sections read","selected_for_methods":true,"title":"SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agents","url":"https://arxiv.org/abs/2401.10935"},{"canonical_identity":"arxiv:2509.02544","decision":"selected and full-text sections read","selected_for_methods":true,"title":"UI-TARS-2 Technical Report: Advancing GUI Agent with Multi-Turn Reinforcement Learning","url":"https://arxiv.org/abs/2509.02544"},{"canonical_identity":"arxiv:2404.07972","decision":"not selected: benchmark/bibliography-only hit or unrelated result; search hit is not method evidence","selected_for_methods":false,"title":"OSWORLD: Benchmarking Multimodal Agents for","url":"https://arxiv.org/pdf/2404.07972"},{"canonical_identity":"arxiv:2505.23301","decision":"not selected: benchmark/bibliography-only hit or unrelated result; search hit is not method evidence","selected_for_methods":false,"title":"Fig. 4. Screenshot of the user study developed with PsychoPy.","url":"https://arxiv.org/pdf/2505.23301"}],"search_id":"search-batch-1"},{"queried_at":"2026-10-18 09:58:58 UTC","queries":[{"domains":["arxiv.org"],"q":"\"CogAgent\" \"ShowUI\" graphical user interfaces"},{"domains":["arxiv.org"],"q":"\"UFO\" \"AppAgent\" agent Windows smartphone"},{"domains":["arxiv.org"],"q":"\"ActionEngine\" \"TClone\" computer use 2026"},{"domains":["arxiv.org"],"q":"\"Qwen-UI-Agent\" \"AsyncWebRL\" \"MementoGUI\""}],"raw_search_response_sha256":"14feae281a95371ea850f96a2e7b0738da76789c367d5cacc06473c86657068f","results":[{"canonical_identity":"arxiv:2404.07972","decision":"not selected: benchmark/bibliography-only hit or unrelated result; search hit is not method evidence","selected_for_methods":false,"title":"OSWORLD: Benchmarking Multimodal Agents for","url":"https://arxiv.org/pdf/2404.07972"},{"canonical_identity":"arxiv:2405.14573","decision":"not selected: benchmark/bibliography-only hit or unrelated result; search hit is not method evidence","selected_for_methods":false,"title":"ANDROIDWORLD: A Dynamic Benchmarking","url":"https://arxiv.org/pdf/2405.14573"}],"search_id":"search-batch-2"},{"queried_at":"2026-10-18 09:59:12 UTC","queries":[{"domains":["arxiv.org"],"q":"\"Qwen-UI-Agent Technical Report\""},{"domains":["arxiv.org"],"q":"\"AsyncWebRL\""},{"domains":["arxiv.org"],"q":"\"MementoGUI\""},{"domains":["arxiv.org"],"q":"\"TClone\" \"GUI\""}],"raw_search_response_sha256":"1c0fd4edbd92d9b0aec876698893d73b490693e4d8ccb6500ef7d8db9808b5cd","results":[{"canonical_identity":"arxiv:2607.28227","decision":"selected and full-text sections read","selected_for_methods":true,"title":"Qwen-UI-Agent Technical Report: Toward Next-Generation Real-World Centric Foundation GUI Agents","url":"https://arxiv.org/abs/2607.28227"},{"canonical_identity":"arxiv:2605.18652","decision":"selected and full-text sections read","selected_for_methods":true,"title":"MementoGUI: Learning Agentic Multimodal Memory Control for Long-Horizon GUI Agents","url":"https://arxiv.org/abs/2605.18652"},{"canonical_identity":"arxiv:2606.05597","decision":"selected and full-text sections read","selected_for_methods":true,"title":"AsyncWebRL: Efficient Multi-Step RL for Visual Web Agents","url":"https://arxiv.org/abs/2606.05597"},{"canonical_identity":"arxiv:2605.17320","decision":"selected and full-text sections read","selected_for_methods":true,"title":"TClone: Low-Latency Forking of Live GUI Environments for Computer-Use Agents","url":"https://arxiv.org/abs/2605.17320"}],"search_id":"search-batch-3"},{"queried_at":"2026-10-18 10:03:06 UTC","queries":[{"domains":["arxiv.org"],"q":"\"Agent S\" \"An Open Agentic Framework\""},{"domains":["arxiv.org"],"q":"\"ShowUI\" \"One Vision-Language-Action Model\""},{"domains":["arxiv.org"],"q":"\"OpenCUA\" \"Open Foundations\""},{"domains":["arxiv.org"],"q":"\"UFO\" \"UI-Focused Agent\""}],"raw_search_response_sha256":"cd79cdf2a84e1e22b62f7ceadd7b054c5980a8ed046d1b27d8ed894649773c6b","results":[{"canonical_identity":"arxiv:2410.08164","decision":"selected and full-text sections read","selected_for_methods":true,"title":"Agent S: An Open Agentic Framework that Uses Computers Like a Human","url":"https://arxiv.org/abs/2410.08164"},{"canonical_identity":"arxiv:2411.17465","decision":"selected and full-text sections read","selected_for_methods":true,"title":"ShowUI: One Vision-Language-Action Model for GUI Visual Agent","url":"https://arxiv.org/abs/2411.17465"},{"canonical_identity":"arxiv:2402.07939","decision":"selected and full-text sections read","selected_for_methods":true,"title":"UFO: A UI-Focused Agent for Windows OS Interaction","url":"https://arxiv.org/abs/2402.07939"},{"canonical_identity":"arxiv:2506.03530","decision":"not selected: benchmark/bibliography-only hit or unrelated result; search hit is not method evidence","selected_for_methods":false,"title":"How Far Are We from Predicting Missing Modalities with Foundation Models?","url":"https://arxiv.org/pdf/2506.03530"},{"canonical_identity":"arxiv:2508.09123","decision":"selected and full-text sections read","selected_for_methods":true,"title":"OpenCUA: Open Foundations for Computer-Use Agents","url":"https://arxiv.org/abs/2508.09123"},{"canonical_identity":"arxiv:2405.14573","decision":"not selected: benchmark/bibliography-only hit or unrelated result; search hit is not method evidence","selected_for_methods":false,"title":"ANDROIDWORLD: A Dynamic Benchmarking","url":"https://arxiv.org/pdf/2405.14573"},{"canonical_identity":"arxiv:2404.07972","decision":"not selected: benchmark/bibliography-only hit or unrelated result; search hit is not method evidence","selected_for_methods":false,"title":"OSWORLD: Benchmarking Multimodal Agents for","url":"https://arxiv.org/pdf/2404.07972"}],"search_id":"search-batch-4"}],"integrated_manuscript_sha256":"37a0aa207a16c5acabccf4401eb7e34b47898b58370ad103c7ba855c85aff569","integration_revision":"Applied independent review for precision, repetition and acronym definitions; original reading receipts retained.","language":"en","limitations":["Selected sections, not every page of every paper, were read; each source lists actual pages or sections.","Primary author reports describe implementations and experiments; no linked project code or agent benchmark was executed.","Source-specific evaluation limits are kept in each note and the prose; cross-paper synthesis is not a matched benchmark study.","Data collection and deployment modalities are separated; screenshot inference does not imply absence of structural labels during training.","Claims about generalization are scoped to mechanisms and documented assumptions, not predictions of safety or deployment success."],"manuscript_path":"research/survey/sections/methods.html","manuscript_sha256":"f3394c76d9be5e32e1d4e455e749e6791a4138a498a59cd430460cf72a6c1da1","review_scope":{"abstract_only_sources_used_for_method_claims":0,"not_a_systematic_exhaustiveness_claim":true,"not_all_catalog_papers_fulltext_reviewed":true,"not_an_independent_reproduction":true,"numeric_leaderboard_comparisons_in_section":0,"papers_read_in_entirety":0,"papers_with_selected_fulltext_sections_read":38,"selection_policy":"Purposive mechanism diversity across observation, grounding, planning, memory, training and runtime; foundational systems plus recent primary-search additions and user-requested Jev-Mobile.","source_material_policy":"Full paper bytes and full extracted text remain only in local cache. This public evidence file contains original annotations, bibliographic facts, locators, and fingerprints, not copied abstracts or source paragraphs.","word_count_including_table":3266},"schema_version":1,"search_deduplication":{"duplicate_result_occurrences_removed":3,"identity_rule":"Exact arXiv numeric identity, stripping version only for search-hit grouping; all cited claims retain the actual version read.","unique_results":[{"batches":["search-batch-1.json"],"canonical_identity":"arxiv:2501.12326","selected_for_methods":true,"urls":["https://arxiv.org/abs/2501.12326"]},{"batches":["search-batch-1.json"],"canonical_identity":"arxiv:2412.04454","selected_for_methods":true,"urls":["https://arxiv.org/abs/2412.04454"]},{"batches":["search-batch-1.json"],"canonical_identity":"arxiv:2401.10935","selected_for_methods":true,"urls":["https://arxiv.org/abs/2401.10935"]},{"batches":["search-batch-1.json"],"canonical_identity":"arxiv:2509.02544","selected_for_methods":true,"urls":["https://arxiv.org/abs/2509.02544"]},{"batches":["search-batch-1.json","search-batch-2.json","search-batch-4.json"],"canonical_identity":"arxiv:2404.07972","selected_for_methods":false,"urls":["https://arxiv.org/pdf/2404.07972"]},{"batches":["search-batch-1.json"],"canonical_identity":"arxiv:2505.23301","selected_for_methods":false,"urls":["https://arxiv.org/pdf/2505.23301"]},{"batches":["search-batch-2.json","search-batch-4.json"],"canonical_identity":"arxiv:2405.14573","selected_for_methods":false,"urls":["https://arxiv.org/pdf/2405.14573"]},{"batches":["search-batch-3.json"],"canonical_identity":"arxiv:2607.28227","selected_for_methods":true,"urls":["https://arxiv.org/abs/2607.28227"]},{"batches":["search-batch-3.json"],"canonical_identity":"arxiv:2605.18652","selected_for_methods":true,"urls":["https://arxiv.org/abs/2605.18652"]},{"batches":["search-batch-3.json"],"canonical_identity":"arxiv:2606.05597","selected_for_methods":true,"urls":["https://arxiv.org/abs/2606.05597"]},{"batches":["search-batch-3.json"],"canonical_identity":"arxiv:2605.17320","selected_for_methods":true,"urls":["https://arxiv.org/abs/2605.17320"]},{"batches":["search-batch-4.json"],"canonical_identity":"arxiv:2410.08164","selected_for_methods":true,"urls":["https://arxiv.org/abs/2410.08164"]},{"batches":["search-batch-4.json"],"canonical_identity":"arxiv:2411.17465","selected_for_methods":true,"urls":["https://arxiv.org/abs/2411.17465"]},{"batches":["search-batch-4.json"],"canonical_identity":"arxiv:2402.07939","selected_for_methods":true,"urls":["https://arxiv.org/abs/2402.07939"]},{"batches":["search-batch-4.json"],"canonical_identity":"arxiv:2506.03530","selected_for_methods":false,"urls":["https://arxiv.org/pdf/2506.03530"]},{"batches":["search-batch-4.json"],"canonical_identity":"arxiv:2508.09123","selected_for_methods":true,"urls":["https://arxiv.org/abs/2508.09123"]}]},"section":"methods","selection_exclusions":[{"decision":"not used in this section","read_scope":"screening of table-of-contents/page headers only; not counted among full-text section reads","reason":"Redundant adjacent textual-state treatment for this selected GUI-method cohort; no full-text-method claim is made.","resource_id":"arxiv:2410.02810","title":"StateAct: Enhancing LLM Base Agents via Self-prompting and State-tracking"}],"sources":[{"authors":["Xiang Deng","Yu Gu","Boyuan Zheng","Shijie Chen","Samuel Stevens","Boshi Wang","Huan Sun","Yu Su"],"bibliography_basis":"frozen catalog retained arXiv bibliography and exact-version primary full text","body_bytes":7635816,"body_sha256":"75d8af953769a53bf75913908e2bb6185c6b338c8d89b7304e7fd8cfb79934e8","citation_key":"arxiv_2306_06070","final_url":"https://arxiv.org/pdf/2306.06070v3","first_submitted_date":"2023-06-09","http_status":200,"media_type":"application/pdf","primary_url":"https://arxiv.org/pdf/2306.06070v3","read_evidence":{"boundary":"Offline steps receive ground-truth history; cross-task, website and domain splits differ; collected English/U.S.-centric sites limit coverage.","entire_paper_read":false,"evidence_level":"fulltext_sections","mechanism":"DOM ranking followed by grouped multiple-choice action selection.","pages":[1,5,6,7,9],"read_at":"2026-10-18T10:07:45.338527+00:00","sections":["3.1 Candidate Generation","3.2 Action Prediction","4.1 Experimental Setup","4.2 Evaluation","6 Limitations"]},"resource_id":"arxiv:2306.06070","retained_abstract_sha256":"de055dfa452130f3576839c99c007053c62a0c96c9bfbcdd79ecad1a9f74f830","retrieved_at":"2026-10-18T10:00:08.226086+00:00","title":"Mind2Web: Towards a Generalist Agent for the Web","version_binding":{"embedded_arxiv_version_marker_found":true,"limitation":null,"method":"Exact official version URL; retained bibliography; embedded marker when available.","requested_exact_version":"2306.06070v3"},"version_read":3,"version_updated_date":"2023-12-09","versioned_abstract_url":"https://arxiv.org/abs/2306.06070v3"},{"authors":["Boyuan Zheng","Boyu Gou","Jihyung Kil","Huan Sun","Yu Su"],"bibliography_basis":"frozen catalog retained arXiv bibliography and exact-version primary full text","body_bytes":9044518,"body_sha256":"ac281a3c5cdbda428e14605e9d96035fd35af82073ce46117c27c34792eea041","citation_key":"arxiv_2401_01614","final_url":"https://arxiv.org/pdf/2401.01614v2","first_submitted_date":"2024-01-03","http_status":200,"media_type":"application/pdf","primary_url":"https://arxiv.org/pdf/2401.01614v2","read_evidence":{"boundary":"Oracle grounding uses humans; marked-image and textual-choice grounding differ. Offline reference paths can reject alternative successful online executions.","entire_paper_read":false,"evidence_level":"fulltext_sections","mechanism":"Screenshot-based action description, then structured element grounding.","pages":[1,2,3,4,7],"read_at":"2026-10-18T10:07:45.338668+00:00","sections":["2.1 Formulation","2.2 Action Generation","2.3 Action Grounding","3.2 Methods","4.3 Analysis"]},"resource_id":"arxiv:2401.01614","retained_abstract_sha256":"a005fb14b21cf17521f4ea966e555d9963092b709bd9280c6e72a9bf0f5fa93a","retrieved_at":"2026-10-18T10:00:23.715060+00:00","title":"GPT-4V(ision) is a Generalist Web Agent, if Grounded","version_binding":{"embedded_arxiv_version_marker_found":true,"limitation":null,"method":"Exact official version URL; retained bibliography; embedded marker when available.","requested_exact_version":"2401.01614v2"},"version_read":2,"version_updated_date":"2024-03-12","versioned_abstract_url":"https://arxiv.org/abs/2401.01614v2"},{"authors":["Wenyi Hong","Weihan Wang","Qingsong Lv","Jiazheng Xu","Wenmeng Yu","Junhui Ji","Yan Wang","Zihan Wang","Yuxuan Zhang","Juanzi Li","Bin Xu","Yuxiao Dong","Ming Ding","Jie Tang"],"bibliography_basis":"frozen catalog retained arXiv bibliography and exact-version primary full text","body_bytes":15601893,"body_sha256":"d9cb086d826eaee9f919e0cbf142193b77f476b0aef6260b34408abe142fe13b","citation_key":"arxiv_2312_08914","final_url":"https://arxiv.org/pdf/2312.08914v3","first_submitted_date":"2023-12-14","http_status":200,"media_type":"application/pdf","primary_url":"https://arxiv.org/pdf/2312.08914v3","read_evidence":{"boundary":"Paper model architecture is distinct from the separately linked CogAgent-9B release; screenshots at inference do not imply structure-free data construction.","entire_paper_read":false,"evidence_level":"fulltext_sections","mechanism":"Low-resolution VLM plus lightweight high-resolution cross-attention; OCR and DOM-derived GUI grounding supervision.","pages":[1,3,4,5],"read_at":"2026-10-18T10:07:45.338670+00:00","sections":["2.1 Architecture","2.2 High-Resolution Cross-Module","2.3 Pre-training","2.4 Multi-task Fine-tuning"]},"resource_id":"arxiv:2312.08914","retained_abstract_sha256":"755198d83a3af9fe6d032f1446ec9bf57bf7e6f314d37787ce8a6aaa16bddc47","retrieved_at":"2026-10-18T10:00:43.171632+00:00","title":"CogAgent: A Visual Language Model for GUI Agents","version_binding":{"embedded_arxiv_version_marker_found":true,"limitation":null,"method":"Exact official version URL; retained bibliography; embedded marker when available.","requested_exact_version":"2312.08914v3"},"version_read":3,"version_updated_date":"2024-12-27","versioned_abstract_url":"https://arxiv.org/abs/2312.08914v3"},{"authors":["Kanzhi Cheng","Qiushi Sun","Yougang Chu","Fangzhi Xu","Yantao Li","Jianbing Zhang","Zhiyong Wu"],"bibliography_basis":"frozen catalog retained arXiv bibliography and exact-version primary full text","body_bytes":10795836,"body_sha256":"b62bd796d8e6d45e13b41ed41a55d021a9df3329bf500d14efed129e8a1e04e2","citation_key":"arxiv_2401_10935","final_url":"https://arxiv.org/pdf/2401.10935v2","first_submitted_date":"2024-01-17","http_status":200,"media_type":"application/pdf","primary_url":"https://arxiv.org/pdf/2401.10935v2","read_evidence":{"boundary":"Action support focuses on click/type; agent-specific training remains necessary for multistep tasks. Training uses structured web annotations.","entire_paper_read":false,"evidence_level":"fulltext_sections","mechanism":"Coordinate-text prediction trained with web/mobile grounding and inverse OCR tasks.","pages":[1,3,4,9],"read_at":"2026-10-18T10:07:45.338671+00:00","sections":["3.1 GUI Grounding","3.2 Data Construction","3.3 Training Details","Limitations"]},"resource_id":"arxiv:2401.10935","retained_abstract_sha256":"c92ec9323eec9362aa07984f9fb1cbfbe98ecae426a9f8adc9d436328a87aed3","retrieved_at":"2026-10-18T10:02:26.938817+00:00","title":"SeeClick: Harnessing GUI Grounding for Advanced Visual GUI Agents","version_binding":{"embedded_arxiv_version_marker_found":true,"limitation":null,"method":"Exact official version URL; retained bibliography; embedded marker when available.","requested_exact_version":"2401.10935v2"},"version_read":2,"version_updated_date":"2024-02-23","versioned_abstract_url":"https://arxiv.org/abs/2401.10935v2"},{"authors":["Hongliang He","Wenlin Yao","Kaixin Ma","Wenhao Yu","Yong Dai","Hongming Zhang","Zhenzhong Lan","Dong Yu"],"bibliography_basis":"frozen catalog retained arXiv bibliography and exact-version primary full text","body_bytes":18497629,"body_sha256":"476ed4a064311f93414f5d5d17d90f10d07d2922bc05e09ee89cea26b617a2cb","citation_key":"arxiv_2401_13919","final_url":"https://arxiv.org/pdf/2401.13919v4","first_submitted_date":"2024-01-25","http_status":200,"media_type":"application/pdf","primary_url":"https://arxiv.org/pdf/2401.13919v4","read_evidence":{"boundary":"Keeps three recent observations plus thought/action history; correction consumes steps; initial benchmark omits access-blocking login/CAPTCHA sites.","entire_paper_read":false,"evidence_level":"fulltext_sections","mechanism":"DOM-derived screenshot marks and auxiliary element text drive a browser action loop.","pages":[1,3,4],"read_at":"2026-10-18T10:07:45.338672+00:00","sections":["3.1 Browsing Environment","3.2 Interaction Formulation","3.3 Observation Space","3.4 Action Space","4.1 Website Selection"]},"resource_id":"arxiv:2401.13919","retained_abstract_sha256":"dbc7121c9606c5987df0941b849c216ad8731bebcc74a061cf3f154379d7b8da","retrieved_at":"2026-10-18T10:03:33.620761+00:00","title":"WebVoyager: Building an End-to-End Web Agent with Large Multimodal Models","version_binding":{"embedded_arxiv_version_marker_found":true,"limitation":null,"method":"Exact official version URL; retained bibliography; embedded marker when available.","requested_exact_version":"2401.13919v4"},"version_read":4,"version_updated_date":"2024-06-06","versioned_abstract_url":"https://arxiv.org/abs/2401.13919v4"},{"authors":["Chi Zhang","Zhao Yang","Jiaxuan Liu","Yanda Li","Yucheng Han","Xin Chen","Zebiao Huang","Bin Fu","Gang Yu"],"bibliography_basis":"frozen catalog retained arXiv bibliography and exact-version primary full text","body_bytes":3786389,"body_sha256":"800ec0437da9183b77fb6fe3db7b4f64e2ff23b88efc887f6809cd00db0af4c8","citation_key":"arxiv_2312_13771","final_url":"https://arxiv.org/pdf/2312.13771v3","first_submitted_date":"2023-12-21","http_status":200,"media_type":"application/pdf","primary_url":"https://arxiv.org/pdf/2312.13771v3","read_evidence":{"boundary":"Android XML supplies identifiers and screenshot overlays; model-chosen Exit is not an independent success verifier.","entire_paper_read":false,"evidence_level":"fulltext_sections","mechanism":"App exploration or human demonstrations produce element-effect documents for later execution.","pages":[1,4,5],"read_at":"2026-10-18T10:07:45.338673+00:00","sections":["3.1 Environment and Action Space","3.2 Exploration","3.3 Deployment"]},"resource_id":"arxiv:2312.13771","retained_abstract_sha256":"61e54d83304755ce89c2ab69f7bb93a7fdc4d68cb5182daad9b5210f809d7ae1","retrieved_at":"2026-10-18T10:04:11.632637+00:00","title":"AppAgent: Multimodal Agents as Smartphone Users","version_binding":{"embedded_arxiv_version_marker_found":true,"limitation":null,"method":"Exact official version URL; retained bibliography; embedded marker when available.","requested_exact_version":"2312.13771v3"},"version_read":3,"version_updated_date":"2026-07-05","versioned_abstract_url":"https://arxiv.org/abs/2312.13771v3"},{"authors":["Junyang Wang","Haiyang Xu","Jiabo Ye","Ming Yan","Weizhou Shen","Ji Zhang","Fei Huang","Jitao Sang"],"bibliography_basis":"frozen catalog retained arXiv bibliography and exact-version primary full text","body_bytes":27114964,"body_sha256":"6b511d598fa2f9bea3692d3acf028ebfe46b36b83181baab60a0152955ebec0d","citation_key":"arxiv_2401_16158","final_url":"https://arxiv.org/pdf/2401.16158v2","first_submitted_date":"2024-01-29","http_status":200,"media_type":"application/pdf","primary_url":"https://arxiv.org/pdf/2401.16158v2","read_evidence":{"boundary":"Although localization avoids XML, reported experiments use Android; other operating systems are future work.","entire_paper_read":false,"evidence_level":"fulltext_sections","mechanism":"OCR and icon detection/CLIP ground textual action requests; reflection checks state changes and completion.","pages":[1,2,4],"read_at":"2026-10-18T10:07:45.338674+00:00","sections":["2.1 Visual Perception","2.2 Instruction Execution","3 Experiments"]},"resource_id":"arxiv:2401.16158","retained_abstract_sha256":"81ac041a3e5f98bfc23c91f768b32ad7adc3f28d8c9d55a285bb29906c8b5a9c","retrieved_at":"2026-10-18T10:04:23.217337+00:00","title":"Mobile-Agent: Autonomous Multi-Modal Mobile Device Agent with Visual Perception","version_binding":{"embedded_arxiv_version_marker_found":true,"limitation":null,"method":"Exact official version URL; retained bibliography; embedded marker when available.","requested_exact_version":"2401.16158v2"},"version_read":2,"version_updated_date":"2024-04-18","versioned_abstract_url":"https://arxiv.org/abs/2401.16158v2"},{"authors":["Chaoyun Zhang","Liqun Li","Shilin He","Xu Zhang","Bo Qiao","Si Qin","Minghua Ma","Yu Kang","Qingwei Lin","Saravan Rajmohan","Dongmei Zhang","Qi Zhang"],"bibliography_basis":"frozen catalog retained arXiv bibliography and exact-version primary full text","body_bytes":11720609,"body_sha256":"55680fbdd9e98455e4b8c93474e3f367f76d86d170b722433ea2e298fd0a8acf","citation_key":"arxiv_2402_07939","final_url":"https://arxiv.org/pdf/2402.07939v5","first_submitted_date":"2024-02-08","http_status":200,"media_type":"application/pdf","primary_url":"https://arxiv.org/pdf/2402.07939v5","read_evidence":{"boundary":"Backend coverage remains a limit: inaccessible controls cannot be operated. PENDING requests user confirmation; model FINISH is not external verification.","entire_paper_read":false,"evidence_level":"fulltext_sections","mechanism":"Windows HostAgent selects/switches applications; AppAgent uses screenshots and UI controls; pywinauto grounds control operations.","pages":[1,4,5,6,7,14],"read_at":"2026-10-18T10:11:21.810586+00:00","sections":["3.1 Agent Framework","3.4 Action Execution","5 Limitations"]},"resource_id":"arxiv:2402.07939","retained_abstract_sha256":"533acb1c9aa3f1c77adb2c67779da86e3d5de95264ba3ec1f81d803f67da7753","retrieved_at":"2026-10-18T10:05:18.193134+00:00","title":"UFO: A UI-Focused Agent for Windows OS Interaction","version_binding":{"embedded_arxiv_version_marker_found":true,"limitation":null,"method":"Exact official version URL; retained bibliography; embedded marker when available.","requested_exact_version":"2402.07939v5"},"version_read":5,"version_updated_date":"2024-05-23","versioned_abstract_url":"https://arxiv.org/abs/2402.07939v5"},{"authors":["Saaket Agashe","Jiuzhou Han","Shuyu Gan","Jiachen Yang","Ang Li","Xin Eric Wang"],"bibliography_basis":"frozen catalog retained arXiv bibliography and exact-version primary full text","body_bytes":26111150,"body_sha256":"ee8c9b38e20b5f747bd78828d96beab0b266eb99baaafe44c3936ff0d89cb92b","citation_key":"arxiv_2410_08164","final_url":"https://arxiv.org/pdf/2410.08164v1","first_submitted_date":"2024-10-10","http_status":200,"media_type":"application/pdf","primary_url":"https://arxiv.org/pdf/2410.08164v1","read_evidence":{"boundary":"Experience learning here is memory/retrieval, not demonstrated gradient RL. Self-evaluation populates memory; bootstrap/online memory availability is part of evaluation conditions.","entire_paper_read":false,"evidence_level":"fulltext_sections","mechanism":"Manager retrieves narrative experience and decomposes tasks; workers retrieve episodic experience, reflect and return FAIL for replanning; OCR augments accessibility tree and single discrete actions provide feedback.","pages":[1,4,5,6,7],"read_at":"2026-10-18T10:11:21.810729+00:00","sections":["3.1 Experience-Augmented Hierarchical Planning","3.2 Self-Supervised Memory Construction","3.3 Agent-Computer Interface","4.1 Experimental Setup"]},"resource_id":"arxiv:2410.08164","retained_abstract_sha256":"284888d16b049767f76448f23f5d35fda25b109fd67f450105950173ae81c97b","retrieved_at":"2026-10-18T10:05:49.111421+00:00","title":"Agent S: An Open Agentic Framework that Uses Computers Like a Human","version_binding":{"embedded_arxiv_version_marker_found":true,"limitation":null,"method":"Exact official version URL; retained bibliography; embedded marker when available.","requested_exact_version":"2410.08164v1"},"version_read":1,"version_updated_date":"2024-10-10","versioned_abstract_url":"https://arxiv.org/abs/2410.08164v1"},{"authors":["Kevin Qinghong Lin","Linjie Li","Difei Gao","Zhengyuan Yang","Shiwei Wu","Zechen Bai","Weixian Lei","Lijuan Wang","Mike Zheng Shou"],"bibliography_basis":"frozen catalog retained arXiv bibliography and exact-version primary full text","body_bytes":20140819,"body_sha256":"8bc023be285e71dab992fb5aeffc2bcbcefe155a67b6ce5e6e7e3a3499bb5db5","citation_key":"arxiv_2411_17465","final_url":"https://arxiv.org/pdf/2411.17465v1","first_submitted_date":"2024-11-26","http_status":200,"media_type":"application/pdf","primary_url":"https://arxiv.org/pdf/2411.17465v1","read_evidence":{"boundary":"Action README and normalized JSON provide an interface, not identical platform semantics; history masking differs by task. No universal efficiency ratio claimed.","entire_paper_read":false,"evidence_level":"fulltext_sections","mechanism":"Connected RGB-patch regions identify redundant visual tokens while retained tokens keep positions; interleaves screenshot-action histories and packs multiple grounding queries per screenshot.","pages":[1,3,4,5,6],"read_at":"2026-10-18T10:11:21.810731+00:00","sections":["2.1 UI-Guided Visual Token Selection","2.2 Interleaved VLA Streaming","2.3 GUI Instructional Tuning"]},"resource_id":"arxiv:2411.17465","retained_abstract_sha256":"8c8f5c0a7fc5f194ffa6ca914619b9402c947d15f100b124505883ad8ff23ac7","retrieved_at":"2026-10-18T10:07:06.161559+00:00","title":"ShowUI: One Vision-Language-Action Model for GUI Visual Agent","version_binding":{"embedded_arxiv_version_marker_found":true,"limitation":null,"method":"Exact official version URL; retained bibliography; embedded marker when available.","requested_exact_version":"2411.17465v1"},"version_read":1,"version_updated_date":"2024-11-26","versioned_abstract_url":"https://arxiv.org/abs/2411.17465v1"},{"authors":["Boyu Gou","Ruohan Wang","Boyuan Zheng","Yanan Xie","Cheng Chang","Yiheng Shu","Huan Sun","Yu Su"],"bibliography_basis":"frozen catalog retained arXiv bibliography and exact-version primary full text","body_bytes":18205895,"body_sha256":"b166b8e09f604e64067d12ded4689e7582df106988b28f018651835843b04e68","citation_key":"arxiv_2410_05243","final_url":"https://arxiv.org/pdf/2410.05243v3","first_submitted_date":"2024-10-07","http_status":200,"media_type":"application/pdf","primary_url":"https://arxiv.org/pdf/2410.05243v3","read_evidence":{"boundary":"Despite web-synthesis motivation, final training corpus also incorporates Android data. Purely visual deployment does not imply structure-free collection; high resolution trades against compute.","entire_paper_read":false,"evidence_level":"fulltext_sections","mechanism":"SeeAct-V separates screenshot planner and coordinate grounder; hybrid HTML/rule/VLM referring expressions train a high-resolution grounding model.","pages":[1,3,4,5],"read_at":"2026-10-18T10:11:21.810732+00:00","sections":["2.1 Overview","2.2 Data Construction","2.3 Model Design"]},"resource_id":"arxiv:2410.05243","retained_abstract_sha256":"08ece826d135b9ab9dfed2a3baf9deeab99489120da749cd67517b636649c5a6","retrieved_at":"2026-10-18T10:08:19.268422+00:00","title":"Navigating the Digital World as Humans Do: Universal Visual Grounding for GUI Agents","version_binding":{"embedded_arxiv_version_marker_found":true,"limitation":null,"method":"Exact official version URL; retained bibliography; embedded marker when available.","requested_exact_version":"2410.05243v3"},"version_read":3,"version_updated_date":"2025-06-17","versioned_abstract_url":"https://arxiv.org/abs/2410.05243v3"},{"authors":["Zhiyong Wu","Zhenyu Wu","Fangzhi Xu","Yian Wang","Qiushi Sun","Chengyou Jia","Kanzhi Cheng","Zichen Ding","Liheng Chen","Paul Pu Liang","Yu Qiao"],"bibliography_basis":"frozen catalog retained arXiv bibliography and exact-version primary full text","body_bytes":2726914,"body_sha256":"ed7ba8eb36d50e36b14e480a3cb89ac9f8fe825d28f763acd8210dcb7ade8bd0","citation_key":"arxiv_2410_23218","final_url":"https://arxiv.org/pdf/2410.23218v1","first_submitted_date":"2024-10-30","http_status":200,"media_type":"application/pdf","primary_url":"https://arxiv.org/pdf/2410.23218v1","read_evidence":{"boundary":"Training APIs and platform exploration differ; ScreenSpot vs corrected ScreenSpot-V2 and planner-assisted vs direct grounding are distinct evaluation conditions.","entire_paper_read":false,"evidence_level":"fulltext_sections","mechanism":"Grounding pretraining precedes action imitation; collection uses HTML plus OS-specific accessibility APIs; basic click/type/scroll operations and extensible custom actions standardize heterogeneous data.","pages":[1,3,4,5,6],"read_at":"2026-10-18T10:11:21.810733+00:00","sections":["3.1 Task Formulation and Training","3.2 Grounding Data Collection","3.3 Unified Action Space","4.1 Evaluation Details"]},"resource_id":"arxiv:2410.23218","retained_abstract_sha256":"e0dbd8ef8807e7886c02ed6296c9d188503fe7b676062fcff1890fd0652f10f3","retrieved_at":"2026-10-18T10:08:56.970087+00:00","title":"OS-ATLAS: A Foundation Action Model for Generalist GUI Agents","version_binding":{"embedded_arxiv_version_marker_found":true,"limitation":null,"method":"Exact official version URL; retained bibliography; embedded marker when available.","requested_exact_version":"2410.23218v1"},"version_read":1,"version_updated_date":"2024-10-30","versioned_abstract_url":"https://arxiv.org/abs/2410.23218v1"},{"authors":["Yiheng Xu","Zekun Wang","Junli Wang","Dunjie Lu","Tianbao Xie","Amrita Saha","Doyen Sahoo","Tao Yu","Caiming Xiong"],"bibliography_basis":"frozen catalog retained arXiv bibliography and exact-version primary full text","body_bytes":27830565,"body_sha256":"d9f0c31bdae82ae2be8ce1921ba5b23b3898cb61f14ad7755ff8290e7d877a6c","citation_key":"arxiv_2412_04454","final_url":"https://arxiv.org/pdf/2412.04454v2","first_submitted_date":"2024-12-05","http_status":200,"media_type":"application/pdf","primary_url":"https://arxiv.org/pdf/2412.04454v2","read_evidence":{"boundary":"Reasoning annotations are generated with known grounded actions and previous instructions; they are supervisory constructions, not measurements of a human causal thought process. Offline steps and online task rollouts remain different.","entire_paper_read":false,"evidence_level":"fulltext_sections","mechanism":"Grounding then planning/reasoning training uses screenshot observations, pyautogui plus platform plugins, template action augmentation and GPT-4o-generated inner monologue.","pages":[1,3,4,5,6],"read_at":"2026-10-18T10:11:21.810734+00:00","sections":["2.2 Agent Framework (continued)","2.3 AGUVIS DATA COLLECTION","2.4 Model Architecture","2.5 Training Paradigm","3.2 Offline GUI Agent Evaluation","3.3 Online GUI Agent Evaluation","4.2 Role of Inner Monologue"]},"resource_id":"arxiv:2412.04454","retained_abstract_sha256":"89dbdf874ace87db12fa321e67ecf818d3964622b045ea7d95e109101820d407","retrieved_at":"2026-10-18T10:09:02.407376+00:00","title":"Aguvis: Unified Pure Vision Agents for Autonomous GUI Interaction","version_binding":{"embedded_arxiv_version_marker_found":true,"limitation":null,"method":"Exact official version URL; retained bibliography; embedded marker when available.","requested_exact_version":"2412.04454v2"},"version_read":2,"version_updated_date":"2025-05-05","versioned_abstract_url":"https://arxiv.org/abs/2412.04454v2"},{"authors":["Yuhao Yang","Yue Wang","Dongxu Li","Ziyang Luo","Bei Chen","Chao Huang","Junnan Li"],"bibliography_basis":"frozen catalog retained arXiv bibliography and exact-version primary full text","body_bytes":3843901,"body_sha256":"169bc7694b456530ce8a6f0336b7efc33a32a21a731e1656caf4934e83f59334","citation_key":"arxiv_2412_16256","final_url":"https://arxiv.org/pdf/2412.16256v2","first_submitted_date":"2024-12-20","http_status":200,"media_type":"application/pdf","primary_url":"https://arxiv.org/pdf/2412.16256v2","read_evidence":{"boundary":"Grounder depends on an external planner and is not trained to correct planner-generated instruction errors.","entire_paper_read":false,"evidence_level":"fulltext_sections","mechanism":"Two-stage element caption/instruction generation and context-conditioned grounding data; text and screenshot-action histories are separate training inputs.","pages":[1,3,4,9],"read_at":"2026-10-18T10:12:33.823923+00:00","sections":["Front matter and abstract","2.1 Large-scale Diverse GUI Data Synthesizing","2.1.2 Context-aware Data Extension from Trajectories","6 Limitations"]},"resource_id":"arxiv:2412.16256","retained_abstract_sha256":"9b49d45aa539c224846a7e14cee9b537d032228e19e86a9cb3cdd9f5004aa429","retrieved_at":"2026-10-18T10:09:49.808316+00:00","title":"Aria-UI: Visual Grounding for GUI Instructions","version_binding":{"embedded_arxiv_version_marker_found":true,"limitation":null,"method":"Exact official version URL; retained bibliography; embedded marker when available.","requested_exact_version":"2412.16256v2"},"version_read":2,"version_updated_date":"2025-07-08","versioned_abstract_url":"https://arxiv.org/abs/2412.16256v2"},{"authors":["Yujia Qin","Yining Ye","Junjie Fang","Haoming Wang","Shihao Liang","Shizuo Tian","Junda Zhang","Jiahao Li","Yunxin Li","Shijue Huang","Wanjun Zhong","Kuanye Li","Jiale Yang","Yu Miao","Woyu Lin","Longxiang Liu","Xu Jiang","Qianli Ma","Jingyu Li","Xiaojun Xiao","Kai Cai","Chuang Li","Yaowei Zheng","Chaolin Jin","Chen Li","Xiao Zhou","Minchao Wang","Haoli Chen","Zhaojian Li","Haihua Yang","Haifeng Liu","Feng Lin","Tao Peng","Xin Liu","Guang Shi"],"bibliography_basis":"frozen catalog retained arXiv bibliography and exact-version primary full text","body_bytes":34825215,"body_sha256":"999d01bdd86eb39df1c322d7ab2a48f9f10d8f8a6cb2af913591612712b3964e","citation_key":"arxiv_2501_12326","final_url":"https://arxiv.org/pdf/2501.12326v1","first_submitted_date":"2025-01-21","http_status":200,"media_type":"application/pdf","primary_url":"https://arxiv.org/pdf/2501.12326v1","read_evidence":{"boundary":"Thought generation with known actions can be post-hoc; thought bootstrapping selects correct-action samples but does not itself prove causal faithfulness. Corrected post-error states are distinct from error-free demonstrations; SFT and DPO checkpoint settings differ.","entire_paper_read":false,"evidence_level":"fulltext_sections","mechanism":"Native screenshot policy predicts thoughts and actions; perception, action traces, tutorial and reasoning augmentation feed training; online trace filtering, correction trajectories and DPO exploit errors.","pages":[1,2,13,14,15,16,17,18,19,20,21,22],"read_at":"2026-10-18T10:12:33.824072+00:00","sections":["Front matter and abstract","4.1 Architecture Overview","4.2 Enhancing GUI Perception","4.3 Unified Action Modeling and Grounding","4.4 Infusing System-2 Reasoning","4.5 Learning from Prior Experience in Long-term Memory","4.6 Training"]},"resource_id":"arxiv:2501.12326","retained_abstract_sha256":"d19aa758e66411a31dd1fea6f07c83d1c828bd4aad86f1564a9d7f3670c3f79f","retrieved_at":"2026-10-18T10:09:55.790354+00:00","title":"UI-TARS: Pioneering Automated GUI Interaction with Native Agents","version_binding":{"embedded_arxiv_version_marker_found":true,"limitation":null,"method":"Exact official version URL; retained bibliography; embedded marker when available.","requested_exact_version":"2501.12326v1"},"version_read":1,"version_updated_date":"2025-01-21","versioned_abstract_url":"https://arxiv.org/abs/2501.12326v1"},{"authors":["Zora Zhiruo Wang","Jiayuan Mao","Daniel Fried","Graham Neubig"],"bibliography_basis":"frozen catalog retained arXiv bibliography and exact-version primary full text","body_bytes":941281,"body_sha256":"ddc6f6fedec48c88ae743b43a464f7eb4aecaab05acd18c29cfb657ab445ebe4","citation_key":"arxiv_2409_07429","final_url":"https://arxiv.org/pdf/2409.07429v1","first_submitted_date":"2024-09-11","http_status":200,"media_type":"application/pdf","primary_url":"https://arxiv.org/pdf/2409.07429v1","read_evidence":{"boundary":"Online workflow admission uses LM success judgment; memory grows across test queries and is grouped by website. It is not a weight-training method or a memory-free evaluation.","entire_paper_read":false,"evidence_level":"fulltext_sections","mechanism":"Induces reusable subroutines with variable placeholders from trajectories and supplies them as memory; offline induction and online test-stream updating differ.","pages":[1,3,4,5],"read_at":"2026-10-18T10:12:33.824073+00:00","sections":["2.2 Workflow Representation","2.3 Inducing and Using Workflows","3.1 WebArena"]},"resource_id":"arxiv:2409.07429","retained_abstract_sha256":"d4c142be3a9c37bbb0d4f401a02de25bd25b4c3968f42ad9a61513abff2b5460","retrieved_at":"2026-10-18T10:10:33.155315+00:00","title":"Agent Workflow Memory","version_binding":{"embedded_arxiv_version_marker_found":true,"limitation":null,"method":"Exact official version URL; retained bibliography; embedded marker when available.","requested_exact_version":"2409.07429v1"},"version_read":1,"version_updated_date":"2024-09-11","versioned_abstract_url":"https://arxiv.org/abs/2409.07429v1"},{"authors":["Saaket Agashe","Kyle Wong","Vincent Tu","Jiachen Yang","Ang Li","Xin Eric Wang"],"bibliography_basis":"frozen catalog retained arXiv bibliography and exact-version primary full text","body_bytes":9525681,"body_sha256":"b87310f7b40407c9413d99bbd6cd027bac381cae77ef4d116dc3070012aa3ae5","citation_key":"arxiv_2504_00906","final_url":"https://arxiv.org/pdf/2504.00906v1","first_submitted_date":"2025-04-01","http_status":200,"media_type":"application/pdf","primary_url":"https://arxiv.org/pdf/2504.00906v1","read_evidence":{"boundary":"Structural specialist programmatically updates cells, so the complete system is not restricted to screenshot-based pointer actions; mixed backends and inherited knowledge are part of its conditions.","entire_paper_read":false,"evidence_level":"fulltext_sections","mechanism":"Manager replans after every subgoal; worker routes actions to visual, OCR/text-span, or structural spreadsheet specialists.","pages":[1,3,4,5],"read_at":"2026-10-18T10:12:33.824075+00:00","sections":["3 Agent S2","3.1 Mixture of Grounding","3.2 Proactive Hierarchical Planning"]},"resource_id":"arxiv:2504.00906","retained_abstract_sha256":"11958f69503db514a91af8721870a4cbc1fa19dabf94e3f4ce4118d2fad9862a","retrieved_at":"2026-10-18T10:10:44.157220+00:00","title":"Agent S2: A Compositional Generalist-Specialist Framework for Computer Use Agents","version_binding":{"embedded_arxiv_version_marker_found":true,"limitation":null,"method":"Exact official version URL; retained bibliography; embedded marker when available.","requested_exact_version":"2504.00906v1"},"version_read":1,"version_updated_date":"2025-04-01","versioned_abstract_url":"https://arxiv.org/abs/2504.00906v1"},{"authors":["Chaoyun Zhang","He Huang","Chiming Ni","Jian Mu","Si Qin","Shilin He","Lu Wang","Fangkai Yang","Pu Zhao","Chao Du","Liqun Li","Yu Kang","Zhao Jiang","Suzhen Zheng","Rujia Wang","Jiaxu Qian","Minghua Ma","Jian-Guang Lou","Qingwei Lin","Saravan Rajmohan","Dongmei Zhang"],"bibliography_basis":"frozen catalog retained arXiv bibliography and exact-version primary full text","body_bytes":4405780,"body_sha256":"ceecfb496791b8b101275520066eb69c3289c7c727a9e69cc844219598acece5","citation_key":"arxiv_2504_14603","final_url":"https://arxiv.org/pdf/2504.14603v2","first_submitted_date":"2025-04-20","http_status":200,"media_type":"application/pdf","primary_url":"https://arxiv.org/pdf/2504.14603v2","read_evidence":{"boundary":"Application-specific bindings and OS integration are dependencies, not evidence of unchanged cross-platform portability. PDF contains obvious ACM template placeholders; do not treat those as publication metadata.","entire_paper_read":false,"evidence_level":"fulltext_sections","mechanism":"Windows host/app finite-state controllers share a blackboard, fuse UIA and visual controls, and prefer registered native APIs with GUI fallback.","pages":[1,3,4,5,6,7,8],"read_at":"2026-10-18T10:12:33.824075+00:00","sections":["3.1 UFO2 as a System Substrate","3.2 HostAgent","3.3 AppAgent","3.4 Hybrid Control Detection","3.5 Unified GUI–API Action Orchestrator"]},"resource_id":"arxiv:2504.14603","retained_abstract_sha256":"dc47a80b48a6a5f126e36fa14639d56d8c8025a453fbcee2cbdb73eb67945561","retrieved_at":"2026-10-18T10:11:01.554138+00:00","title":"UFO2: The Desktop AgentOS","version_binding":{"embedded_arxiv_version_marker_found":true,"limitation":null,"method":"Exact official version URL; retained bibliography; embedded marker when available.","requested_exact_version":"2504.14603v2"},"version_read":2,"version_updated_date":"2025-04-25","versioned_abstract_url":"https://arxiv.org/abs/2504.14603v2"},{"authors":["Xinyuan Wang","Bowen Wang","Dunjie Lu","Junlin Yang","Tianbao Xie","Junli Wang","Jiaqi Deng","Xiaole Guo","Yiheng Xu","Chen Henry Wu","Zhennan Shen","Zhuokai Li","Ryan Li","Xiaochuan Li","Junda Chen","Boyuan Zheng","Peihang Li","Fangyu Lei","Ruisheng Cao","Yeqiao Fu","Dongchan Shin","Martin Shin","Jiarui Hu","Yuyan Wang","Jixuan Chen","Yuxiao Ye","Danyang Zhang","Dikang Du","Hao Hu","Huarong Chen","Zaida Zhou","Haotian Yao","Ziwei Chen","Qizheng Gu","Yipu Wang","Heng Wang","Diyi Yang","Victor Zhong","Flood Sung","Y. Charles","Zhilin Yang","Tao Yu"],"bibliography_basis":"frozen catalog retained arXiv bibliography and exact-version primary full text","body_bytes":21581239,"body_sha256":"fd1c1e7b4d467acf63fc743c3925c7ceb015e5b1625f62c22295c0b9e2d82a70","citation_key":"arxiv_2508_09123","final_url":"https://arxiv.org/pdf/2508.09123v3","first_submitted_date":"2025-08-12","http_status":200,"media_type":"application/pdf","primary_url":"https://arxiv.org/pdf/2508.09123v3","read_evidence":{"boundary":"Collected modalities include accessibility trees although inference uses screenshots. Released versus used training trajectory totals differ, so no conflation of data sizes. Pass@n is not single-rollout performance.","entire_paper_read":false,"evidence_level":"fulltext_sections","mechanism":"Natural human screen recordings and device signals are reduced into state-action pairs; pre-movement frames reduce cursor leakage; reflector/generator/summarizer construct reasoning and mask incorrect steps in SFT.","pages":[1,3,4,5,6,7],"read_at":"2026-10-18T10:13:32.125470+00:00","sections":["2 AgentNet Collection","2.2 AgentNet Tool","3.1 Synthesizing Reflective Long CoT Reasoning","3.2 Context Encoding and Test-Time Reasoning","3.3 Training Data Mixtures"]},"resource_id":"arxiv:2508.09123","retained_abstract_sha256":"40e46f27128d066358fcee6279b26bb2b708dacaf1f7b5794b755520fe892243","retrieved_at":"2026-10-18T10:11:08.501504+00:00","title":"OpenCUA: Open Foundations for Computer-Use Agents","version_binding":{"embedded_arxiv_version_marker_found":true,"limitation":null,"method":"Exact official version URL; retained bibliography; embedded marker when available.","requested_exact_version":"2508.09123v3"},"version_read":3,"version_updated_date":"2025-10-04","versioned_abstract_url":"https://arxiv.org/abs/2508.09123v3"},{"authors":["Yiheng Xu","Dunjie Lu","Zhennan Shen","Junli Wang","Zekun Wang","Yuchen Mao","Caiming Xiong","Tao Yu"],"bibliography_basis":"frozen catalog retained arXiv bibliography and exact-version primary full text","body_bytes":4439073,"body_sha256":"7bdb3828c245ef1ad1cf60ead2140e1bd5f6ff01f8a189f50914cb07f49ef603","citation_key":"arxiv_2412_09605","final_url":"https://arxiv.org/pdf/2412.09605v2","first_submitted_date":"2024-12-12","http_status":200,"media_type":"application/pdf","primary_url":"https://arxiv.org/pdf/2412.09605v2","read_evidence":{"boundary":"Replay uses screenshot and accessibility-tree observations and Playwright actions; VLM filtering is not an infallible task-success oracle. Tutorial coverage and replay feasibility bound the produced distribution.","entire_paper_read":false,"evidence_level":"fulltext_sections","mechanism":"Filters tutorial corpora, standardizes task prerequisites/steps/outcomes, and replays with a BrowserGym agent before VLM trajectory evaluation.","pages":[1,4,5,6],"read_at":"2026-10-18T10:13:32.126388+00:00","sections":["2.1 Tutorial Collection and Standardization","2.2.1 Trajectory Data Schema","2.2.2 Guided Replay with Tutorials","2.2.3 Evaluation of Trajectory"]},"resource_id":"arxiv:2412.09605","retained_abstract_sha256":"8ee630d434cef63f63c368752aeae14b0a60d49c3fde33b3a79ecbfc8fc4d06e","retrieved_at":"2026-10-18T10:11:32.110232+00:00","title":"AgentTrek: Agent Trajectory Synthesis via Guiding Replay with Web Tutorials","version_binding":{"embedded_arxiv_version_marker_found":true,"limitation":null,"method":"Exact official version URL; retained bibliography; embedded marker when available.","requested_exact_version":"2412.09605v2"},"version_read":2,"version_updated_date":"2025-03-03","versioned_abstract_url":"https://arxiv.org/abs/2412.09605v2"},{"authors":["Vardaan Pahuja","Yadong Lu","Corby Rosset","Boyu Gou","Arindam Mitra","Spencer Whitehead","Yu Su","Ahmed Awadallah"],"bibliography_basis":"frozen catalog retained arXiv bibliography and exact-version primary full text","body_bytes":2133180,"body_sha256":"038a14d81f9587ac869667d77f93ce33eac65d18e8037402680c091204e8cac8","citation_key":"arxiv_2502_11357","final_url":"https://arxiv.org/pdf/2502.11357v4","first_submitted_date":"2025-02-17","http_status":200,"media_type":"application/pdf","primary_url":"https://arxiv.org/pdf/2502.11357v4","read_evidence":{"boundary":"Verifier sees screenshots plus final-page markdown; login/payment/CAPTCHA boundaries halt collection. Retrospectively aligned intent is not identical to fulfilling a fixed user goal.","entire_paper_read":false,"evidence_level":"fulltext_sections","mechanism":"Task proposer and refiner co-evolve task intent during actual web navigation; summarizer and verifier retain coherent trajectories.","pages":[1,3,4],"read_at":"2026-10-18T10:13:32.126390+00:00","sections":["3 Data Recipe","3.1 Website Selection","3.2 Data Generation Pipeline"]},"resource_id":"arxiv:2502.11357","retained_abstract_sha256":"d9f459a02b7a464d9e1e49579943d1b93d46308c1b00e795c903d2fa1502a2b2","retrieved_at":"2026-10-18T10:11:37.955753+00:00","title":"Explorer: Scaling Exploration-driven Web Trajectory Synthesis for Multimodal Web Agents","version_binding":{"embedded_arxiv_version_marker_found":true,"limitation":null,"method":"Exact official version URL; retained bibliography; embedded marker when available.","requested_exact_version":"2502.11357v4"},"version_read":4,"version_updated_date":"2025-05-30","versioned_abstract_url":"https://arxiv.org/abs/2502.11357v4"},{"authors":["Qiushi Sun","Kanzhi Cheng","Zichen Ding","Chuanyang Jin","Yian Wang","Fangzhi Xu","Zhenyu Wu","Chengyou Jia","Liheng Chen","Zhoumianze Liu","Ben Kao","Guohao Li","Junxian He","Yu Qiao","Zhiyong Wu"],"bibliography_basis":"frozen catalog retained arXiv bibliography and exact-version primary full text","body_bytes":3202773,"body_sha256":"65bb449a7e9d433f29b088e0ed5d52f22833572b7a67533b48cf07e88f14292d","citation_key":"arxiv_2412_19723","final_url":"https://arxiv.org/pdf/2412.19723v3","first_submitted_date":"2024-12-27","http_status":200,"media_type":"application/pdf","primary_url":"https://arxiv.org/pdf/2412.19723v3","read_evidence":{"boundary":"Exploration is model-free except input-field content; TRM uses GPT-4o and last three states plus low-level instructions. Reward-weighted SFT sampling is not online policy-gradient RL.","entire_paper_read":false,"evidence_level":"fulltext_sections","mechanism":"Rule-based interaction exploration discovers transitions, model annotations turn them into low/high-level tasks, then tasks are rolled out; graded trajectory sampling retains some incomplete but useful traces.","pages":[1,3,4,5],"read_at":"2026-10-18T10:13:32.126392+00:00","sections":["3.1 Interaction-Driven Functional Discovery","3.2 Reverse Task Synthesis","3.3 Trajectory Reward Model","4.1 Experimental Settings"]},"resource_id":"arxiv:2412.19723","retained_abstract_sha256":"40b5e3090c25f80d8d786f623812459b329acc2772f945419a585ffb27c54db3","retrieved_at":"2026-10-18T10:11:42.264735+00:00","title":"OS-Genesis: Automating GUI Agent Trajectory Construction via Reverse Task Synthesis","version_binding":{"embedded_arxiv_version_marker_found":true,"limitation":null,"method":"Exact official version URL; retained bibliography; embedded marker when available.","requested_exact_version":"2412.19723v3"},"version_read":3,"version_updated_date":"2025-06-27","versioned_abstract_url":"https://arxiv.org/abs/2412.19723v3"},{"authors":["Zehan Qi","Xiao Liu","Iat Long Iong","Hanyu Lai","Xueqiao Sun","Wenyi Zhao","Yu Yang","Xinyue Yang","Jiadai Sun","Shuntian Yao","Tianjie Zhang","Wei Xu","Jie Tang","Yuxiao Dong"],"bibliography_basis":"frozen catalog retained arXiv bibliography and exact-version primary full text","body_bytes":10184269,"body_sha256":"79a93b02cca5586d9e3e2742c5af53f683c632dc0eaed3b7bf87f53daaa2d462","citation_key":"arxiv_2411_02337","final_url":"https://arxiv.org/pdf/2411.02337v3","first_submitted_date":"2024-11-04","http_status":200,"media_type":"application/pdf","primary_url":"https://arxiv.org/pdf/2411.02337v3","read_evidence":{"boundary":"Outcome model uses final HTML and action history, so reward correctness is a substantive assumption. Its HTML-based WebArena-Lite conditions differ from screenshot-only or unrestricted open-web settings.","entire_paper_read":false,"evidence_level":"fulltext_sections","mechanism":"HTML/history web policy learns from curriculum tasks, an outcome reward model, KL-constrained updates and replay.","pages":[1,3,4,5],"read_at":"2026-10-18T10:13:32.126394+00:00","sections":["2 Self-Evolving Online Curriculum RL","2.1 Self-evolving New Instruction","2.2 Reinforcement Learning for LLMs in Online Web Environments"]},"resource_id":"arxiv:2411.02337","retained_abstract_sha256":"db039a2592cd79a1f8f7f1c985c2ddf42447c63d1f58d2815ba1d6ef2369da47","retrieved_at":"2026-10-18T10:11:47.855035+00:00","title":"WebRL: Training LLM Web Agents via Self-Evolving Online Curriculum Reinforcement Learning","version_binding":{"embedded_arxiv_version_marker_found":true,"limitation":null,"method":"Exact official version URL; retained bibliography; embedded marker when available.","requested_exact_version":"2411.02337v3"},"version_read":3,"version_updated_date":"2025-01-27","versioned_abstract_url":"https://arxiv.org/abs/2411.02337v3"},{"authors":["Run Luo","Lu Wang","Wanwei He","Longze Chen","Jiaming Li","Xiaobo Xia"],"bibliography_basis":"frozen catalog retained arXiv bibliography and exact-version primary full text","body_bytes":722727,"body_sha256":"6e9d838b0c87ac3e21e266748924be44869abe8f1e818b7e1bb066f746d9bc9f","citation_key":"arxiv_2504_10458","final_url":"https://arxiv.org/pdf/2504.10458v4","first_submitted_date":"2025-04-14","http_status":200,"media_type":"application/pdf","primary_url":"https://arxiv.org/pdf/2504.10458v4","read_evidence":{"boundary":"This is verifiable per-example action reward, not demonstrated multi-turn environment-success reinforcement learning; small selected training set does not count upstream base-model or candidate data cost.","entire_paper_read":false,"evidence_level":"fulltext_sections","mechanism":"GRPO compares candidate responses using annotated action-type, point-in-box, text-F1 and format rewards on curated GUI examples.","pages":[1,3,4,5],"read_at":"2026-10-18T10:13:32.126395+00:00","sections":["3 GUI-R1 Framework","3.1 Preliminaries","3.2 Verifiable Rewards in Unified Action Space","3.3 Training Data Curation"]},"resource_id":"arxiv:2504.10458","retained_abstract_sha256":"b875617b795e0f8d7d6348e22ec85ccc9ab4dd931e66850182348927b1e5209f","retrieved_at":"2026-10-18T10:11:59.935447+00:00","title":"GUI-R1 : A Generalist R1-Style Vision-Language Action Model For GUI Agents","version_binding":{"embedded_arxiv_version_marker_found":true,"limitation":null,"method":"Exact official version URL; retained bibliography; embedded marker when available.","requested_exact_version":"2504.10458v4"},"version_read":4,"version_updated_date":"2025-10-01","versioned_abstract_url":"https://arxiv.org/abs/2504.10458v4"},{"authors":["Haoming Wang","Haoyang Zou","Huatong Song","Jiazhan Feng","Junjie Fang","Junting Lu","Longxiang Liu","Qinyu Luo","Shihao Liang","Shijue Huang","Wanjun Zhong","Yining Ye","Yujia Qin","Yuwen Xiong","Yuxin Song","Zhiyong Wu","Aoyan Li","Bo Li","Chen Dun","Chong Liu","Daoguang Zan","Fuxing Leng","Hanbin Wang","Hao Yu","Haobin Chen","Hongyi Guo","Jing Su","Jingjia Huang","Kai Shen","Kaiyu Shi","Lin Yan","Peiyao Zhao","Pengfei Liu","Qinghao Ye","Renjie Zheng","Shulin Xin","Wayne Xin Zhao","Wen Heng","Wenhao Huang","Wenqian Wang","Xiaobo Qin","Yi Lin","Youbin Wu","Zehui Chen","Zihao Wang","Baoquan Zhong","Xinchun Zhang","Xujing Li","Yuanfan Li","Zhongkai Zhao","Chengquan Jiang","Faming Wu","Haotian Zhou","Jinlin Pang","Li Han","Qi Liu","Qianli Ma","Siyao Liu","Songhua Cai","Wenqi Fu","Xin Liu","Yaohui Wang","Zhi Zhang","Bo Zhou","Guoliang Li","Jiajun Shi","Jiale Yang","Jie Tang","Li Li","Qihua Han","Taoran Lu","Woyu Lin","Xiaokang Tong","Xinyao Li","Yichi Zhang","Yu Miao","Zhengxuan Jiang","Zili Li","Ziyuan Zhao","Chenxin Li","Dehua Ma","Feng Lin","Ge Zhang","Haihua Yang","Hangyu Guo","Hongda Zhu","Jiaheng Liu","Junda Du","Kai Cai","Kuanye Li","Lichen Yuan","Meilan Han","Minchao Wang","Shuyue Guo","Tianhao Cheng","Xiaobo Ma","Xiaojun Xiao","Xiaolong Huang","Xinjie Chen","Yidi Du","Yilin Chen","Yiwen Wang","Zhaojian Li","Zhenzhu Yang","Zhiyuan Zeng","Chaolin Jin","Chen Li","Hao Chen","Haoli Chen","Jian Chen","Qinghao Zhao","Guang Shi"],"bibliography_basis":"frozen catalog retained arXiv bibliography and exact-version primary full text","body_bytes":17410516,"body_sha256":"6a2df5206955e0eab044e1bc4e0ed9239439e0377f79d179044da41a0bd5c603","citation_key":"arxiv_2509_02544","final_url":"https://arxiv.org/pdf/2509.02544v2","first_submitted_date":"2025-09-02","http_status":200,"media_type":"application/pdf","primary_url":"https://arxiv.org/pdf/2509.02544v2","read_evidence":{"boundary":"Function-based game rewards, answer judging and generative GUI-General outcome rewards are distinct; hybrid tools and human annotation are substantive conditions, not a single pure-visual or fully autonomous recipe.","entire_paper_read":false,"evidence_level":"fulltext_sections","mechanism":"GUI-centered model combines screenshots, GUI actions and SDK/terminal tools; interactive correction and CT/SFT/RL data routing precede PPO with stateful asynchronous rollouts and domain-model merging.","pages":[1,2,4,5,7,9,10,11,12,13],"read_at":"2026-10-18T10:14:45.616362+00:00","sections":["2.1 Formulation","2.2 All-in-One GUI Sandbox","2.3 Data Flywheel","2.4.2 Interactive Annotation","2.5 Multi-turn Reinforcement Learning","2.6 Parameter Interpolation"]},"resource_id":"arxiv:2509.02544","retained_abstract_sha256":"800e417f4a2f720beb436ddd2e7bc60f0dd2483b44ac007a72a5eab2c6001ff8","retrieved_at":"2026-10-18T10:12:03.350197+00:00","title":"UI-TARS-2 Technical Report: Advancing GUI Agent with Multi-Turn Reinforcement Learning","version_binding":{"embedded_arxiv_version_marker_found":true,"limitation":null,"method":"Exact official version URL; retained bibliography; embedded marker when available.","requested_exact_version":"2509.02544v2"},"version_read":2,"version_updated_date":"2025-09-05","versioned_abstract_url":"https://arxiv.org/abs/2509.02544v2"},{"authors":["Junhong Shen","Hao Bai","Lunjun Zhang","Yifei Zhou","Amrith Setlur","Shengbang Tong","Diego Caples","Nan Jiang","Tong Zhang","Ameet Talwalkar","Aviral Kumar"],"bibliography_basis":"frozen catalog retained arXiv bibliography and exact-version primary full text","body_bytes":23185281,"body_sha256":"be76d2989976e3d59e20189821c3dfccc7b6008e2e29f1a2741b567fd5b01c65","citation_key":"arxiv_2506_07976","final_url":"https://arxiv.org/pdf/2506.07976v2","first_submitted_date":"2025-06-09","http_status":200,"media_type":"application/pdf","primary_url":"https://arxiv.org/pdf/2506.07976v2","read_evidence":{"boundary":"Observations include accessibility tree and marked screenshots. Prompted rechecking can turn correct answers wrong; longer horizons have credit-assignment costs. No general claim that more steps always helps.","entire_paper_read":false,"evidence_level":"fulltext_sections","mechanism":"Extends interaction budget to gather information, recheck and recover; trains a gradually lengthened horizon curriculum using filtered successful-rollout behavior cloning and recency-weighted replay.","pages":[1,3,4,5,7,8],"read_at":"2026-10-18T10:14:45.616504+00:00","sections":["3 Problem Setup","4.1 Scaling Interaction by Acting Longer","4.2 Interaction vs Per-Step Compute","5.1 Fixed Long Horizons","5.2 Curriculum Over Interaction Horizon"]},"resource_id":"arxiv:2506.07976","retained_abstract_sha256":"e4b6c8b297e88dae689ba2a48724a441698cd5309a3bbe1198f38343e88aceb1","retrieved_at":"2026-10-18T10:12:22.820764+00:00","title":"Thinking vs. Doing: Agents that Reason by Scaling Test-Time Interaction","version_binding":{"embedded_arxiv_version_marker_found":true,"limitation":null,"method":"Exact official version URL; retained bibliography; embedded marker when available.","requested_exact_version":"2506.07976v2"},"version_read":2,"version_updated_date":"2025-06-10","versioned_abstract_url":"https://arxiv.org/abs/2506.07976v2"},{"authors":["Hongbin Zhong","Fazle Faisal","Luis França","Tanakorn Leesatapornwongsa","Adriana Szekeres","Kexin Rong","Suman Nath"],"bibliography_basis":"frozen catalog retained arXiv bibliography and exact-version primary full text","body_bytes":2460740,"body_sha256":"ec8339a07ee3c83d4f416a27cf9e8bf254efa26968b7bc9fdf3e07ff09e38ab6","citation_key":"arxiv_2602_20502","final_url":"https://arxiv.org/pdf/2602.20502v2","first_submitted_date":"2026-02-24","http_status":200,"media_type":"application/pdf","primary_url":"https://arxiv.org/pdf/2602.20502v2","read_evidence":{"boundary":"Application-specific exploration, locators and coverage are upfront dependencies. Single sketch-planning call does not mean zero auxiliary calls: state filtering and optional semantic helpers exist.","entire_paper_read":false,"evidence_level":"fulltext_sections","mechanism":"Offline crawl constructs interface-template and operation graph; planner generates Python sketch, linker inserts paths and compiler resolves Playwright actions; failures localize for repair.","pages":[1,3,4,5,6],"read_at":"2026-10-18T10:14:45.616506+00:00","sections":["3 State-Machine Memory","3.1 Application State Representation","3.2 Automated SMG Construction","4.1 Planner","4.2 Executor","4.3 Patcher (opening)"]},"resource_id":"arxiv:2602.20502","retained_abstract_sha256":"e179c833d43a88098d576e31e726eb3e4289f728659697364d0b448282df3fc1","retrieved_at":"2026-10-18T10:12:47.197774+00:00","title":"ActionEngine: From Reactive to Programmatic Web Agents via State Machine Memory","version_binding":{"embedded_arxiv_version_marker_found":true,"limitation":null,"method":"Exact official version URL; retained bibliography; embedded marker when available.","requested_exact_version":"2602.20502v2"},"version_read":2,"version_updated_date":"2026-09-28","versioned_abstract_url":"https://arxiv.org/abs/2602.20502v2"},{"authors":["Yutong Huang","Vikranth Srivatsa","Alex Asch","Hansin Tushar Patwa","Yiying Zhang"],"bibliography_basis":"frozen catalog retained arXiv bibliography and exact-version primary full text","body_bytes":2535276,"body_sha256":"cd6d543feb3ec804b35f91ae9dd897a563038f9230067abd5eaafd88976e46af","citation_key":"arxiv_2605_17320","final_url":"https://arxiv.org/pdf/2605.17320v1","first_submitted_date":"2026-05-17","http_status":200,"media_type":"application/pdf","primary_url":"https://arxiv.org/pdf/2605.17320v1","read_evidence":{"boundary":"Linux/container-local recoverability is not remote-service rollback: external connections are closed and re-established, with network restrictions or approval needed for external side effects. No general safety guarantee or numeric speed comparison repeated.","entire_paper_read":false,"evidence_level":"fulltext_sections","mechanism":"Forkable GUI workspace uses sibling containers, copy-on-write state and asynchronous durable checkpointing; supports branch, rollback and selected commit.","pages":[1,4,5,8],"read_at":"2026-10-18T10:14:45.616507+00:00","sections":["3 TClone Overview","4 TClone Design","4.4 Network Subsystem","4.5 GUI Subsystem"]},"resource_id":"arxiv:2605.17320","retained_abstract_sha256":"51576fb9bde23f769d3200a0d593ad610eb27248917132aff1bcf6eb7ff03cea","retrieved_at":"2026-10-18T10:12:53.041131+00:00","title":"TClone: Low-Latency Forking of Live GUI Environments for Computer-Use Agents","version_binding":{"embedded_arxiv_version_marker_found":true,"limitation":null,"method":"Exact official version URL; retained bibliography; embedded marker when available.","requested_exact_version":"2605.17320v1"},"version_read":1,"version_updated_date":"2026-05-17","versioned_abstract_url":"https://arxiv.org/abs/2605.17320v1"},{"authors":["Hao Bai","Rui Yang","Chenlu Ye","Spencer Whitehead","Aviral Kumar","Tong Zhang"],"bibliography_basis":"frozen catalog retained arXiv bibliography and exact-version primary full text","body_bytes":1122936,"body_sha256":"423c18c48b337d91ecd7fe548cd67a910ccf4cc373a4b400674e9f0b2bc4cba4","citation_key":"arxiv_2606_05597","final_url":"https://arxiv.org/pdf/2606.05597v3","first_submitted_date":"2026-06-04","http_status":200,"media_type":"application/pdf","primary_url":"https://arxiv.org/pdf/2606.05597v3","read_evidence":{"boundary":"Reported single-seed WebGym OOD experiments use GPT-4o outcome rubric and additive-memory prompts. The diagnosis is reported in that setup, not a universal causal law. Exact v3 title matches frozen catalog despite search snippet variation.","entire_paper_read":false,"evidence_level":"fulltext_sections","mechanism":"Continuous rollout workers and lightweight screenshot references overlap training/rollout; decoupled importance correction handles stale policies, and constant trajectory weighting addresses long-failure underweighting.","pages":[1,3,4,7],"read_at":"2026-10-18T10:14:45.616508+00:00","sections":["3.1 System","3.2 Algorithm","4.1 Setup","5.2 Ablations"]},"resource_id":"arxiv:2606.05597","retained_abstract_sha256":"8a8afe44ffa73739505070c5f8004d8b4bbe98455fc0f5f3cef4b450bf441c74","retrieved_at":"2026-10-18T10:12:58.265627+00:00","title":"AsyncWebRL: Efficient Asynchronous Reinforcement Learning for Multi-Step Visual Web Agents","version_binding":{"embedded_arxiv_version_marker_found":true,"limitation":null,"method":"Exact official version URL; retained bibliography; embedded marker when available.","requested_exact_version":"2606.05597v3"},"version_read":3,"version_updated_date":"2026-08-07","versioned_abstract_url":"https://arxiv.org/abs/2606.05597v3"},{"authors":["Ziyun Zeng","Hang Hua","Bocheng Zou","Mu Cai","Rogerio Feris","Jiebo Luo"],"bibliography_basis":"frozen catalog retained arXiv bibliography and exact-version primary full text","body_bytes":2054732,"body_sha256":"bf6e672269ac55c35870ab2b93d652b472c30fd6ad9c98797b00127d58132305","citation_key":"arxiv_2605_18652","final_url":"https://arxiv.org/pdf/2605.18652v1","first_submitted_date":"2026-05-18","http_status":200,"media_type":"application/pdf","primary_url":"https://arxiv.org/pdf/2605.18652v1","read_evidence":{"boundary":"Main evaluations are offline reference-action and model-judged metrics; memory-benchmark episodic accumulation uses a separate set. No claim of proven live task completion.","entire_paper_read":false,"evidence_level":"fulltext_sections","mechanism":"Four LoRA memory operators learn event selection, compression, episodic writing and retrieval; selected ROI crops and text augment a frozen action backbone.","pages":[1,3,4,5,6,7],"read_at":"2026-10-18T10:14:45.616509+00:00","sections":["3 Data Curation","4.1 Framework","4.2 Memory System Design","4.3 Training MementoCore","4.4 Benchmarking","5.1 Experimental Setup"]},"resource_id":"arxiv:2605.18652","retained_abstract_sha256":"d96c8844274f79adea167169a0853e3a7cd82fee4a8a666913f97898a0836e7c","retrieved_at":"2026-10-18T10:13:01.881533+00:00","title":"MementoGUI: Learning Agentic Multimodal Memory Control for Long-Horizon GUI Agents","version_binding":{"embedded_arxiv_version_marker_found":true,"limitation":null,"method":"Exact official version URL; retained bibliography; embedded marker when available.","requested_exact_version":"2605.18652v1"},"version_read":1,"version_updated_date":"2026-05-18","versioned_abstract_url":"https://arxiv.org/abs/2605.18652v1"},{"authors":["Hanzhang Zhou","Panrong Tong","Xu Zhang","Quyu Kong","Chenglin Cai","Tianyu Xia","Gongjie Zhang","Jianan Zhang","Long Li","Long Chen","Lei Wang","Gaole Dai","Pengxiang Li","Liangyu Chen","Yue Wang","Steven Hoi"],"bibliography_basis":"frozen catalog retained arXiv bibliography and exact-version primary full text","body_bytes":32712659,"body_sha256":"705de6c54fa4eec0e5f81792cd323695e1d0274333886973cbad82d3c436fdb0","citation_key":"arxiv_2607_28227","final_url":"https://arxiv.org/pdf/2607.28227v1","first_submitted_date":"2026-07-30","http_status":200,"media_type":"application/pdf","primary_url":"https://arxiv.org/pdf/2607.28227v1","read_evidence":{"boundary":"Real-device evaluation uses AutoJudge rather than deterministic state checks; report explicitly retains human oversight in agent-driven capability development. Do not repeat frontier-ranking assertions or treat all action modes as GUI-only.","entire_paper_read":false,"evidence_level":"fulltext_sections","mechanism":"Combines screenshot/CLI/API observations, ordered action batches, device-health scheduling, domain-expert SFT merging, targeted action-error RL and end-to-end environment interaction.","pages":[1,2,3,4,6,9,10,13,14,15,44],"read_at":"2026-10-18T10:14:45.616510+00:00","sections":["2.1 System Overview and Action Space","2.2.2 Real-Device Runtime","2.2.3 Hybrid GUI+CLI","2.4.1 SFT","2.4.2 Action RL","2.4.3 Online RL","7 Limitations"]},"resource_id":"arxiv:2607.28227","retained_abstract_sha256":"52fb1b49501132f1dbe3564daa02cca22a93fc8d7c28dec8ee6070a23889574a","retrieved_at":"2026-10-18T10:13:06.605062+00:00","title":"Qwen-UI-Agent Technical Report: Toward Next-Generation Real-World Centric Foundation GUI Agents","version_binding":{"embedded_arxiv_version_marker_found":true,"limitation":null,"method":"Exact official version URL; retained bibliography; embedded marker when available.","requested_exact_version":"2607.28227v1"},"version_read":1,"version_updated_date":"2026-07-30","versioned_abstract_url":"https://arxiv.org/abs/2607.28227v1"},{"authors":["Daniel Beechey","Derek Yuen","Jianheng Liu","Dezhao Luo","Tiantian He","Weilin Luo","Jun Wang","Kun Shao"],"bibliography_basis":"fresh official Atom title-search record supplied by root and exact-version primary full text","body_bytes":2622535,"body_sha256":"e4711466886372760daa83384cb4d129d54fdcfe320be6e91fb7b4971d1a61de","citation_key":"arxiv_2606_20622","final_url":"https://arxiv.org/pdf/2606.20622v1","first_submitted_date":"2026-05-26","fresh_bibliography_evidence":"research/survey/evidence/title-search.json","fresh_bibliography_source_page":3,"http_status":200,"media_type":"application/pdf","primary_url":"https://arxiv.org/pdf/2606.20622v1","read_evidence":{"boundary":"Current implementation uses manually selected SPA-Bench tasks, an external LLM judge and heuristic history window. Autonomous curriculum/verifier/memory are roadmap stages, not demonstrated completed capabilities.","entire_paper_read":false,"evidence_level":"fulltext_sections","mechanism":"Cloud Android workers asynchronously feed complete trajectories to PPO; minimal cleanup and explicit setup/execution/teardown expose persistent state and distinguish infrastructure truncation.","pages":[1,3,4,5,6,7,8],"read_at":"2026-10-18T10:16:14.836671+00:00","sections":["2 Agent-Environment Interface and Policy Optimisation","3 Scalable Asynchronous Mobile Environment","3.2 Persistent State Management","3.3 Task Lifecycle","4.2 Rollout and Optimisation","4.3 Anatomy of the Self-Evolving Agent"]},"resource_id":"arxiv:2606.20622","retained_abstract_sha256":"9e761b20caa6882605ecfd6cd001aeda4a2d36b2e702eea9a6d70451fc0ea9eb","retrieved_at":"2026-10-18T10:14:00.181958+00:00","title":"Darwin Mobile Agent: A Roadmap for Self-Evolution","version_binding":{"embedded_arxiv_version_marker_found":true,"limitation":null,"method":"Exact official version URL; retained bibliography; embedded marker when available.","requested_exact_version":"2606.20622v1"},"version_read":1,"version_updated_date":"2026-05-26","versioned_abstract_url":"https://arxiv.org/abs/2606.20622v1"},{"authors":["Xiaoming Ren","Ru Zhen","Chao Li","Yang Song","Qiuxia Hou","Yanhao Zhang","Peng Liu","Qi Qi","Quanlong Zheng","Qi Wu","Zhenyi Liao","Binqiang Pan","Haobo Ji","Haonan Lu"],"bibliography_basis":"fresh official Atom title-search record supplied by root and exact-version primary full text","body_bytes":3843750,"body_sha256":"80375cd810dcda2e9ccca2c6fbee44730de8449972036c548bde312566b20673","citation_key":"arxiv_2605_05765","final_url":"https://arxiv.org/pdf/2605.05765v2","first_submitted_date":"2026-05-07","fresh_bibliography_evidence":"research/survey/evidence/title-search.json","fresh_bibliography_source_page":3,"http_status":200,"media_type":"application/pdf","primary_url":"https://arxiv.org/pdf/2605.05765v2","read_evidence":{"boundary":"Behavior cloning here means runtime skill capture, not shown policy-weight imitation training; literal trace replay is reserved for future development. Evaluation is demonstrations, not a broad controlled benchmark.","entire_paper_read":false,"evidence_level":"fulltext_sections","mechanism":"Android stack aligns screen/camera/speech, combines XML/OCR/vision grounding, and distills user interactions into reusable skill cards and activity/deeplink entry routes.","pages":[1,3,4,5,6,7],"read_at":"2026-10-18T10:16:14.837461+00:00","sections":["2 Framework","3 Omni Perception","4 Omni Memory","5.1 Omni Action in App Ecosystem","5.2 Trajectory Cloned Execution"]},"resource_id":"arxiv:2605.05765","retained_abstract_sha256":"2ece93c00c5556dfe50ddd0ba3f52b725f34d1bee18ea3565951bb78343d8ed0","retrieved_at":"2026-10-18T10:14:10.101459+00:00","title":"X-OmniClaw Technical Report: A Unified Mobile Agent for Multimodal Understanding and Interaction","version_binding":{"embedded_arxiv_version_marker_found":true,"limitation":null,"method":"Exact official version URL; retained bibliography; embedded marker when available.","requested_exact_version":"2605.05765v2"},"version_read":2,"version_updated_date":"2026-05-21","versioned_abstract_url":"https://arxiv.org/abs/2605.05765v2"},{"authors":["Sijia Li","Xiaoyu Tan","Shahir Ali","Niels Schmidt","Gengchen Ma","Xihe Qiu"],"bibliography_basis":"fresh official Atom title-search record supplied by root and exact-version primary full text","body_bytes":3795184,"body_sha256":"ef3bbc8adcc147239ea910e0c59904d7a26ee1610a7a4d1613be2cb845098536","citation_key":"arxiv_2601_19306","final_url":"https://arxiv.org/pdf/2601.19306v1","first_submitted_date":"2026-01-27","fresh_bibliography_evidence":"research/survey/evidence/title-search.json","fresh_bibliography_source_page":5,"http_status":200,"media_type":"application/pdf","primary_url":"https://arxiv.org/pdf/2601.19306v1","read_evidence":{"boundary":"The uncertainty signal is a model-based proxy, not a calibrated probability of task failure. Access to accurate external app information is a dependency; AndroidWorld is the reported test environment.","entire_paper_read":false,"evidence_level":"fulltext_sections","mechanism":"Mismatch between predicted and observed interface descriptions triggers retrieval from docs/code/trajectories into structured AppCards with functional and parameter knowledge.","pages":[1,3,4,5],"read_at":"2026-10-18T10:16:14.837469+00:00","sections":["3.1 Problem Formulation","3.2 Curiosity-Driven Information Gain Estimation","3.3 Retrieving and Embedding External Knowledge"]},"resource_id":"arxiv:2601.19306","retained_abstract_sha256":"223b9a915e5ea9849644318e1e5426e64ad8655ab46d0b007c30e1967130dc50","retrieved_at":"2026-10-18T10:14:20.444793+00:00","title":"Curiosity Driven Knowledge Retrieval for Mobile Agents","version_binding":{"embedded_arxiv_version_marker_found":true,"limitation":null,"method":"Exact official version URL; retained bibliography; embedded marker when available.","requested_exact_version":"2601.19306v1"},"version_read":1,"version_updated_date":"2026-01-27","versioned_abstract_url":"https://arxiv.org/abs/2601.19306v1"},{"authors":["Shang Ma","Xusheng Xiao","Yanfang Ye"],"bibliography_basis":"fresh official Atom title-search record supplied by root and exact-version primary full text","body_bytes":1032315,"body_sha256":"5e2cb2f053983140904ec15c097d060ff18315c6c3c956bbcacbec26921709c5","citation_key":"arxiv_2510_06042","final_url":"https://arxiv.org/pdf/2510.06042v3","first_submitted_date":"2025-10-07","fresh_bibliography_evidence":"research/survey/evidence/title-search.json","fresh_bibliography_source_page":6,"http_status":200,"media_type":"application/pdf","primary_url":"https://arxiv.org/pdf/2510.06042v3","read_evidence":{"boundary":"Optimality/correctness are relative to the modeled graph and selected target; they do not guarantee accurate grounding, complete graph or faithful user-goal translation. Requires static source analysis in this design.","entire_paper_read":false,"evidence_level":"fulltext_sections","mechanism":"Static app-code analysis initializes UI transition graph, online exploration updates it, and a symbolic PDDL planner guides a modular UI executor.","pages":[1,4,5,6],"read_at":"2026-10-18T10:16:14.837470+00:00","sections":["3.3 UI Automation to Classical Planning","4.1 UTG Builder","4.2 Node Selector","4.3 Plan Generator","4.4 UI Explorer","5.1 Datasets"]},"resource_id":"arxiv:2510.06042","retained_abstract_sha256":"4bf0a561460c080b14da6fd13f6d92781e01e52b08dc691fe019e70b88e44c86","retrieved_at":"2026-10-18T10:14:29.861666+00:00","title":"Agent+P: Guiding UI Agents via Symbolic Planning","version_binding":{"embedded_arxiv_version_marker_found":true,"limitation":null,"method":"Exact official version URL; retained bibliography; embedded marker when available.","requested_exact_version":"2510.06042v3"},"version_read":3,"version_updated_date":"2026-09-27","versioned_abstract_url":"https://arxiv.org/abs/2510.06042v3"},{"authors":["Jingru Fan","Yufan Dang","Jingyao Wu","Huatao Li","Runde Yang","Xiyuan Yang","Yuheng Wang","Chen Qian"],"bibliography_basis":"fresh official Atom title-search record supplied by root and exact-version primary full text","body_bytes":139883243,"body_sha256":"6c2591dad8893f87cffdea211584a3733dc2137dafa9416c1885d1b0126fd540","citation_key":"arxiv_2509_02444","final_url":"https://arxiv.org/pdf/2509.02444v2","first_submitted_date":"2025-09-02","fresh_bibliography_evidence":"research/survey/evidence/title-search.json","fresh_bibliography_source_page":6,"http_status":200,"media_type":"application/pdf","primary_url":"https://arxiv.org/pdf/2509.02444v2","read_evidence":{"boundary":"Replay is explicitly for stable layouts and fixed tasks; calibration assumes valid detected widgets and may choose the wrong nearest widget. Function-call routes described here are user-selected, not model-selected. Do not repeat zero-latency or generic performance assertions.","entire_paper_read":false,"evidence_level":"fulltext_sections","mechanism":"Region-based coordinate correction with a history of ineffective corrections, hierarchical task DAGs, and semantic retrieval of successful logs for direct replay.","pages":[1,5,6,34,37,38,42,48,49,50],"read_at":"2026-10-18T10:50:25.723665+00:00","sections":["4.2 OCR/OR Region Localization","4.2.2 Coordinate Calibration","4.3 Multi-Agent Voting","5.2 Task Decomposition","6.2 Experience Reuse","6.3 Function Calling"]},"resource_id":"arxiv:2509.02444","retained_abstract_sha256":"49a887b12b3395821aa100a66e61aac808309ef947ae752475df3d2db9a4fcd8","retrieved_at":"2026-10-18T10:14:33.619660+00:00","title":"AppCopilot: Toward General, Accurate, Long-Horizon, and Efficient Mobile Agent","version_binding":{"embedded_arxiv_version_marker_found":false,"limitation":"AppCopilot PDF front matter has no extracted arXiv version marker; exact version is bound by the requested official v2 URL and official Atom metadata. The front-matter title and ordered authors were read.","method":"Exact official version URL; retained bibliography; embedded marker when available.","requested_exact_version":"2509.02444v2"},"version_read":2,"version_updated_date":"2025-10-17","versioned_abstract_url":"https://arxiv.org/abs/2509.02444v2"},{"authors":["Xin Wang","Zhiyao Cui","Hao Li","Ya Zeng","Chenxu Wang","Ruiqi Song","Yihang Chen","Kun Shao","Qiaosheng Zhang","Jinzhuo Liu","Siyue Ren","Shuyue Hu","Zhen Wang"],"bibliography_basis":"fresh official Atom title-search record supplied by root and exact-version primary full text","body_bytes":2084072,"body_sha256":"3c3474b771af7bdd500e6aaa91b83d39ec0d455e59bb83df906e7a94700ad2b7","citation_key":"arxiv_2508_18040","final_url":"https://arxiv.org/pdf/2508.18040v1","first_submitted_date":"2025-08-25","fresh_bibliography_evidence":"research/survey/evidence/title-search.json","fresh_bibliography_source_page":6,"http_status":200,"media_type":"application/pdf","primary_url":"https://arxiv.org/pdf/2508.18040v1","read_evidence":{"boundary":"Personalization depends on the validity of inferred/stored user information and retrieval access. This is instruction completion plus interaction, not demonstrated permanent model-weight personalization.","entire_paper_read":false,"evidence_level":"fulltext_sections","mechanism":"Detects user-specific missing elements, fills them by memory retrieval or agent exploration of likely apps, and asks the user when unresolved.","pages":[1,3,4],"read_at":"2026-10-18T10:50:25.723877+00:00","sections":["The Framework of PerPilot","Personalization Perception","Personalization Completion","PerInstruct Data Construction"]},"resource_id":"arxiv:2508.18040","retained_abstract_sha256":"57d5fc98a2eb2232f237d71b34060df0acef89cfeaa4bc247580a4dda3223e0b","retrieved_at":"2026-10-18T10:18:36.598801+00:00","title":"PerPilot: Personalizing VLM-based Mobile Agents via Memory and Exploration","version_binding":{"embedded_arxiv_version_marker_found":true,"limitation":null,"method":"Exact official version URL; retained bibliography; embedded marker when available.","requested_exact_version":"2508.18040v1"},"version_read":1,"version_updated_date":"2025-08-25","versioned_abstract_url":"https://arxiv.org/abs/2508.18040v1"},{"authors":["Linghua Zhang"],"bibliography_basis":"frozen catalog retained arXiv bibliography and exact-version primary full text","body_bytes":144356,"body_sha256":"ddc55d413582d0a17eec43bb37ffaeb348c4228b434694cc653853de2b32d33a","citation_key":"arxiv_2609_30186","final_url":"https://arxiv.org/html/2609.30186v1","first_submitted_date":"2026-09-24","http_status":200,"media_type":"text/html; charset=utf-8","primary_url":"https://arxiv.org/html/2609.30186v1","read_evidence":{"boundary":"No coordinate fallback for tree-missing targets; local is decision scope, not on-device inference. Compared full systems differ in executors; no small-VLM replacement ablation. Efficiency measures are conditional on successful trajectories; no numerical claim repeated.","entire_paper_read":false,"evidence_level":"fulltext_sections","extracted_text_line_ranges":[[217,388],[456,509],[684,701]],"mechanism":"VLM sets local goals and exact text; a remote typed decision service selects observation-bound action candidates; per-action reobservation rebuilds IDs and returns DONE/BLOCKED to the planner.","pages":[],"read_at":"2026-10-18T10:50:25.723908+00:00","sections":["2 Problem Setup","3.1 Delegation State","3.2 Current Accessibility-Tree Candidates","3.3 Typed Local Decision","3.4 Budgets and Boundary","5 Experimental Design","8 Limitations"]},"resource_id":"arxiv:2609.30186","retained_abstract_sha256":"b387c1cfe77f6e06477415944b3923189a1f0ca30be0680f01815cb65b0067bc","retrieved_at":"2026-10-18T10:49:32.541714+00:00","title":"Jev-Mobile: Jev as an Executor for Mobile GUI Agents","version_binding":{"embedded_arxiv_version_marker_found":true,"limitation":null,"method":"Exact official version URL; retained bibliography; embedded marker when available.","requested_exact_version":"2609.30186v1"},"version_read":1,"version_updated_date":"2026-09-24","versioned_abstract_url":"https://arxiv.org/abs/2609.30186v1"}],"version_and_bibliography_notes":[{"issue":"Search snippet title differs from retained catalog title.","resolution":"Exact v3 PDF title matches the frozen catalog; v3 full-text title is used.","resource_id":"arxiv:2606.05597"},{"issue":"PDF carries placeholder conference/date/DOI template text.","resolution":"No publication facts inferred from the template. Citation uses retained arXiv v2 metadata.","resource_id":"arxiv:2504.14603"},{"issue":"No arXiv version marker found in extracted PDF text.","resolution":"Exact v2 primary URL and fresh official Atom bibliography bind the source. No claim that an embedded version stamp was seen.","resource_id":"arxiv:2509.02444"},{"issue":"PDF report date and arXiv submission stamp are adjacent calendar dates.","resolution":"Retain official Atom first-submission and update timestamps for bibliography; do not replace them with report-cover date.","resource_id":"arxiv:2607.28227"}]}