GUI grounding
Source-reviewed research and assets tagged with GUI grounding.
Membership follows reviewed tags; it does not establish a dependency between neighboring records. Dates preserve source precision. Annotations retain their original language.
Gemini Computer Use
Gemini capability for generating interface actions from screenshots.
GroundGUI-8B
Screenshot grounding checkpoint that emits a single click in a normalized coordinate space.
GUI-Actor 7B Qwen2-VL
GUI-Actor checkpoint fine-tuned from Qwen2-VL-7B-Instruct, with about 8.4B BF16 parameters, whose config names a pointer variant of the Qwen2-VL class rather than the stock one, under MIT.
GUI-G1
GUI-grounding training code that modifies reasoning templates, reward functions, and the GRPO objective, with a linked model release.
GUI-G1 3B v1
A Qwen2.5-VL-3B checkpoint released for R1-style visual grounding experiments, predicting interface target locations from screenshots and instructions.
GUI-G2
GUI grounding models at 3B and 7B trained with Gaussian rewards for click proximity to element centers and region coverage, with variance scaled to element size.
GUI-G2 7B
A Qwen2.5-VL grounding checkpoint trained with Gaussian point and coverage rewards whose granularity adapts to interface-element scale.
GUI-Owl 7B
A Qwen2.5-VL-based GUI checkpoint released with Mobile-Agent-v3 for grounded actions and specialized roles in its agent framework.
GUI-R1
GUI action-model training and evaluation code with verifiable rewards, plus linked reinforcement-learning data and checkpoints.
GUI-R1 checkpoint bundle
A single Hub release containing GUI-R1 3B and 7B action-model checkpoints trained with reinforcement learning across interface platforms.
Holo1 7B
A Qwen2.5-VL-based action model released for the Surfer-H web-agent system, supporting policy, interface-localization and validation roles.
Holo2 4B
A Qwen3-VL-based checkpoint for navigation and multi-step GUI actions across web, desktop and mobile interfaces.
Holo3 35B-A3B
A sparse Qwen3.5-based GUI-action checkpoint released for navigation and computer-use workflows.
Holo4 35B-A3B BF16
A Qwen3.6-based computer-use checkpoint designed to request GUI actions, code execution and tool calls through an external agent harness.
InfiGUI-G1
InfiGUI-G1 3B and 7B grounding models trained with Adaptive Exploration Policy Optimization, which samples several candidate answers per pass under an adaptive exploration reward.
InfiGUI-G1 7B
A GUI-grounding checkpoint trained with adaptive exploration rewards and multiple candidate answers to improve instruction-to-element alignment.
InfiGUI-R1 3B
A Qwen2.5-VL checkpoint trained with the Actor2Reasoner procedure to combine interface grounding with planning and recovery reasoning.
Magma
Microsoft foundation model for multimodal agents, pretrained with Set-of-Mark and Trace-of-Mark objectives on UI, robotics and video data and fine-tunable for web and mobile UI navigation.
MAI-UI 8B
A released MAI-UI checkpoint combining mobile GUI actions, agent-user interaction and tool-use-oriented navigation.
OmniParser v2 model bundle
A screen-parsing weight bundle containing icon-region detectors and an icon captioner for converting screenshots into agent-readable interface elements.
OpenCUA 32B
A Qwen2.5-VL-based computer-use checkpoint that combines screenshot history with planning and grounded executable actions.
OpenCUA 72B
A Qwen2.5-VL-based computer-use checkpoint that combines screenshot history and reflective reasoning to predict executable desktop actions.
OS-Atlas
OS-Atlas GUI action models, including grounding bases fine-tuned from InternVL2-4B and Qwen2-VL-7B-Instruct that accept any image size and output 0-1000 relative coordinates.
OS-Atlas Base 4B
OS-Atlas base checkpoint with about 4.1B parameters, fine-tuned from InternVL2-4B and loaded in Transformers through custom InternVL chat model code.
OS-Atlas Base 7B
OS-Atlas base checkpoint fine-tuned from Qwen2-VL-7B-Instruct, using the standard Qwen2-VL architecture in Transformers with about 8.3B BF16 parameters.
Phi-Ground
Microsoft GUI grounding models from an empirical study of grounding training, from data collection to model training, released as Phi-Ground-4B-7C with evaluation code.
Phi-Ground checkpoint
Microsoft Phi-Ground GUI grounding checkpoint fine-tuned from Phi-3.5-vision-instruct, shipped as two PyTorch weight shards with custom Phi-3-V configuration and processing code under MIT.
POINTS-GUI
POINTS-GUI-G-8B GUI grounding model, trained from a POINTS-1.5 base without native grounding through a data pipeline that unifies coordinates, filters noise and adds hard cases.
POINTS-GUI-G 8B
A specialized interface-grounding checkpoint that returns normalized point coordinates for a requested element in a screenshot.
ScaleCUA 7B
A Qwen2.5-VL-based computer-use checkpoint with direct grounding and agent action modes across desktop, mobile and web interfaces.
SeeClick checkpoint
The released SeeClick visual grounding checkpoint for mapping an interface screenshot and referring instruction to a point or bounding box.
Seg2Click Qwen2.5-VL 3B
Seg2Click checkpoint based on Qwen2.5-VL-3B with a custom action-mask decoder.
Seg2Click Qwen3.5 0.8B
Compact Seg2Click checkpoint with a merged Qwen3.5 backbone and an action-mask decoder.
Seg2Click Qwen3.5 2B
Seg2Click checkpoint using a 2B Qwen3.5 backbone to predict a mask and select an interface click.
Seg2Click Qwen3.5 4B
Seg2Click checkpoint pairing a 4B Qwen3.5 backbone with pixel-level action-mask prediction.
Seg2Click Qwen3.5 9B
Seg2Click checkpoint pairing a 9B Qwen3.5 backbone with a decoder for valid GUI click regions.
ShowUI
Lightweight GUI-agent vision-language-action model from Show Lab and Microsoft, released with UI-guided token selection, training code, the ShowUI-web and ShowUI-desktop datasets and vLLM inference.
ShowUI 2B
ShowUI vision-language-action checkpoint fine-tuned from Qwen2-VL-2B-Instruct, published as a single PyTorch weight file under the MIT license and tagged for GUI agents and computer use.
ShowUI-π
Vision-language-action model and training code for GUI dragging, with links to checkpoints and continuous-action evaluation data.
ShowUI-π checkpoint
A compact visual-action checkpoint that predicts continuous GUI drag trajectories from a screenshot and a natural-language instruction.
SpiritSight-Agent 8B base
An InternVL2-based visual GUI checkpoint distributed for grounding and interface navigation, with the publisher recommending task-specific fine-tuning of the base release.
UGround
Code, training data and weights for universal visual grounding of GUI elements from screenshots, including Qwen2-VL-based UGround-V1 at 2B, 7B and 72B and offline and online evaluation suites.
UGround V1 7B
UGround-V1 checkpoint fine-tuned from Qwen2-VL-7B, with about 8.3B BF16 parameters in the standard Qwen2-VL architecture, provided as safetensors and a PyTorch file under Apache-2.0.
UI-AGILE
GUI agent trained by RL with a continuous reward, a simple-thinking reward and cropping-based resampling on about 9k samples, plus a plug-in decomposed grounding-and-selection inference method.
UI-AGILE checkpoint
The released UI-AGILE model for GUI grounding with reward shaping, crop-based resampling and decomposed selection during inference.
UI-R1
Rule-based reinforcement learning for GUI action prediction, releasing UI-R1-3B, trained from Qwen2.5-VL-3B, the efficiency-oriented UI-R1-E-3B, and training data.
UI-R1 3B
A Qwen2.5-VL checkpoint trained with reinforcement learning for GUI action prediction and evaluated on mobile, desktop and web grounding tasks.
UI-TARS
Native GUI agent models; UI-TARS-1.5 adds reinforcement-learned reasoning before acting and an open 7B checkpoint, and UI-TARS-2 extends to games, code and tool use.
UI-TARS 1.5 7B
UI-TARS 1.5 GUI-agent checkpoint in the Qwen2.5-VL architecture with about 8.3B parameters stored in FP32, released under Apache-2.0 for English image-text-to-text use in Transformers.
UI-Venus
GUI agent model family; UI-Venus-2, at 9B and 27B, acts on mobile, web and desktop in one end-to-end agent, trained with keypoint-based multi-judge task verification.
UI-Venus 2 9B
UI-Venus 2 GUI-agent checkpoint fine-tuned from Qwen3.5-9B, with about 9.4B BF16 parameters in the Qwen3.5 multimodal architecture, loaded through Transformers for image-text-to-text.
UIX-Qwen2
A MultiUI-trained visual-language checkpoint for text-rich webpage understanding and interface tasks.
ZonUI-3B checkpoint
Qwen2.5-VL-3B-based GUI grounding checkpoint that predicts click locations from screenshots and text instructions across desktop, mobile and web screens.
AgentStudio
Toolkit and Ubuntu-based environment for building and evaluating generalist computer-use agents through GUI and API interaction.
Showing 301–360 of 380 resources.
Get full records and evidence through the topic export manifest. Source records can be passed to the standard RAG and citation exporters after cloning the catalog.