AwesomeCUAGitHub ↗
Research papers

Mapping Natural Language Instructions to Mobile UI Action Sequences

Introduces PIXELHELP for predicting mobile UI action sequences from instructions, together with action-phrase annotations and synthetic UI action descriptions for training. Separate Transformers extract instruction steps and ground object descriptions using interface content and screen position.

AI-assisted editorial review against a primary source. This is a catalog annotation, not an independent reproduction or endorsement.

Original source ↗Read as MarkdownRecord JSON

What is this resource?

Resource type
Research papers
Environment
unspecified
Topics
mobile-navigation, grounding, data-generation, action-prediction
Record publication date
2020-05-07
Evidence retrieved
2026-10-16T06:31:20Z
Resource ID
arxiv:2005.03776

Recorded paper version

arXiv v2

Version updated: 2020-06-05T02:11:56Z.

This identifies the retained arXiv metadata snapshot; newly observed changes require separate review.

Abstract fingerprint3af7b45455ffbd616e8cdf5612b19275ecb7bf07b1065af9f5b724ae19e96fb4

Authors

Yang Li, Jiacong He, Xin Zhou, Yuan Zhang, Jason Baldridge

Cite this paper

Download BibTeX · Download CSL JSON

Citations preserve the selected source edition, its authors and recorded date precision. Unrecorded venues and identifiers are omitted.

Publication record

Title
Mapping Natural Language Instructions to Mobile UI Action Sequences
Authors
Yang Li, Jiacong He, Xin Zhou, Yuan Zhang, Jason Baldridge
Publication date (recorded precision)
2020-07
Venue
Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics
Publisher
Association for Computational Linguistics
Pages
8198-8210
DOI
10.18653/v1/2020.acl-main.729

Publication source ↗ · Publisher or author PDF ↗

Publication BibTeX · Publication CSL JSON

This publication is a separately recorded edition of the same work. Its citation uses its own title, authors and date; the retained arXiv citation above is unchanged.

Date source: Explicit publisher metadata: 2020/7; precision=month.

Source notes: Publisher-hosted proceedings record; distinct from the retained arXiv preprint snapshot.

Explore related topics

Mobile navigationGUI groundingData generationAction prediction

Scope and limitations

The annotation and scope were reviewed against the primary paper abstract. Experimental results and released artifacts have not been independently reproduced.

Primary evidence and verification

Source retrieval is separate from experiment reproduction. Use the original publication or publisher page when citing claims about the resource.

  1. arxiv api

    Retrieved 2026-10-16T06:31:20Z. Supports: name, authors, published_at, updated_at, identifiers.arxiv, details.arxiv_snapshot.

    Content fingerprintcbe8bfcbce4170a94a6cf42de01fbe0a448e7b1d29fa75a0de877105a98ec08e
  2. editorial primary review

    Retrieved 2026-10-17T09:53:48+00:00. Supports: details.publication.authors, details.publication.date_basis, details.publication.date_parts, details.publication.doi, details.publication.identity_basis, details.publication.notes, details.publication.pages, details.publication.pdf_url, details.publication.publisher, details.publication.title, details.publication.type, details.publication.url, details.publication.venue.

    Content fingerprintc196294e19378d08d71a7cfd41c2e368f5a6b49ee7c3601257537b9ae5c42add
  3. editorial primary review

    Retrieved 2026-10-16T06:31:20Z. Supports: name, kind, summary, environments, topics, scope.

    Content fingerprintcbe8bfcbce4170a94a6cf42de01fbe0a448e7b1d29fa75a0de877105a98ec08e

Suggest a correction

Open a source-backed correction. Include this record ID and the original source. Read our editorial policy and coverage boundaries.