# Multimodal Web Navigation with Instruction-Finetuned Foundation Models

Trains WebGUM offline from demonstrations by jointly tuning an instruction-following language model and a vision encoder over webpage screenshots and HTML. The agent predicts browser actions and is evaluated on MiniWoB, WebShop, and Mind2Web.

- Catalog URL: https://awesomecua.github.io/resources/arxiv-2305-11854/
- Original source: https://arxiv.org/abs/2305.11854
- Kind: paper
- Review: curated
- Scope: core
- Environments: web
- Topics: web-navigation, imitation-learning, action-prediction
- Evidence checked: 2026-10-16T06:31:20Z
- ID: arxiv:2305.11854

## Limitations

Reviewed against the retained original arXiv abstract. Experimental results and artifact availability have not been independently reproduced.

- Record JSON: https://awesomecua.github.io/resources/arxiv-2305-11854/record.json

## Authors

Hiroki Furuta, Kuang-Huei Lee, Ofir Nachum, Yutaka Matsuo, Aleksandra Faust, Shixiang Shane Gu, Izzeddin Gur

## Citation exports

- BibTeX: https://awesomecua.github.io/resources/arxiv-2305-11854/citation.bib
- CSL JSON: https://awesomecua.github.io/resources/arxiv-2305-11854/citation.csl.json

## Publication record

- Title: Multimodal Web Navigation with Instruction-Finetuned Foundation Models
- Authors: Hiroki Furuta, Kuang-Huei Lee, Ofir Nachum, Yutaka Matsuo, Aleksandra Faust, Shixiang Shane Gu, Izzeddin Gur
- Publication date (recorded precision): 2024-05-31
- Publication source: https://proceedings.iclr.cc/paper_files/paper/2024/hash/7ef7d8359036afd8c2378d82c21058a4-Abstract-Conference.html
- Publication BibTeX: https://awesomecua.github.io/resources/arxiv-2305-11854/publication-citation.bib
- Publication CSL JSON: https://awesomecua.github.io/resources/arxiv-2305-11854/publication-citation.csl.json
- venue: International Conference on Learning Representations
- volume: 2024
- pages: 29528-29558
- date basis: Explicit publisher metadata: 2024-05-31; precision=day.
- identity basis: Same work: exact normalized landing title, corresponding author list, and matching specific contribution. Publisher and arXiv versions remain distinct.
- notes: The ICLR landing shortens Shixiang Shane Gu to Shixiang Gu. Publication authors use the full PDF title-page form. Volume 2024 and pages 29528–29558 are explicit publisher citation metadata, not values inferred from the event year or PDF page count. Publisher-hosted proceedings record; distinct from the retained arXiv preprint snapshot.
- Publication evidence: https://proceedings.iclr.cc/paper_files/paper/2024/hash/7ef7d8359036afd8c2378d82c21058a4-Abstract-Conference.html, https://proceedings.iclr.cc/paper_files/paper/2024/file/7ef7d8359036afd8c2378d82c21058a4-Paper-Conference.pdf

## Recorded arXiv snapshot

- Version: 4
- Version source: https://arxiv.org/abs/2305.11854v4
- First submission: 2023-05-19T17:44:34Z
- Version updated: 2024-02-25T16:21:00Z
- Abstract SHA-256: ac0ea5a462890317ea29ade32c879d016181afbe5962e2d28cc175e251df41e6
- Abstract hash normalization: sha256-utf8-whitespace-collapsed-v1

## License declarations

- code: not recorded
- data: not recorded
- document: not recorded
- weights: not recorded

## Primary evidence

- https://export.arxiv.org/api/query?id_list=1802.08802%2C1808.09132%2C2005.03776%2C2101.09465%2C2103.16057%2C2104.08560%2C2105.13231%2C2107.03374%2C2107.13731%2C2108.03353%2C2202.02312%2C2202.08137%2C2204.01691%2C2205.11029%2C2207.01206%2C2209.08199%2C2209.08655%2C2209.14927%2C2209.15099%2C2210.02663%2C2210.03629%2C2210.03945%2C2211.07615%2C2301.10165%2C2301.13280%2C2302.04761%2C2303.07280%2C2303.08774%2C2303.17491%2C2303.17580%2C2304.04370%2C2304.07061%2C2305.04091%2C2305.05662%2C2305.08144%2C2305.10601%2C2305.11854%2C2305.12487%2C2305.14257%2C2305.14325%2C2305.16504%2C2305.16653%2C2305.19308%2C2306.00245%2C2306.06070%2C2306.07863%2C2307.10088%2C2307.12856%2C2307.13854%2C2308.01990&max_results=50 (retrieved 2026-10-16T06:31:20Z; SHA-256 cbe8bfcbce4170a94a6cf42de01fbe0a448e7b1d29fa75a0de877105a98ec08e)
- https://proceedings.iclr.cc/paper_files/paper/2024/hash/7ef7d8359036afd8c2378d82c21058a4-Abstract-Conference.html (retrieved 2026-10-17T09:53:48+00:00; SHA-256 6669190c7fc2a37f9cd68d0c49431b8f17b626c16bb3ef4093fc83a6ac62acd3)
- https://proceedings.iclr.cc/paper_files/paper/2024/file/7ef7d8359036afd8c2378d82c21058a4-Paper-Conference.pdf (retrieved 2026-10-17T09:59:46+00:00; SHA-256 da26ea085252edfda4c084058aa81ea023f885ff97c67f2ffae0698bab6f568b)
- https://export.arxiv.org/api/query?id_list=1802.08802%2C1808.09132%2C2005.03776%2C2101.09465%2C2103.16057%2C2104.08560%2C2105.13231%2C2107.03374%2C2107.13731%2C2108.03353%2C2202.02312%2C2202.08137%2C2204.01691%2C2205.11029%2C2207.01206%2C2209.08199%2C2209.08655%2C2209.14927%2C2209.15099%2C2210.02663%2C2210.03629%2C2210.03945%2C2211.07615%2C2301.10165%2C2301.13280%2C2302.04761%2C2303.07280%2C2303.08774%2C2303.17491%2C2303.17580%2C2304.04370%2C2304.07061%2C2305.04091%2C2305.05662%2C2305.08144%2C2305.10601%2C2305.11854%2C2305.12487%2C2305.14257%2C2305.14325%2C2305.16504%2C2305.16653%2C2305.19308%2C2306.00245%2C2306.06070%2C2306.07863%2C2307.10088%2C2307.12856%2C2307.13854%2C2308.01990&max_results=50 (retrieved 2026-10-16T06:31:20Z; SHA-256 cbe8bfcbce4170a94a6cf42de01fbe0a448e7b1d29fa75a0de877105a98ec08e)