0

VisualWebArena

Active

910 visually grounded web tasks across three self-hosted sites (Classifieds, Shopping, Reddit) requiring image understanding to complete.

Open
Publisher
Carnegie Mellon University
Capabilities
Browser UseImage UnderstandingPlanning
Domain
agentic
Format
Web Arena
Size
910 tasks
License
Apache-2.0
Published
Jan 2024
Notable for
Benchmark for evaluating browser use, image understanding and planning in the agentic domain.
Canonical
jykoh.com/vwa
Also on
github.com/web-arena-x/visualwebarena

Cite

Notes

Only stored in your browser.

Related tools

1
View all

Implementations, trainers, datasets and scaffolds linked to this eval.

BrowserGym

ServiceNow Research

ServiceNow's unified Gym-style framework for web agents - wraps WebArena, MiniWoB, VisualWebArena, WorkArena, AssistantBench, WebLINX, and more under one Playwright-backed interface.

Trains towardRL EnvPlanningTool CallingBrowser Use

Papers

2

VisualWebArena: Evaluating Multimodal Agents on Realistic Visual Web Tasks

ACL · 2024

CMU benchmark extending WebArena with 910 visually grounded tasks across Classifieds, Shopping, and Reddit, evaluating multimodal browsing agents.

introduces

VisualWebArena: Evaluating Multimodal Agents on Realistic Visual Web Tasks

ACL · 2024

CMU benchmark extending WebArena with 910 visually grounded tasks across Classifieds, Shopping, and Reddit, evaluating multimodal browsing agents.

Contributors

2
JJing Yu KohGGraham Neubig

FAQ

What is VisualWebArena?
910 visually grounded web tasks across three self-hosted sites (Classifieds, Shopping, Reddit) requiring image understanding to complete.
What capabilities does VisualWebArena test?
VisualWebArena evaluates browser use, image understanding, planning.
How can a model improve its VisualWebArena score?
Tools linked to VisualWebArena on Sophon include BrowserGym - RL environments, datasets, and scaffolds that target this eval.
What license is VisualWebArena under?
VisualWebArena is available under Apache-2.0.