VeriWeb: Verifiable Long-Chain Web Benchmark for Agentic Information-Seeking

Note

This project was originally named VeriGUI. As our initial data collection focused on web-based tasks that primarily involve information-seeking rather than GUI interaction, we now define this part as the standalone VeriWeb benchmark, while desktop and other GUI-oriented scenarios will be released as a separate benchmark (in progress). We apologize for any resulting confusion.

Overview

VeriWeb is a novel verifiable long-chain web benchmark designed to facilitate the evaluation and development of web agents within realistic web environments. Unlike existing efforts that mainly focus on single-fact retrieval and rely on outcome-only verification, VeriWeb emphasizes long-chain complexity and subtask-level verifiability to better reflect realistic knowledge-intensive scenarios.

Updates

[Jan 21, 2026] Fixed and updated data: 658 659 662 665 683 687 698 701 703 743 744 750 V3_80 V2_12 V5_61 V6_20
[Oct 23, 2025] 🔥 We have released the updated 302 web-based task trajectories!
[Jul 21, 2025] 🔥 We have released the first batch of 130 web-based task trajectories!

Key Features

🔗 Long-Chain Complexity

302 realistic information-seeking tasks across 5 real-world domains
Long-chain web trajectories decomposed into multiple interdependent subtasks
Tasks combine breadth-oriented search and depth-oriented search
Agents must retrieve, reason, and synthesize evidence from diverse web pages

✅ Subtask-Level Verifiability

Fine-grained evaluation at each intermediate subtask, not only the final outcome
Fixed, verifiable target outputs for every subtask while supporting diverse exploration strategies
Each subtask can serve as an independent starting point, enabling evaluation at different stages of a task
Rich supervision signals for diagnosing failure modes

🧑‍🎨 Human-Expert Annotation

All tasks and trajectories carefully created and annotated by human experts
High-quality task instructions, subtask decompositions, and answer annotations
Each task includes a complete human demonstration with detailed observation and action logs

Visualize Tool

Usage

Open VeriGUI.2077ai.org
Select the corresponding task data folder
View the visualization results

Features

Interactive event timeline visualization
Support for various event types (MOUSE_DRAG, MOUSE_UP, TAB_CHANGE, etc.)
Video playback synchronization
Jump to specific actions functionality

Dataset Structure

VeriGUI/
├── task_001/
│   ├── data.json         # Complete task annotation
│   └── video.mp4         # Video recording of task execution
└── task_002/
      ├── data.json
      └── video.mp4

Task Structure

📋 Complete Task
├── instruct (String): Complete task instruction
├── result (String): Final answer for the complete task
└── actions (Array): List of subtask
    │
    └── 📝 Subtask
        ├── instruct (String): Subtask-level instruction
        ├── result (String): Subtask-level answer for the subtask
        └── innerActions (Array): Low-level GUI actions within this step
            │
            └── 🖱

## 数据概览
VeriWeb 面向网页信息检索和验证任务，当前镜像包含 JSON、图像、视频及相关说明文件，适合智能体评测和网页操作研究。

609 个文件

浏览文件

数据集版权信息

本数据集的许可证为 Apache License 2.0。如有违反相关条款，请联系 WEHUB，我们将及时处理。查看许可证

2077AIDataFoundation/VeriWeb 已完整同步