从报名到提交的完整参赛指引
Your complete guide from registration to submission
在 Octo 平台使用邮箱注册账号，加入赛事空间并完成组队。组队时请填写队员的 GitHub 账号，后续官方会为每支队伍创建私有比赛仓库（内含比赛代码模板），队员通过 GitHub 账号获取仓库权限并提交代码。
报名后即可开始备赛。阅读论文了解 Protocol III，基于开源项目熟悉 Agent 设计与评测流程，提前构建你的 Agent。
比赛模板发布后，将基于开源项目开发好的 Agent 迁移到模板中，只需简单适配。
正式评测前一周开放。确保依赖安装和运行环境配置正确，通过冒烟测试后方可提交正式评测。
Push 代码到 main 分支，通过赛事 Bot 发起评测。系统自动拉取代码、安装依赖、在云端沙箱中执行评测任务并返回评分。评测窗口期内可多次提交，取最优成绩。
Sign up on the Octo platform with your email, join the competition space, and form your team. Please provide each member's GitHub username during team formation — a private competition repository (containing the code template) will be created for each team, and members will be granted access via their GitHub accounts to submit code.
Start preparing right after registration. Read the paper, understand Protocol III, explore the open-source project to learn Agent design and evaluation, and start building your Agent.
Once the competition template is released, migrate your Agent into it with only minor adaptations needed.
Opens one week before formal evaluation. Ensure dependencies and runtime are properly configured; you must pass the smoke test before submitting.
Push code to the main branch, initiate evaluation via the Competition Bot. The system automatically pulls code, installs dependencies, runs your Agent in cloud sandboxes, and returns your score. Multiple submissions allowed; best score counts.
报名后即可开始备赛：
Start preparing right after registration:
⚠️ 注意：评测环境通过 CDP 提供云端浏览器，Agent 必须基于 Playwright 进行所有浏览器交互操作。
⚠️ Note: The evaluation environment provides cloud browsers via CDP. All browser interactions must be performed through Playwright.
① 阅读论文，了解 Protocol III — WebRetriever 定义了三种评测协议，本次比赛采用 Protocol III（端到端任务协议）。建议结合 论文 和 数据集 了解任务格式与评分标准。
① Read the paper & understand Protocol III — WebRetriever defines three evaluation protocols; this competition uses Protocol III (End-to-End Task Protocol). Read the paper and explore the dataset to understand the task format and scoring criteria.
② 熟悉开源项目 — 建议先 Clone 开源仓库 并运行示例，重点了解两部分：Agent 设计（src/agent/，基于 UI-TARS 1.5 的完整实现，包括 VLM 交互、浏览器控制、Prompt 模板等）和 NavEval 自动评测（src/eval/，帮助你在本地自动评估 Agent 的导航效果）。
② Explore the open-source project — Clone the open-source repository and run the examples. Focus on two key parts: Agent design (src/agent/, a complete implementation based on UI-TARS 1.5, covering VLM interaction, browser control, and prompt templates) and NavEval automated evaluation (src/eval/, helps you evaluate your Agent's navigation performance locally).
💡 注意：开源项目提供的是 Protocol I（导航协议）的示例实现，相比本次比赛采用的 Protocol III，不包含信息提取部分——Protocol III 要求 Agent 不仅导航到正确页面，还需提取目标信息作为最终答案。由于信息提取的实现方式多样，且与所设计的 Agent 架构密切相关，需要参赛者自行设计与实现。
💡 Note: The open-source project demonstrates Protocol I (Navigation Protocol). Compared to Protocol III used in this competition, it does not include the information extraction component — Protocol III requires Agents to not only navigate to the correct page, but also extract target information as the final answer. Since extraction approaches vary widely and depend on your Agent design, participants need to implement this part on their own.
③ 比赛模板与资源 — 比赛代码模板将在后续发布，它是在开源项目基础上进行精炼，并对评测接口做了标准化，届时只需简单适配即可迁移。比赛期间，模型推理资源由选手自行管理（商业 API 或自研/自部署模型均可），评测环境（云端浏览器沙箱）由官方统一提供。
③ Competition template & resources — The competition code template will be released later. It is a streamlined version of the open-source project with standardized evaluation interfaces — migrating your Agent requires only minor adaptations. During the competition, model inference resources are managed by each team (commercial APIs or self-hosted models), while the evaluation environment (cloud browser sandboxes) is provided by the organizers.
🛠️ 备赛调试（选手自行完成）
🚀 正式评测（官方提供环境）
🛠️ Preparation (Your Setup)
🚀 Formal Evaluation (Official Environment)
A: 共 100 道基于 Protocol III 的端到端评测任务，覆盖多行业真实网站。Agent 需从指定网页出发，自主导航并提取目标信息返回最终答案。
A: 100 end-to-end tasks based on Protocol III, covering real websites across multiple industries. Agents must navigate from a designated webpage and extract target information to return the final answer.
⚠️ 比赛题目均为本次挑战赛独立设计，与开源数据集中的评测题目互不重叠。
⚠️ All competition tasks are exclusively designed for this challenge and do not overlap with any tasks in the open-source dataset.
A: 是的。Agent 通过 OpenAI 兼容 API 调用模型，选手自行管理推理资源——商业 API（OpenAI、Claude 等）或自部署开源模型均可。
A: Yes. Agents call models via OpenAI-compatible APIs. Teams manage their own resources — commercial APIs (OpenAI, Claude, etc.) or self-hosted models are both fine.
A: 备赛期间需要自行搭建浏览器调试环境（如本地启动 Chrome 并通过 Playwright 连接 CDP）。正式评测时，官方会提供云端浏览器沙箱，CDP URL 自动传入你的 Agent，无需自行搭建。
A: During the preparation period, you need to set up your own browser debugging environment (e.g., launch Chrome locally and connect via Playwright + CDP). During formal evaluation, cloud browser sandboxes are provided by the organizers with CDP URLs passed directly to your Agent — no setup required on your end.
A: 参赛 Agent 通过 OpenAI 兼容 API 调用模型，分以下两种情况：
1. 商业闭源模型
各厂商允许使用的最高版本：
| 厂商 | 最高版本 |
|---|---|
| OpenAI | GPT-5.4 |
| Anthropic | Claude 4.6 |
| Gemini 3.1 | |
| xAI | Grok 4.3 |
| 智谱 AI | GLM-5V-Turbo |
| 月之暗面 | Kimi-K2.6 |
| 阿里千问 | Qwen 3.7 |
超出上述版本的禁止使用。未在上表列出的厂商，只允许使用 2026 年 7 月 16 日之前发布的模型（以厂商官方公开发布公告日期为准），之后发布的禁止使用。
2. 自研/自部署模型
允许使用自行训练微调/部署的模型，赛后需向组委会提交模型权重进行校验。
⚠️ 组委会将在评测过程中通过多种方式审查模型调用情况，严查以下行为：自研/自部署模型后台套壳其他高性能模型、闭源模型通过中转接口调用禁用版本。一经发现作弊行为，评测成绩视为无效。
A: Agents call models via OpenAI-compatible APIs. Rules differ by model type:
1. Commercial Closed-Source Models
Maximum allowed version per provider:
| Provider | Max Version |
|---|---|
| OpenAI | GPT-5.4 |
| Anthropic | Claude 4.6 |
| Gemini 3.1 | |
| xAI | Grok 4.3 |
| Zhipu AI | GLM-5V-Turbo |
| Moonshot | Kimi-K2.6 |
| Alibaba Qwen | Qwen 3.7 |
Versions above these limits are prohibited. For providers not listed above, only models released on or before July 16, 2026 are allowed (based on the provider's official public release announcement date); models released after that date are prohibited.
2. Self-Developed / Self-Deployed Models
Teams may use self-trained, fine-tuned, or self-deployed models. After the competition, teams must submit model weights to the organizing committee for verification.
⚠️ The organizing committee will conduct multi-faceted audits during evaluation, specifically targeting: self-deployed models secretly proxying other high-performance models, and closed-source models routing through intermediary APIs to access prohibited versions. Any violation will result in disqualification.
A: 最高 8 个任务并发；单次模型请求超时 3 分钟；每个任务最多 100 步；每个任务无重试，失败即 0 分。
A: Up to 8 concurrent tasks; 3-minute timeout per model request; max 100 steps per task; no retries — failure scores 0.
A: 不允许。Protocol III 考察从指定网页出发的导航与信息提取能力。赛后将进行操作轨迹验证，确认结果通过浏览器操作获取而非搜索引擎。
A: No. Protocol III evaluates navigation and extraction from a designated webpage. Post-competition trajectory verification ensures results were obtained via browser operations, not search engines.
A: 比赛模板是开源项目的精简版，保留核心 Agent 框架并预设了评测接口。你在开源项目上开发的 Agent 只需简单适配即可迁移到比赛模板中使用。
A: The template is a streamlined version of the open-source project with pre-configured evaluation interfaces. Agent logic developed on the open-source project can be directly migrated.
A: 选手提交代码仓库，评测系统自动拉取代码、安装依赖，通过标准入口脚本将任务文件、输出目录和 CDP URL 作为命令行参数传入运行。浏览器沙箱由官方提供并自动分配，无需额外认证。轨迹截图、操作记录等均由选手代码采集并写入输出目录，评测系统自动读取结果评分。
A: Teams submit their code repository. The evaluation system pulls the code, installs dependencies, and invokes your entry script with task file, output directory, and CDP URLs as command-line arguments. Browser sandboxes are provided and assigned automatically — no additional authentication is needed. Trajectory screenshots, action logs, and other artifacts are collected by your code and written to the output directory; the system reads the results for scoring.
A: 可以。只要满足评测接口要求——接受 CDP URL 和任务作为输入，按规定格式输出结果即可，框架不限。
A: Yes. As long as your Agent accepts CDP URLs and tasks as input and produces results in the required output format, any framework is allowed.
A: 没有数据合规或地域限制。可以自由调用任何模型 API（OpenAI、Claude、Gemini 等）。
A: No. There are no data compliance or regional restrictions. You are free to call any model API (OpenAI, Claude, Gemini, etc.).
A: 由参赛者的 Agent 代码在运行过程中采集并保存到输出目录。开源项目中有完整的实现供参考（截图保存、动作记录、XHR/Fetch 请求捕获等），评测系统会自动读取输出目录中的结果进行评分。
A: Your Agent code collects and saves these to the output directory during execution. The open-source project provides complete reference implementations (screenshot saving, action logging, XHR/Fetch capture, etc.). The evaluation system automatically reads the results from the output directory for scoring.
A: 每道任务输出到 {output_dir}/{task_idx}_{task_id}/，目录结构如下：
output/
├── locks/ # 多进程协调锁文件
├── 0_f0fe04a2.../
│ ├── trajectory/ # 每步原始截图（0.png, 1.png, ...）
│ ├── trajectory_visual/ # 带动作标注的截图
│ ├── result.json # 任务结果：状态、动作、思考、URL
│ └── capture.json # 捕获的 XHR/Fetch 网络请求
└── logs/
└── worker_0_YYYYMMDD.log # 每个 Worker 的日志文件
其中 result.json 的核心字段示例：
{
"task_idx": 0,
"task_id": "f0fe04a2...",
"task": "查看 iPad Air 3 的屏幕更换指南",
"website": "https://zh.ifixit.com/",
"status": "SUCCESS",
"actions": ["click(100, 200)", "type('iPad Air 3')"],
"thoughts": ["分析页面...", "找到目标..."],
"urls": ["https://...", "https://..."],
"agent_answer": "找到了屏幕更换指南，共12个步骤"
}
评分逻辑：每道任务的评判标准为：Agent 导航到目标页面，且提取的信息（agent_answer）语义正确即为通过——不要求与标准答案完全一致（例如标准答案为“2024年3月15日”，回答“3月15日”或“March 15, 2024”均视为正确）。最终得分为通过任务数占总任务数的比例。agent_answer 是评分核心字段，需要参赛者自行实现答案提取逻辑。
A: Each task outputs to {output_dir}/{task_idx}_{task_id}/ with this structure:
output/
├── locks/ # Multi-process coordination locks
├── 0_f0fe04a2.../
│ ├── trajectory/ # Raw screenshots at each step (0.png, 1.png, ...)
│ ├── trajectory_visual/ # Annotated screenshots with action overlays
│ ├── result.json # Task result: status, actions, thoughts, URLs
│ └── capture.json # Captured XHR/Fetch network requests
└── logs/
└── worker_0_YYYYMMDD.log # Per-worker log files
Key fields in result.json:
{
"task_idx": 0,
"task_id": "f0fe04a2...",
"task": "View the Screen Replacement guide for iPad Air 3",
"website": "https://zh.ifixit.com/",
"status": "SUCCESS",
"actions": ["click(100, 200)", "type('iPad Air 3')"],
"thoughts": ["Analyzing page...", "Found target..."],
"urls": ["https://...", "https://..."],
"agent_answer": "Found screen replacement guide with 12 steps"
}
Scoring logic: A task is considered passed when the Agent navigates to the target page and the extracted information (agent_answer) is semantically correct — an exact match with the ground truth is not required (e.g., if the ground truth is "March 15, 2024", answers like "3/15/2024" or "2024-03-15" are also accepted). The final score is the ratio of passed tasks to total tasks. agent_answer is the core scoring field — participants must implement their own answer extraction logic.