← CHI 2026 map

CHI '26 · Honorable mention · full-paper review · confidence medium-high

"Here, Let Me Help": An Empirical Study of User Interventions in Human–Web Agent Collaboration

Joohee Kim , Sungbeom Cho , Duc M. Nguyen , Jaehyeong Jeon , Minjeong Shin , Sungahn Ko

DOI PDF Program page Page JSON

This is a solid empirical CHI paper with a clear contribution: it reframes web-agent evaluation around user interventions rather than only task success. The taxonomy is useful, the study design is credible, and the discussion connects the findings to design implications. Its main limits are the novice participant pool, the narrow domain set, and the coarse proxy used for implicit intervention.

Axes Lens

Rare contribution shape, typical evidence profile. The point here is not a score. It is to show what kind of claim the paper makes, and whether the evidence pattern is unusual or baseline in this 268 -review set.

Review Summary

This paper is best read as a process-level HCI contribution to web-agent research rather than as an agent-systems paper. The authors make a persuasive case that outcome-centric evaluation misses the actual collaborative work users do when agents struggle, and they back that claim with a controlled in-lab study on live websites. The study design is fairly strong for this kind of question: 30 participants, 12 tasks, three domains, within-subjects exposure, interaction logs, screen recordings, and post-task ratings. The results are not just descriptive; they identify a structured relationship between intervention reasons and intervention forms, including explicit versus implicit interventions, and they show that task structure and website implementation shape intervention behavior in different ways. That makes the taxonomy more than a naming exercise: it is tied to observed behavior and to design implications about prompt scaffolds, visible progress cues, structured interaction options, and proactive mixed-initiative support. The paper’s novelty is therefore empirical and conceptual rather than algorithmic. It does not propose a new web agent architecture, but it does provide a useful framework for understanding collaboration with imperfect agents in realistic browsing settings. The limitations are also well handled and should temper any overclaiming. The participant pool is young and mostly novice to web agents, so the findings are strongest for early adoption rather than mature usage. The domain set is intentionally limited to shopping, travel, and information-seeking, so the taxonomy may not transfer unchanged to higher-stakes settings like finance or healthcare. The operationalization of implicit intervention is necessarily approximate, because mouse movement and screen traces are used as proxies for intent. Even so, the paper is careful about these constraints and explicitly frames future work around broader populations, more domains, and richer behavioral signals. Overall, this is a credible and useful CHI honorable-mention paper: not groundbreaking in the sense of introducing a new technical system, but strong in its empirical grounding, its framing of human–agent collaboration, and its practical implications for designing web agents that support users as active collaborators rather than passive end users.

What Changed

Canon before

The paper sits in the CHI web-agent / human-AI collaboration line, but the supplied evidence shows it is not a generic autonomy benchmark paper. Its canon is closer to empirical HCI work that studies how people actually repair, steer, and monitor imperfect agents in situ, rather than treating success rate alone as the main object of analysis.

Departure from common sense

The work departs from the common-sense framing that a web agent should simply be judged by whether it finishes the task alone. Instead, it treats user interruption, correction, monitoring, and even manual termination as meaningful collaborative behaviors that reveal how people work around agent limitations in real time.

Actual novelty

The main novelty is an empirically grounded taxonomy of user interventions in human–web agent collaboration, paired with process-level evidence about why interventions happen and how they map to intervention forms. The paper also contributes a controlled in-lab study design on live websites that links intervention reasons, intervention types, and subjective usability outcomes.

Evidence

The full text provides substantive grounding for the review: the abstract states the study goal and contribution; the introduction frames the gap beyond outcome-centric evaluation; the method section specifies a controlled in-lab study with 30 participants, 12 tasks, 3 domains, and live websites; the results and discussion sections define intervention reasons, intervention types, and limitations. Together these support a conservative but real expert review rather than a metadata-only summary.

Limits

Method limits

The study is controlled and ecologically motivated, but it is still an in-lab design with 30 university-affiliated participants, mostly novice to web agents. The operationalization of implicit intervention is also coarse, relying on mouse movement and screen recordings as proxies for intent, which the authors explicitly note may affect the observed distribution of implicit intervention types.

Deployment limits

The findings are grounded in a generalist web agent on six pre-specified websites across shopping, travel, and information-seeking tasks. The paper itself cautions that behavior may differ with other agent architectures, stronger or weaker models, and real-world deployments outside the lab, especially where stakes, user expertise, or interface ecology differ substantially.

Boundary conditions

The results are bounded by early-stage adoption, novice web-agent users, and the selected task domains. The authors explicitly note that higher-stakes domains such as finance or healthcare may produce different intervention patterns, and that more experienced users may intervene earlier, tolerate delays less, and rely on different repair strategies.

Manual check: false

Position in field

This is a strong CHI-style empirical contribution in the human–web agent collaboration space: it shifts attention from outcome-only evaluation toward intervention dynamics, and it complements prior mixed-initiative systems by explaining when users step in, what they do, and why. Its value is less as a new agent architecture than as a behavioral and design framework for future collaborative web agents.

Abstract

Web agents aim to execute complex online tasks from high-level instructions, yet fully autonomous execution remains challenging in practice. We present an empirical study of user interventions in human–web agent collaboration, moving beyond outcome-based metrics to examine how interventions unfold during execution. We conducted a controlled in-lab study with 30 participants whose interactions reflected early-stage web agent adoption across 12 structured tasks in shopping, travel, and information-seeking domains using live websites. Analyzing interaction logs, user inputs, and screen recordings, we identify diverse behaviors and propose a taxonomy capturing both the reasons for intervention and the forms they take. We distinguish explicit interventions, where users halt or override actions, from implicit interventions, where users guide or prepare the environment without stopping execution. Our findings reveal how task structure and execution breakdowns shape intervention behaviors to provide process-level evidence for designing web agents that better support users as active collaborators.