← CHI 2026 map

CHI '26 · Honorable mention · full-paper review · confidence medium-high

Privy: Envisioning and Mitigating Privacy Risks for Consumer-facing AI Product Concepts

Hao-Ping (Hank) Lee , Yu-Ju Yang , Matthew Bilik , Isadora Krsek , Thomas Serban von Davier , Kyzyl Monteiro , Jason Lin , Shivani Agarwal , Jodi Forlizzi , Sauvik Das

DOI PDF Program page Page JSON

Privy is a solid CHI artifact paper: it turns privacy impact assessment into a structured practitioner workflow and backs the claim with a controlled study plus expert review. The main contribution is practical and evaluative rather than theoretical, and the strongest evidence is for the specific concept-assessment setting.

Axes Lens

Rare contribution shape, typical evidence profile. The point here is not a score. It is to show what kind of claim the paper makes, and whether the evidence pattern is unusual or baseline in this 268 -review set.

Review Summary

Privy reads as a well-scoped CHI artifact contribution with a clear practitioner problem and a credible evaluation story. The paper’s central move is to treat privacy risk identification and mitigation for AI product concepts as a structured task that can be scaffolded for non-experts, rather than assuming practitioners will reliably improvise with generic guidance. That is a meaningful departure from common-sense expectations about privacy work, especially in consumer-facing AI settings where risks are often novel and hard to enumerate. The novelty is primarily in the tool itself: two versions of Privy, one LLM-powered and one template-based, both aimed at helping practitioners produce privacy impact assessments that are useful enough to be judged by independent experts. The validation is reasonably strong for a CHI paper of this type: a between-subjects controlled study with 24 practitioners and 13 expert reviewers gives the authors a concrete basis for claiming improved assessment quality, and the abstract explicitly says the LLM version amplified the effects. At the same time, the paper is careful enough to surface important limits. The system appears better at helping users envision risks than at generating mitigations, and the authors note that mitigation needs richer information. The study context also constrains generalization: researcher-provided concepts, time-limited tasks, and a practitioner sample that may not reflect broader organizational privacy practice. So the paper’s value is strongest as evidence that structured, AI-assisted scaffolding can improve privacy assessment quality in a bounded task class, not as proof that the approach solves privacy review in the wild.

What Changed

Canon before

Prior CHI work on privacy support has often emphasized guidelines, checklists, or expert-facing methods; this paper positions Privy as a practitioner-facing tool for structured privacy impact assessment in AI product concepts.

Departure from common sense

The paper argues that practitioners cannot rely on intuition or generic guidance alone for AI privacy work: they need a structured tool to surface risks and mitigations in novel product concepts. That framing departs from the common-sense assumption that privacy review is mainly a matter of expert judgment or compliance checklists.

Actual novelty

The paper’s novelty is a practitioner-facing tool, Privy, with two implementations—LLM-powered and template-based—that supports both identifying privacy risks and proposing mitigations for consumer-facing AI product concepts. The authors also claim that no prior work had produced LLM-powered systems for both tasks together.

Evidence

The paper presents Privy and evaluates it in a between-subjects controlled study with 24 practitioners, whose outputs were reviewed by 13 independent privacy experts. The abstract reports that experts deemed the resulting privacy assessments high quality, that practitioners identified relevant risks and mitigation strategies, and that the LLM version improved these effects. The paper also states limitations around mitigation being harder than risk envisioning and around the study context.

Limits

Method limits

The evaluation is a controlled study with 24 separate practitioners, so the evidence is strongest for the specific task setting and participant pool rather than broad real-world deployment. The paper also notes that mitigation was less effective than risk envisioning, suggesting the system’s support is uneven across subtasks.

Deployment limits

The paper’s own limitations indicate that richer information is needed for mitigation, and that the study used researcher-provided product concepts under time constraints. These conditions limit direct transfer to open-ended, high-stakes, or fully self-directed product development workflows.

Boundary conditions

Findings are bounded by the study design: consumer-facing AI product concepts, practitioners without privacy expertise, and review by independent privacy experts. The paper also notes limited generalizability beyond the sampled practitioner populations and the specific assessment setting.

Manual check: Verify that the quoted evidence spans in the source text correspond to the claims and that the limitation wording matches the paper’s exact section text.

Position in field

Privy sits at the intersection of privacy impact assessment support and AI-assisted practitioner tooling. Its contribution is less a new privacy theory than a concrete artifact and evaluation showing that structured, tool-mediated assessment can improve privacy reasoning for non-experts.

Abstract

AI creates and exacerbates privacy risks, yet practitioners lack effective resources to identify and mitigate these risks. We present Privy, a tool that guides practitioners without privacy expertise through structured privacy impact assessments to: (i) identify relevant risks in novel AI product concepts, and (ii) propose appropriate mitigations. Privy was shaped by a formative study with 11 practitioners, which informed two versions --- one LLM-powered, the other template-based. We evaluated these two versions of Privy through a between-subjects, controlled study with 24 separate practitioners, whose assessments were reviewed by 13 independent privacy experts. Results show that Privy helps practitioners produce privacy assessments that experts deemed high quality: practitioners identified relevant risks and proposed appropriate mitigation strategies. These effects were augmented in the LLM-powered version. Practitioners themselves rated Privy as being useful and usable, and their feedback illustrates how it helps overcome long-standing awareness, motivation, and ability barriers in privacy work.