Large Language Models in Peer-Run Community Behavioral Health Services: Understanding Peer Specialists and Service Users’ Perspectives on Opportunities, Risks, and Mitigation Strategies
This is a strong CHI honorable-mention paper because it moves LLM-in-care discourse away from generic clinical support and into peer-run behavioral health, where authority and trust are fundamentally relational. The contribution is less about a system and more about a careful, grounded reframing of what responsible LLM use could mean in community-led care.
Axes Lens
Rare contribution shape, typical evidence profile. The point here is not a score. It is to show what kind of claim the paper makes, and whether the evidence pattern is unusual or baseline in this 268 -review set.
Contribution shape
- Knowledge form
- normative knowledge typical · 31/268
- Novelty type
- framework typical · 59/268
- Abstraction level
- practice typical · 85/268
- Generalization target
- organizational context typical · 20/268
- Validation mode
- qualitative study typical · 63/268
Evidence profile
- Evidence strength
- moderate typical · 105/268
- Claim alignment
- strong typical · 231/268
- Overclaim risk
- medium typical · 210/268
Review Summary
This paper’s main strength is that it identifies a genuinely different problem space for LLMs in care: peer-run behavioral health services, where lived experience is not a supplement to expertise but the basis of the service model. That shift matters because it changes what counts as a good design question. Rather than asking whether an LLM can improve efficiency or accuracy in a clinical workflow, the authors ask how an LLM-based recommendation system might reshape in-room dynamics, relational authority, trust, and autonomy. That is a meaningful departure from common-sense assumptions that AI in care is primarily a decision-support or productivity tool. The paper’s most important novelty is the “lived-experience-in-the-loop” framing, which is best understood as a normative design principle for governance and collaboration rather than a technical mechanism. It extends human-in-the-loop thinking by insisting that experiential authority and co-constructed trust are central, not peripheral, to system design in this context. The validation is appropriately scoped: the study is formative, qualitative, and based on comicboarding workshops with 16 peer specialists and 10 service users at CSPNJ. That scope is well matched to the claim type, because the paper is not trying to prove efficacy of a deployed system; it is eliciting stakeholder perceptions, tensions, and mitigation strategies. The limitations are also clear and important: there was no hands-on interaction with a real LLM, and the method cannot establish feasibility, usability, or real-world harms. Overall, the paper is strongest as a conceptual and empirical foundation for future governance-sensitive design work in community-led care, and its contribution is credible precisely because it stays within the bounds of a formative qualitative study.
What Changed
Canon before
CHI work on LLMs in care often centers clinical decision support, usability, or safety; this paper instead frames peer-run behavioral health as a relational, community-led setting where authority and trust are co-produced.
Departure from common sense
The paper argues that LLMs in peer-run behavioral health should not be treated as neutral clinical tools or decision-makers; instead, their role depends on how they are introduced, constrained, and co-used, because they can alter relational authority and trust in the room.
Actual novelty
The paper’s novel contribution is a design principle and framing shift: “lived-experience-in-the-loop,” which extends beyond generic human-in-the-loop ideas by centering peer specialists’ and service users’ experiential authority, co-constructed trust, and relational collaboration in high-stakes community care.
Evidence
This is a formative qualitative co-design study with 16 peer specialists and 10 service users at a statewide peer-run organization. The paper uses comicboarding workshops to elicit opportunities, risks, and mitigation strategies for an LLM-based recommendation system, and the discussion reframes LLMs as relational collaborators rather than clinical tools. The evidence supports a normative design contribution and a bounded empirical finding about stakeholder perceptions, not system performance.
“ Findings show that depending on how LLMs are introduced, constrained, and co-used, they can reconfigure in-room dynamics by sustaining, undermining, or amplifying the relational authority that grounds peer support. We identify opp”
actual novelty · Abstract / Introduction / Discussion · confidence 0.98
“ Partnering with Collaborative Support Programs of New Jersey (CSPNJ), a statewide PRO in the Northeastern United States, we used comicboarding, a co-design method, to conduct workshops with 16 peer specialists and 10 service users exploring perceptions of integrating an LLM-based recommendation system into peer support. Findings show that d”
departure from common sense · Abstract · confidence 0.98
“ Google Scholar [115] Xuhai Xu, Bingsheng Yao, Yuanzhe Dong, Saadia Gabriel, Hong Yu, James Hendler, Marzyeh Ghassemi, Anind K Dey, and Dakuo Wang. 2024. Mental-llm: Leveraging large language models for mental health prediction via online text data”
limitation · 6 Limitations and Future Work · confidence 0.99
“ As large language models (LLMs) enter this domain, their scale, conversationality, and opacity introduce new challenges for situatedness, trust, and autonomy. Partnering with Collaborative Support Programs of New Jersey (CSPNJ), a statewide PRO in the Northeastern United States, we used comicboarding, a co-de”
validation scope · Abstract / Study Design · confidence 0.99
Limits
Method limits
The study is formative and qualitative, using comicboarding to support early design exploration rather than evaluating a deployed system. It does not measure effectiveness, usability, safety, or downstream outcomes of an actual LLM intervention.
Deployment limits
Findings are specific to a peer-run behavioral health organization and to the imagined integration of an LLM-based recommendation system. Transfer to other care settings, other organizational models, or fully deployed systems will require additional validation.
Boundary conditions
The contribution is bounded by a high-stakes, community-led, peer-run behavioral health context where relational authority, trust, and autonomy are central. The design implications depend on careful introduction, constraint, and co-use of LLMs alongside peer judgment.
Position in field
The paper extends CHI’s growing LLM-in-care literature by shifting from clinical augmentation toward peer-run, lived-experience-centered governance. Its main value is conceptual and normative: it reframes trust, authority, and collaboration in a setting where lived experience is the core expertise.