← CHI 2026 map

CHI '26 · Honorable mention · full-paper review · confidence medium-high

Designing Scaffolding Cards to Facilitate LLM-Based Socratic Instruction: An Exploratory Study of Response Strategies to Support Learning

Lujin Mao , Linyuan Dong , Wenan Li , Xiangen Hu , Kun-Pyo Lee , Zhibin Zhou

DOI PDF Program page Page JSON

This is a well-scoped CHI paper that turns a familiar LLM-learning problem into a more interesting interaction design question: how to scaffold the learner’s response, not just the model’s output. The contribution is strongest as a design framework plus an exploratory evaluation, with clear limits around personalization and longitudinal evidence.

Axes Lens

Rare contribution shape, typical evidence profile. The point here is not a score. It is to show what kind of claim the paper makes, and whether the evidence pattern is unusual or baseline in this 268 -review set.

Review Summary

This paper’s strongest contribution is conceptual: it challenges the default LLM-learning pattern of asking for answers and instead positions the model as a Socratic partner that asks questions while the learner is scaffolded to respond. That is a sensible but nontrivial shift, because it reframes the design problem from prompt quality to learner agency and response construction. The 18 Scaffolding Cards are the concrete artifact that operationalizes this idea, and the paper’s language about state-aware, agency-preserving, and function-transparent support gives the work a coherent design lens rather than a mere list of tips. The validation is appropriate for an exploratory CHI contribution: a formative study informs the artifact, and a mixed-methods evaluation with 34 participants provides evidence of improved critical thinking, cognitive load allocation, and learning satisfaction. At the same time, the paper is careful enough to acknowledge that the cards are fixed, not personalized, and not tested for long-term fading or repeated-session effects. That matters because the most compelling deployment story would require adaptation to learner state and progression over time. So the paper is best read as a strong early-stage design and empirical contribution: it offers a useful framework and an artifact with promising evidence, but it does not yet establish broad generalizability or durable classroom impact. The award level is consistent with that profile: notable and well-executed, but still exploratory rather than field-settling.

What Changed

Canon before

Prior CHI work on LLM-supported learning often emphasizes answer generation, prompting, or generic scaffolding; this paper shifts the focus to Socratic instruction and response-strategy scaffolds that help learners construct answers rather than receive them.

Departure from common sense

The paper’s core move is to avoid treating the LLM as an answer engine and instead use it to pose questions while scaffolding the learner’s response construction. That is a meaningful departure from the default “ask for the answer” interaction pattern in LLM use.

Actual novelty

The paper’s novelty is the combination of 18 Scaffolding Cards with a framing of scaffolding as state-aware, agency-preserving, and function-transparent support for LLM-based Socratic instruction. The contribution is not just a card set, but a design reconfiguration of how support is delivered in learner-LLM dialogue.

Evidence

The paper presents a formative study with N=20 to derive 18 Scaffolding Cards from dialogue logs and interviews, then evaluates them in a mixed-methods study with N=34. The abstract reports improvements in critical thinking, cognitive load allocation, and learning satisfaction relative to a no-scaffold condition, while the limitations note fixed cards, no personalization, and open questions about fading and familiarity effects.

Limits

Method limits

The evidence base is exploratory rather than definitive: the formative phase is small (N=20), and the evaluation is a mixed-methods study with N=34. The design also uses a fixed card set, so the study does not test adaptive recommendation or personalization.

Deployment limits

The cards are not yet personalized or recommended dynamically, so deployment in real educational settings would need adaptation to learner state, domain, and progression. The paper also does not establish long-term effectiveness across repeated sessions.

Boundary conditions

The reported benefits are bounded by the study’s participant pool, the SPL system context, and the specific Socratic interaction format. The paper itself flags uncertainty about how learner familiarity stage affects outcomes and whether scaffolding should fade over time.

Manual check: No manual check needed for schema completeness; the main caution is that the evidence spans provided are broad and the review should be interpreted as grounded in the supplied packet only.

Position in field

This sits at the intersection of LLM-based learning support, Socratic tutoring, and scaffolding design. Its main field contribution is to reframe scaffolding from generic help toward response-strategy support that preserves learner agency in LLM-mediated instruction.

Abstract

The overreliance on large language models (LLMs)-generated answers poses risks to the development of learners’ critical thinking. Socratic instruction, which follows “tutor asks, student answers” approach, could mitigate overreliance by engaging learners with LLM-generated questions rather than passively seeking answers from LLMs. However, learners without effective response strategies often produce superficial answers and therefore undermine Socratic instruction. To bridge the gap, we first conducted a formative study (N=20) to analyze learners’ dialogue logs and interviews, deriving 18 Scaffolding Cards as response strategies to guide learners in framing their answers. A subsequent mixed-methods study (N=34) demonstrated that Scaffolding Cards improved critical thinking, optimized cognitive load allocation, and increased learning satisfaction compared to that without scaffolds. Our work reconfigures scaffolding by incorporating state-aware, agency-preserving, and function-transparent support. We further provide actionable implications for designing responsive and personalized scaffolding to facilitate learner-LLM interaction, introducing innovative perspectives for reclaiming learner agency in LLM-driven education.