2026 · arXiv · Field expert review · confidence high

Mechanistic Interpretability of Brain-to-Speech Models Across Speech Modes

Maryam Maghsoudi, Ayushi Mishra

BibTeX

@misc{mechanistic-interpretability-of-brain-to-speech-models-across-speech-modes,
  title = {Mechanistic Interpretability of Brain-to-Speech Models Across Speech Modes},
  author = {Maryam Maghsoudi and Ayushi Mishra},
  year = {2026},
  note = {arXiv},
  eprint = {2602.01247},
  archivePrefix = {arXiv},
  url = {http://arxiv.org/abs/2602.01247v1},
}

Offers useful activation-intervention diagnostics, but donor replacement bypasses recipient information, baseline/patch scores are unresolved, and winner counts exceed the stated layer width; strong causal conclusions are not established.

Verdict: full-text draftPriority: highConfidence: highBasis: full text + summaryCoverage: high

Reading guidance

Verdict: full-text draft · priority high · confidence high
Why it matters: A candidate toolkit for auditing how a decoder responds to cross-mode activation changes, with useful matched-control and multi-scale intervention ideas.
What to trust: Basis: full text + summary. Coverage: high. 11 evidence records back the review.
What is weak: Whole-layer replacement removes the recipient path at that site. Smooth interpolation is expected from continuous computation and does not identify an intrinsic manifold. Neuron selection uses mean effects across the analyzed dataset. Convolutional winner counts and mode-specific model references conflict with the architectural description; model neurons are not biological neurons. One participant and one architecture; training details, fold counts, target alignment and held-out subspace selection are absent. Figure 5 uses a k=1 reference whereas the method describes an unpatched baseline. Full-patch scores are inconsistent with donor baselines under a shared serial model. Table 5 convolutional winner counts exceed the described 64 channels. Diagnostic interventions require paired donor-mode brain recordings or activations for the same content. No deployable imagined-only transfer method, new-user test, intelligibility assessment or real-time demonstration. One participant, one described model, paired speech-mode sentences and activation diagnostics. No biological intervention, intelligibility, open communication or clinical outcome. Overclaim risk: High: model activation dependence is presented as evidence of biological causal organization and adequate imagined inputs, while critical quantitative and protocol inconsistencies remain..
Read before: SSI review rubric
Read next: SSI archive

Axes

Task: speech-reconstruction
Modality: Invasive stereotactic EEG for vocalized, mimed and imagined speech; interventions additionally use paired donor-mode hidden activations.
Hardware: Existing stereotactic intracranial EEG electrodes in VOCALMind; acquisition channel counts and preprocessing are not detailed in this paper.
Body site: brain
Output: speech-audio
Vocabulary: paired sentence-level speech-mode tasks
Metrics: Table 2 baseline PCC concat: vocalized 0.7519, imagined 0.7254, mimed 0.5686; MCD 2.7928/2.8830/3.2948. Table 3 full vocalized-to-imagined/mimed patch PCC 0.954 and MCD 1.63; imagined-to-vocalized PCC 0.177. Table 1 imagined-from-vocalized KEEP-Conv/RAND-Conv PCC 0.666/0.564 and KEEP-RNN/RAND-RNN 0.462/0.398. Table 5 reports 96-120 distinct convolutional winners despite a 64-channel architecture. Author-reported values, internally unresolved and not independently reproduced.
Evaluation mode: Offline intervention analysis of a neural speech decoder, using flattened mel-spectrogram PCC, spectral distortion, baseline pitch measures, donor activation replacement, interpolation and matched random scrubbing controls.
Review confidence: high
Overclaim risk: High: model activation dependence is presented as evidence of biological causal organization and adequate imagined inputs, while critical quantitative and protocol inconsistencies remain.

Expert take

The useful idea is to ask which internal activations affect a brain-to-speech decoder, rather than treating prediction accuracy as an explanation. Channel/time interventions and matched random scrubbing controls are appropriate starting points. The strongest claims nevertheless exceed what the reported setup establishes. In Equation 1, replacing an entire activation tensor in the serial network causes the remaining network to operate on the donor representation; keeping the imagined input nominally fixed does not preserve its information downstream. High donor-conditioned performance therefore cannot rule out inferior imagined-signal quality or demonstrate successful imagined-only decoding. The numerical reporting also needs repair: the full vocalized patch reaches PCC 0.954, while the listed vocalized baseline is 0.752, even though the method describes the same fixed model; separate checkpoint language elsewhere makes the actual comparison unclear. More directly, a convolutional layer described as 64 channels is assigned 96 to 120 distinct winning neurons in Table 5. That cannot be reconciled with the stated channel-level definition without an unreported aggregation or different architecture. Claims of universally smooth monotonic interpolation are also inconsistent with nonmonotonic spectral-distortion curves, and interpolation through a continuous decoder is not sufficient evidence of a shared biological causal manifold. The KEEP-versus-random results suggest localized relative influence in some directions, but do not recover full donor performance and are not independently selected on held-out data. This paper is best treated as an exploratory interpretability proposal whose raw results, intervention semantics and counting conventions must be clarified before its mechanistic conclusions are used.

True value

A candidate toolkit for auditing how a decoder responds to cross-mode activation changes, with useful matched-control and multi-scale intervention ideas.

What changed

Canon before

Intervening on a neural network can identify dependencies in that network, but substituting donor activations is distinct from extracting more information from recipient brain signals. Smooth responses to interpolated activations do not by themselves establish a shared biological manifold.

Delta from canon

Studies where donor-mode activations influence a convolutional/bidirectional recurrent speech decoder through full replacement, temporal/channel subsets and neuron-level ranking.

Position in field

Offline interpretability analysis of an intracranial neural speech decoder, not a validated assistive communication system.

Evidence

“ The study combines full activation patching, convex interpolation, channel/time localization, matched random scrubbing and neuron ranking in a speech decoder. ”

actual_novelty · Sections 5-6; PDF pp. 3-8 · confidence 0.99

“ VOCALMind data come from one participant, and Table 5 reports 200 sentence samples per mode-pair condition; no cross-person experiment is described. ”

validation_scope · Section 4.1 and Table 5; PDF pp. 3 and 14 · confidence 0.99

“ Equation 1 computes the patched output as the post-layer network applied to donor activation. Reviewer assessment: in the stated serial architecture, complete replacement bypasses the recipient representation rather than improving information extraction from it. ”

limitation · Section 5, Equation 1; PDF p. 3 · confidence 0.99

“ Table 3 lists vocalized baseline PCC 0.752 but full vocalized donor patch PCC 0.954. Methods describe a fixed model, while Appendix E refers to activations from a vocalized model. Reviewer assessment: checkpoint and evaluation semantics need reconciliation. ”

limitation · Sections 5.4 and Appendix B.1/E; Table 3; PDF pp. 6, 11-13 · confidence 0.99

“ The convolutional output is described as 64 channels, but Table 5 lists 96-120 unique convolutional winner neurons. Reviewer assessment: these counts exceed the stated channel universe unless an unreported aggregation or different architecture is used. ”

limitation · Sections 4.2/5.2 and Table 5; PDF pp. 3-4 and 14, rendered table inspected · confidence 0.99

“ Table 1 imagined-from-vocalized KEEP-Conv/RAND-Conv PCC is 0.666/0.564, while KEEP-RNN/RAND-RNN is 0.462/0.398; full patch is 0.954. Reviewer assessment: localized control benefits are not full performance recovery. ”

metric · Table 1; PDF p. 7 · confidence 0.99

“ Figure 7 includes nonmonotonic recurrent MCD curves despite broad prose claims of monotonic interpolation. Reviewer assessment: smooth interpolation through a continuous decoder does not by itself establish a shared biological causal manifold. ”

limitation · Section 6.1, Appendix C and Figure 7; PDF pp. 7, 12-13, figure rendered · confidence 0.99

“ Top-k neurons are ranked by mean single-neuron effects across the analyzed dataset; a separate held-out selection/evaluation procedure is not specified. ”

limitation · Sections 5.5-5.6; PDF p. 6 · confidence 0.99

“ Figure 5 labels delta PCC relative to k=1, whereas methods describe changes relative to an unpatched baseline. Reviewer assessment: reference conditions must be specified consistently before interpreting improvements. ”

limitation · Sections 5.4-6 and Figure 5; PDF pp. 6-8 · confidence 0.99

“ Table 2 baseline PCC is 0.7519 for vocalized, 0.7254 for imagined and 0.5686 for mimed. Reviewer assessment: these baselines do not place mimed speech between the other two modes in decoding performance. ”

metric · Table 2 and Discussion; PDF pp. 8 and 11 · confidence 0.99

“ Evaluation centers on spectrogram correlation/distortion and intervention diagnostics; no word-recognition or listening-comprehension outcome, new participant or live communication test is reported. ”

validation_scope · Section 6 and Appendices A-F; PDF pp. 6-14 · confidence 0.99

Limits

Technical limits

Whole-layer replacement removes the recipient path at that site. Smooth interpolation is expected from continuous computation and does not identify an intrinsic manifold. Neuron selection uses mean effects across the analyzed dataset. Convolutional winner counts and mode-specific model references conflict with the architectural description; model neurons are not biological neurons.

Evaluation limits

One participant and one architecture; training details, fold counts, target alignment and held-out subspace selection are absent. Figure 5 uses a k=1 reference whereas the method describes an unpatched baseline. Full-patch scores are inconsistent with donor baselines under a shared serial model. Table 5 convolutional winner counts exceed the described 64 channels.

Deployment limits

Diagnostic interventions require paired donor-mode brain recordings or activations for the same content. No deployable imagined-only transfer method, new-user test, intelligibility assessment or real-time demonstration.

Scope limits

One participant, one described model, paired speech-mode sentences and activation diagnostics. No biological intervention, intelligibility, open communication or clinical outcome.