2026 · arXiv · Field expert review · confidence high

Lip-Siri: Contactless Open-Sentence Silent Speech with Wi-Fi Backscatter

Ye Tian, Haohua Du, Chao Gu, Junyang Zhang, Shanyue Wang, Hao Zhou, Jiahui Hou, Xiang-Yang Li

BibTeX

@misc{lip-siri-contactless-open-sentence-silent-speech-with-wi-fi-backscatter,
  title = {Lip-Siri: Contactless Open-Sentence Silent Speech with Wi-Fi Backscatter},
  author = {Ye Tian and Haohua Du and Chao Gu and Junyang Zhang and Shanyue Wang and Hao Zhou and Jiahui Hou and Xiang-Yang Li},
  year = {2026},
  note = {arXiv},
  eprint = {2601.18177},
  archivePrefix = {arXiv},
  url = {http://arxiv.org/abs/2601.18177v1},
}

A credible contactless Wi-Fi backscatter SSI prototype with 85.61% word accuracy and 36.87% sentence WER, but lexicon-composed sentences are not unseen-word recognition and deployment still requires SDR hardware plus manual boundaries.

Verdict: full-text draftPriority: highConfidence: highBasis: full text + summaryCoverage: high

Reading guidance

Verdict: full-text draft · priority high · confidence high
Why it matters: Demonstrates a non-wearable near-field RF sensing architecture and a practical path from lip-motion traces to lexicon-constrained sentence output, with explicit held-out-user and scenario tests.
What to trust: Basis: full text + summary. Coverage: high. 12 evidence records back the review.
What is weak: The tag isolates a shifted component but does not remove all non-target paths; contaminated windows are discarded and the rejection rate is not reported. The main cohort uses random sample splitting, leaving possible sentence or repetition overlap unspecified. Lexicon composition is not evidence of unseen-word recognition. Manual start/end clicks prevent a hands-free streaming interpretation of the reported latency. Jumping substantially worsens RF WER, while tag geometry and whole-system power are unmeasured. The 12-user experiment uses a random 80/20 sample split without an explicit sentence-disjoint protocol, while only three users are held out as new users. The lexicon is initialized from a fixed English handbook and its final size is not reported; adding words is only supported when they can be composed from existing subwords. Rejection rate from contamination gating, confidence intervals, repeated-seed variation, ablations, exact beam width, optimizer settings, and full power are not reported. The demonstrated system is an SDR prototype using a USRP-RIO transmitter, USRP-2943R receiver, Ethernet-connected PC, and a backscatter tag within 50 cm. Users manually click start and end buttons, the tag is fixed on a bracket, and a complete commercial-Wi-Fi implementation is explicitly outside scope. Violent motion can substantially degrade RF recognition, and tag-facing geometry is not characterized beyond the tested scenarios. Fifteen-volunteer SDR prototype evaluation with 340 handbook sentences, 3,398 reported word occurrences, three held-out users, and selected indoor/motion scenarios. No clinical participants, unrestricted vocabulary, sentence-disjoint split for the main cohort, adversarial privacy analysis, or continuous hands-free use. Overclaim risk: High if lexicon-composed sentences are described as unseen-word or unrestricted open-vocabulary recognition, if a 717.5 ms end-button latency is described as hands-free streaming, or if passive-tag power/cost is generalized to the entire SDR-plus-PC system..
Read before: SSI review rubric
Read next: SSI archive

Axes

Task: speech-recognition
Modality: Contactless Wi-Fi frequency-shifted backscatter sensing of lip-motion traces; SDR transmitter/receiver prototype.
Hardware: Frequency-shifted passive backscatter tag on a bracket or nearby object, connected antenna, USRP-RIO transmitter, USRP-2943R receiver, LabVIEW control, Ethernet-connected PC with Intel Core i5-11500 and 32 GB RAM; a camera records lip motion for visual comparison.
Body site: face; lip
Output: text
Vocabulary: Lexicon-guided subword sequence decoding; composable sentence combinations within a scenario lexicon.
Metrics: Overall average word prediction accuracy 85.61% and continuous sentence WER 36.87%; representative visual SSI WER 32.3% on recorded videos. Three new users range from 81.5% to 85.5% word accuracy and 36.3% to 39.6% WER. The silent-assistant 100-word lexicon case reports 92% accuracy and 90.5% under 50 dB audio interference. Reported latency is 717.5 ms from end-button click to displayed result for a 10-second signal, consisting of no more than 50 ms transmission, 525 ms processing, and 142.5 ms inference. Values are author-reported and not independently reproduced.
Evaluation mode: Offline word accuracy and continuous-sentence WER on a random 80/20 split plus three held-out users; sentence-length, signal-strength, time-interval, office/library/bedroom, silent-assistant, and eight-condition motion/lighting/mask comparisons; visual-SSI comparison from recorded video; end-button-to-result latency measurement.
Review confidence: high
Overclaim risk: High if lexicon-composed sentences are described as unseen-word or unrestricted open-vocabulary recognition, if a 717.5 ms end-button latency is described as hands-free streaming, or if passive-tag power/cost is generalized to the entire SDR-plus-PC system.

Expert take

Lip-Siri is a useful contactless RF prototype: it places a frequency-shifted backscatter tag on a nearby object rather than on the user's face, then turns filtered lip-motion traces into variable-length subword sequences. The reported base experiment uses 15 volunteers, 340 sentences and 3,398 word occurrences, with three users held out; average word accuracy is 85.61% and sentence WER is 36.87%. Those results support feasibility, not a general open-vocabulary claim. The lexicon is built from a handbook and new words are usable only when composed from existing subwords; the paper does not test words outside that lexicon or provide a sentence-disjoint split for the other users. The SDR prototype also depends on a tag within 50 cm, a PC, and manual start/end clicks. Its 717.5 ms figure is measured from the end click to the displayed result for a 10-second signal, not end-to-end hands-free streaming latency. Figure 13 shows useful tolerance to moderate conditions but a large RF WER under jumping and materially higher WER with a mask. Whole-system power, rejection coverage, exact lexicon/beam settings, and component ablations remain unresolved.

True value

Demonstrates a non-wearable near-field RF sensing architecture and a practical path from lip-motion traces to lexicon-constrained sentence output, with explicit held-out-user and scenario tests.

What changed

Canon before

Sensing-based silent speech interfaces commonly use wearable or held sensors, or dedicated radar, and usually classify a fixed list of words or sentences. Wireless backscatter lip sensing has also generally required a tag near the face.

Delta from canon

Moves the sensing tag from the user's body or chin to a nearby stand or monitor and decodes continuous lip-motion traces into variable-length subword sequences constrained by an extensible lexicon.

Position in field

A contactless Wi-Fi backscatter sentence-decoding prototype that extends RF silent-speech work beyond fixed command sets, while remaining an SDR-based laboratory system with lexicon and split limitations.

Evidence

“ The abstract presents Lip-Siri as a Wi-Fi backscatter SSI supporting open-vocabulary sentence recognition through lexicon-guided subword decoding, with a frequency-shifted tag and Transformer encoder-decoder. ”

author_claim · Abstract; PDF p. 1 · confidence 0.99

“ The prototype uses a frequency-shifted passive backscatter tag placed on a stand or monitor, allowing the user to face the tag without attaching or holding it near the chin. ”

fact · Section V-A1; PDF pp. 5-6 · confidence 0.99

“ The system extracts a first-order shifted backscatter component, computes phase differences, applies a 0–50 Hz low-pass filter, rejects windows above median plus three MAD, and uses VMD to suppress slower modes. ”

fact · Section V-A2; PDF p. 6 · confidence 0.99

“ Lip-motion units are approximately segmented with 0.2 s, 5 s, and 0.1 s windows; 67-dimensional time/frequency/differential features are standardized and clustered with K-means. The signal-processing description separately cites VMD with K=4 as its implementation example. ”

fact · Sections V-B1–V-B2 and Algorithm 1; PDF pp. 6-7 · confidence 0.99

“ The lexicon starts from characters in a scenario corpus and repeatedly merges frequent adjacent symbols; the decoder predicts variable-length subword sequences with a 12-layer, 12-head encoder, 6-layer, 4-head decoder, and beam search. ”

fact · Sections V-C1–V-C2; PDF pp. 7-8 · confidence 0.99

“ The paper defines open-sentence recognition as variable-length decoding over a lexicon and states that adding a word does not require retraining only when it is composable from existing subwords. This does not test recognition of words outside the lexicon or unrestricted open-vocabulary speech. ”

limitation · Section V-C2 and Table I caption; PDF pp. 8 and 11 · confidence 0.99

“ The evaluation recruits 15 volunteers over six months, uses 340 handbook sentences and 3,398 reported words, asks users to repeat base sentences at least twice, and collects 11,700 base samples. Three users are reserved as new users; the remaining data use a random 80/20 training/testing split. ”

validation_scope · Sections VI-A–VI-B, Data Collection and evaluation setup; PDF pp. 8–9 · confidence 0.99

“ The reported overall result is 85.61% average word accuracy and 36.87% average continuous-sentence WER; a recorded-video visual SSI comparison reports 32.3% WER. ”

metric · Section VI-B; Fig. 9; PDF p. 9 · confidence 0.99

“ For three new users, average word accuracy ranges from 81.5% to 85.5% and sentence WER ranges from 36.3% to 39.6%. ”

metric · Section VI-F; Fig. 11; PDF pp. 9–10 · confidence 0.99

“ The paper does not specify sentence-disjoint grouping for the random 80/20 split, unique vocabulary size, final subword lexicon size, beam width, optimizer, epochs, contamination rejection rate, confidence intervals, or component ablations. ”

limitation · Sections VI-A, V-B, and V-C; PDF pp. 6-9 · confidence 0.99

“ The interface requires a user to click start before lip-reading and end afterward. For a 10-second signal, the paper reports no more than 50 ms transmission, 525 ms signal processing, 142.5 ms inference, and 717.5 ms total latency from end click to displayed result. ”

metric · Section VI-G; PDF p. 10 · confidence 0.99

“ In eight scenario comparisons with two volunteers, RF WER is 35.8% in normal conditions, 38.5% with head rotation, 83.5% while jumping, 33.4% while typing, 39.0% while walking, and 46.5% with a mask; the prototype uses SDR hardware and the paper does not measure whole-system power. ”

limitation · Sections VIII-A–VIII-B; Fig. 13; PDF p. 11 · confidence 0.99

Limits

Technical limits

The tag isolates a shifted component but does not remove all non-target paths; contaminated windows are discarded and the rejection rate is not reported. The main cohort uses random sample splitting, leaving possible sentence or repetition overlap unspecified. Lexicon composition is not evidence of unseen-word recognition. Manual start/end clicks prevent a hands-free streaming interpretation of the reported latency. Jumping substantially worsens RF WER, while tag geometry and whole-system power are unmeasured.

Evaluation limits

The 12-user experiment uses a random 80/20 sample split without an explicit sentence-disjoint protocol, while only three users are held out as new users. The lexicon is initialized from a fixed English handbook and its final size is not reported; adding words is only supported when they can be composed from existing subwords. Rejection rate from contamination gating, confidence intervals, repeated-seed variation, ablations, exact beam width, optimizer settings, and full power are not reported.

Deployment limits

The demonstrated system is an SDR prototype using a USRP-RIO transmitter, USRP-2943R receiver, Ethernet-connected PC, and a backscatter tag within 50 cm. Users manually click start and end buttons, the tag is fixed on a bracket, and a complete commercial-Wi-Fi implementation is explicitly outside scope. Violent motion can substantially degrade RF recognition, and tag-facing geometry is not characterized beyond the tested scenarios.

Scope limits

Fifteen-volunteer SDR prototype evaluation with 340 handbook sentences, 3,398 reported word occurrences, three held-out users, and selected indoor/motion scenarios. No clinical participants, unrestricted vocabulary, sentence-disjoint split for the main cohort, adversarial privacy analysis, or continuous hands-free use.