Do optical, inertial, and muscle signals provide complementary information about a short hand gesture? This study compares their contributions when each gesture is recognized as one complete event.
We used the wrist-worn BioPoint device and a sequence model to evaluate individual modalities and every combination. The focus was not only baseline recognition, but whether fusion helped when the arm position or recording day changed.
The question
A single EMG channel samples a limited region of muscle activity. Acceleration and photoplethysmography (PPG) provide different observations of the same movement, potentially filling gaps in that limited spatial measurement.
The study asked how much each modality contributed and whether a sequence model could combine them effectively. Testing new positions and days then examined whether the combination remained useful when conditions changed.
The study at a glance
- Participants
- Twelve people, seven male and five female, aged 22–59.
- Sensors
- One EMG channel at 2,000 Hz, three-axis acceleration at 50 Hz, and PPG at 50 Hz.
- Gesture vocabulary
- Seven classes: rest, wrist flexion/extension, hand open/close, double tap, and finger snap.
- Conditions
- Twenty baseline repetitions, four additional limb positions, and another session three to seven days later.
How we approached it
Active gestures were segmented using changes in the accelerometer recording. Each resulting event was divided into 200 ms windows advanced every 100 ms. Wavelet features summarized each modality within those windows.
Each modality had its own long short-term memory network to model the sequence. Their final representations were then concatenated before the classification layers. This late-fusion design allows the temporal structure of each signal type to be learned before combining the evidence.
Technical detail: the evaluation
Baseline accuracy used leave-one-repetition-out evaluation within each participant. The position and day tests used conditions absent from the baseline training recordings. These are user-dependent models, not zero-shot cross-user recognizers.
All modality comparisons used gesture segments derived from acceleration. Thus even an EMG-only classification result in this experiment does not describe a complete EMG-only gesture-detection pipeline.
What we found
The combination of all sensors achieved over 96% baseline accuracy and significantly outperformed each individual sensor. It also significantly outperformed the individual modalities in the unseen-position analysis.
Across days, however, full fusion significantly exceeded EMG alone and PPG alone but did not significantly exceed acceleration alone. The reported effect size for that last comparison was 0.09. This qualification matters: the results support complementary sensing, but not a uniform fusion advantage over every comparator in every condition.
Acceleration was itself a strong modality for this gesture set. The EMG and PPG channels alone were less adequate for reliable recognition of the full vocabulary. Different gesture subsets could change that balance and were left as an avenue for further study.
Reading the figures
The traces illustrate different responses to the same short movements. Separate network branches process the modalities before their representations meet at the classification head.
Read this as one prediction for a complete gesture sequence. The model does not provide a simultaneous continuous estimate of hand posture, and the diagram does not demonstrate how it rejects unrelated daily activity.
The three panels show how the sensor comparison changes with the test condition: baseline, unseen positions, and another day. The critical-difference diagrams below each panel show which rankings are statistically distinguishable. In particular, the across-day result should not be described as a significant improvement over accelerometry alone.
What the results do not establish
The analysis used segmented prompted gestures and twelve participants. It did not assess an always-on device during unrestricted activity or quantify false activations outside the vocabulary.
The device’s compact form factor motivates the work, but comfort was not the primary measured outcome. The strong accelerometer contribution and its role in segmentation also limit conclusions about a deployment that removes inertial sensing.
Looking back
The paper’s useful result is the condition-specific comparison of complementary measurements. Keeping the across-day exception beside the baseline gain gives a more accurate account of what fusion accomplished.
A focused follow-up would evaluate segmentation and classification together on continuous recordings. Comparing false activations and missed gestures for each sensor combination would establish whether the recognition advantage translates into a more reliable event-driven interface.
Read further
Ethan Eddy, Evan Campbell, Ulysse Côté-Allard, Scott Bateman, and Erik Scheme. Discrete Gesture Recognition Using Multimodal PPG, IMU, and Single-Channel EMG Recorded at the Wrist. IEEE Sensors Letters 8(9), 5503604, August 21, 2024. DOI: 10.1109/LSENS.2024.3447240.
Figure 3 and Table 1 separate baseline, position, and day results. The methods explain accelerometer-based segmentation and late fusion; the results text specifies the nonsignificant cross-day comparison.