Hangeul Input System Employing Minimal Sensor based Gesture Recognition

Jaehyeon Park Virtual Environments Lab
Speaker

Jaehyeon Park
| Virtual Environments Lab

Abstract

This paper proposes a novel method for text input using human gestures, departing from conventional input systems. We adopt an approach that utilizes Inertial Measurement Units (IMU) to track and recognize human gestures, replacing traditional input devices such as mice and keyboards. Considering the complexity of the standard Korean keyboard layout (Dubeolsik), this study employs a Cheonjiin keyboard-based method to enhance efficiency, defining 10 unique pattern gestures.For gesture recognition in the input system, we apply feature extraction using Dynamic Time Warping (DTW) and sliding window algorithms, and utilize an ensemble learning approach combining Support Vector Machine (SVM) and Light Gradient Boosting Machine (LGBM). The trained model processes real-time sensor data through Python and socket communication, with Unity receiving the prediction data for text input. Additionally, we employ the SMPL-X model to provide an intuitive visualization of the users gesture inputs.

List