ArtBoost: Synthetic Articulatory Data Augmentation for Acoustic-to-Articulatory Inversion

Speaker

Hyungkyu Kim
| Immersive Reality and Integrated Systems Lab

Abstract

Recent AAI models rely on costly and limited EMA data. We propose ArtBoost, which extracts pseudo articulatory trajectories from visible facial anchors in large-scale speech-mesh datasets for pre-training, followed by fine-tuning on real EMA data. Experiments show consistent PCC and RMSE improvements across different AAI models, demonstrating that speech-mesh data can provide scalable articulatory supervision for AAI.

Hyungkyu Kim received the B.S. degree from Konkuk University, Seoul, South Korea, in 2023, and the M.S. degree from Chung-Ang University, Seoul, South Korea, in 2025. He is currently pursuing the Ph.D. degree at the Graduate School of Advanced Imaging Science, Multimedia & Film (GSAIM), Chung-Ang University, Seoul, South Korea. His research interests include speech signal processing, multimodal learning, and 3D generative models.

List
  • 중앙대학교 첨단영상대학원
  • BK21 FOUR 중앙대학교 인공지능-콘텐츠 미래산업 교육연구단
  • NRF 한국연구재단
  • 중앙대학교 영상콘텐츠융합연구소
  • 김영삼도서관
  • 동작문화재단