Szczegóły publikacji
Opis bibliograficzny
RGB and geometry-aware representations of mouth articulations for sign language understanding / Nurzhigit ONGALOV, Bogdan KWOLEK // W: Artificial Neural Networks and machine learning - ICANN 2026 : 35th International Conference on Artificial Neural Networks : Padua, Italy, September 14–17, 2026 : proceedings , Pt. 4 / eds. Luca Pasa, [et al.]. — Cham : Springer Nature, cop. 2027. — ( Lecture Notes in Computer Science ; ISSN 0302-9743 ; 17092 ). — ISBN: 978-3-032-38406-5; e-ISBN: 978-3-032-38407-2. — S. 760–772. — Bibliogr., Abstr. — Publikacja dostępna online od: 2026-09-14
Autorzy (2)
Słowa kluczowe
Dane bibliometryczne
| ID BaDAP | 170131 |
|---|---|
| Data dodania do BaDAP | 2026-09-29 |
| DOI | 10.1007/978-3-032-38407-2_60 |
| Rok publikacji | 2027 |
| Typ publikacji | materiały konferencyjne (aut.) |
| Otwarty dostęp |  |
| Wydawca | Springer |
| Konferencja | International Conference on Artificial Neural Networks 2026 |
| Czasopismo/seria | Lecture Notes in Computer Science |
Abstract
Mouth articulations convey essential grammatical and semantic information in sign languages, yet most recognition systems remain focused on manual signals while neglecting non-manual cues such as mouthing. We evaluate continuous vowel articulation recognition across three modalities: classical handcrafted geometric descriptors, Spatio-Temporal Graph Convolutional Networks (ST-GCNs) operating on lip keypoint streams, and deep networks processing raw RGB frames. We propose HS-MAGCN, a hierarchical multi-branch anatomical graph convolutional network that decomposes lip landmarks into four anatomically motivated regions, fusing spatio-temporal graph representations with handcrafted descriptors via learnable gating. Evaluated under a strict cross-subject protocol on a continuous Japanese Sign Language dataset, HS-MAGCN achieves 70.4% accuracy and 69.6% macro F1, outperforming all RGB-based baselines while relying solely on geometric keypoints and thereby preserving signer privacy.
