Szczegóły publikacji
Opis bibliograficzny
How to connect speech foundation models and Large Language Models? : what matters and what does not / Francesco Verdini, Pierfrancesco Melucci, Stefano Perna, Francesco Cariaggi, Marco Gaido, Sara Papi, Szymon MAZUREK, Marek KASZTELNIK, Luisa Bentivogli, Sébastien Bratières, Paolo Merialdo, Simone Scardapane // W: Interspeech 2025 [Dokument elektroniczny] : 17–21 August 2025, Rotterdam, The Netherlands. — Wersja do Windows. — Dane tekstowe. — [France : ISCA], [2025]. — ( Interspeech : proceedings of the ... Annual Conference of the International Speech Communication Association ; ISSN 2958-1796 ). — S. 1813–1817. — Wymagania systemowe: Adobe Reader. — Bibliogr. s. 1817, Abstr. — Sz. Mazurek - dod. afiliacja: ACC Cyfronet AGH
Autorzy (12)
- Verdini Francesco
- Melucci Pierfrancesco
- Perna Stefano
- Cariaggi Francesco
- Gaido Marco
- Papi Sara
- AGHMazurek Szymon
- AGHKasztelnik Marek
- Bentivogli Luisa
- Bratières Sébastien
- Merialdo Paolo
- Scardapane Simone
Słowa kluczowe
Dane bibliometryczne
| ID BaDAP | 162237 |
|---|---|
| Data dodania do BaDAP | 2025-09-10 |
| Tekst źródłowy | URL |
| DOI | 10.21437/Interspeech.2025-2245 |
| Rok publikacji | 2025 |
| Typ publikacji | materiały konferencyjne (aut.) |
| Otwarty dostęp |  |
| Konferencja | Interspeech 2025 |
| Czasopismo/seria | Interspeech |
Abstract
The remarkable performance achieved by Large Language Models (LLM) has driven research efforts to leverage them for a wide range of tasks and input modalities. In speechto-text (S2T) tasks, the emerging solution consists of projecting the output of the encoder of a Speech Foundational Model (SFM) into the LLM embedding space through an adapter module. However, no work has yet investigated how much the downstream-task performance depends on each component (SFM, adapter, LLM) nor whether the best design of the adapter depends on the chosen SFM and LLM. To fill this gap, we evaluate the combination of 5 adapter modules, 2 LLMs (Mistral and Llama), and 2 SFMs (Whisper and SeamlessM4T) on two widespread S2T tasks, namely Automatic Speech Recognition and Speech Translation. Our results demonstrate that the SFM plays a pivotal role in downstream performance, while the adapter choice has moderate impact and depends on the SFM and LLM.
