Szczegóły publikacji
Opis bibliograficzny
Ada-QPacknet - multi-task forget-free continual learning with quantization driven adaptive pruning / Marcin PIETROŃ, Dominik ŻUREK, Kamil FABER, Roberto Corizzo // W: ECAI 2023 : 26th European Conference on Artificial Intelligence : including 12th conference on Prestigious Applications of Intelligent Systems (PAIS 2023) : September 30 - October 4, 2023, Kraków, Poland : proceedings / ed. by Kobi Gal, [et al.] ; European Association for Artificial Intelligence (EurAI), Polish Artificial Intelligence Society (PSSI). — Amsterdam : IOS Press BV, cop. 2023. — (Frontiers in Artificial Intelligence and Applications ; ISSN 0922-6389 ; vol. 372). — ISBN: 978-1-64368-436-9; e-ISBN: 978-1-64368-437-6. — S. 1882-1889. — Bibliogr. s. 1888-1889, Abstr. — Dod. abstrakt dostępny w: https://ecai2023.eu/acceptedpapers [2023-11-06]
Autorzy (4)
- AGHPietroń Marcin
- AGHŻurek Dominik
- AGHFaber Kamil
- Corizzo Roberto
Dane bibliometryczne
| ID BaDAP | 149125 |
|---|---|
| Data dodania do BaDAP | 2023-11-06 |
| Tekst źródłowy | URL |
| DOI | 10.3233/FAIA230477 |
| Rok publikacji | 2023 |
| Typ publikacji | materiały konferencyjne (aut.) |
| Otwarty dostęp |  |
| Creative Commons | |
| Wydawca | IOS Press |
| Konferencja | European Conference on Artificial Intelligence 2023 |
| Czasopismo/seria | Frontiers in Artificial Intelligence and Applications |
Abstract
Continual learning (CL) is a challenging machine learning setting that is attracting the interest of an increasing number of researchers. Among recent CL works, architectural strategies appear particularly promising due to their potential to expand and adapt the model architecture as new tasks are presented. However, existing solutions do not efficiently exploit model sparsity due to the adoption of constant pruning ratios. Moreover, current approaches exhibit a tendency to quickly saturate model capacity since the number of weights is limited and each weight is restricted to a single value. In this paper, we propose Ada-QPacknet, a novel architectural CL method that resorts to adaptive pruning and quantization. These two features allow our model to overcome the two crucial issues of effective exploitation of model sparsity and efficient use of model capacity. Specifically, adaptive pruning restores model capacity by reducing the number of weights assigned to each task to a smaller subset of weights that preserves the performance of the full set, allowing other weights to be used for future tasks. Adaptive quantization separates each weight into multiple components with adaptively reduced bit-width, allowing a single weight to solve more than one task without significant performance drops, leading to improved exploitation of model capacity. Experimental results on benchmark CL scenarios show that our proposed method achieves better results in terms of accuracy than existing rehearsal, regularization, and architectural CL strategies. Moreover, our method significantly outperforms forget-free competitors in terms of efficient exploitation of model capacity.
