Szczegóły publikacji
Opis bibliograficzny
Neural representations reveal distinct modes of class fitting in residual convolutional networks / Michał JAMROŻ, Marcin KURDZIEL // W: AAAI-23 [Dokument elektroniczny] : thirty-seventh AAAI conference on Artificial intelligence ; thirty-fifth conference on Innovative applications of artificial intelligence ; thirteenth symposium on Educational advances in artificial intelligence : February 7-14, 2023, Washington DC, USA / ed. by Brian Williams, Yiling Chen, Jennifer Neville. — Wersja do Windows. — Dane tekstowe. — Washington : Association for the Advancement of Artificial Intelligence, cop. 2023. — (Proceedings of the ... AAAI Conference on Artificial Intelligence ; ISSN 2159-5399 ; Vol. 37). — e-ISBN: 978-1-57735-880-0. — S. 7988-7995. — Wymagania systemowe: Adobe Reader. — Tryb dostępu: https://ojs.aaai.org/index.php/AAAI/article/view/25966/25738 [2023-06-30]. — Bibliogr. s. 7995, Abstr. — Publikacja dostępna online od: 2023-06-26. --- Opublikowane w części: AAAI-23 Technical Tracks 7
Autorzy (2)
Słowa kluczowe
Dane bibliometryczne
| ID BaDAP | 147548 |
|---|---|
| Data dodania do BaDAP | 2023-07-26 |
| DOI | 10.1609/aaai.v37i7.25966 |
| Rok publikacji | 2023 |
| Typ publikacji | materiały konferencyjne (aut.) |
| Otwarty dostęp |  |
| Konferencje | Thirty-Seventh AAAI Conference on Artificial Intelligence, Thirty-Fifth Conference on Innovative Applications of Artificial Intelligence |
| Czasopismo/seria | Proceedings of the ... AAAI Conference on Artificial Intelligence |
Abstract
We leverage probabilistic models of neural representations to investigate how residual networks fit classes. To this end, we estimate class-conditional density models for representations learned by deep ResNets. We then use these models to characterize distributions of representations across learned classes. Surprisingly, we find that classes in the investigated models are not fitted in a uniform way. On the contrary: we uncover two groups of classes that are fitted with markedly different distributions of representations. These distinct modes of class-fitting are evident only in the deeper layers of the investigated models, indicating that they are not related to low-level image features. We show that the uncovered structure in neural representations correlate with memorization of training examples and adversarial robustness. Finally, we compare class-conditional distributions of neural representations between memorized and typical examples. This allows us to uncover where in the network structure class labels arise for memorized and standard inputs.