Szczegóły publikacji
Opis bibliograficzny
A Bayesian nonparametrics view into deep representations / Michał JAMROŻ, Marcin KURDZIEL, Mateusz Opala // W: NeurIPS 2020 [Dokument elektroniczny] : 34th conference on Neural Information Processing Systems : December 6–12, 2020, Vancouver, Canada / ed. by H. Larochelle, [et al.]. — Wersja do Windows. — Dane tekstowe. — La Jolla : Neural Information Processing Systems, cop. 2020. — (Advances in Neural Information Processing Systems ; ISSN 1049-5258 ; vol. 33). — e-ISBN: 978-1-7138-2954-6. — S. 1440–1450. — Tryb dostępu: https://proceedings.neurips.cc/paper/2020/file/0ffaca95e3e524... [2021-04-30]. — Bibliogr., Abstr. — Toż.: http://www.proceedings.com/59066.html [2021-07-07]. — Dostęp do pełnego tekstu po zalogowaniu
Autorzy (3)
Dane bibliometryczne
| ID BaDAP | 133969 |
|---|---|
| Data dodania do BaDAP | 2021-07-02 |
| Rok publikacji | 2020 |
| Typ publikacji | materiały konferencyjne (aut.) |
| Otwarty dostęp |  |
| Konferencja | Advances in Neural Information Processing Systems [NeurIPS] |
| Czasopismo/seria | Advances in Neural Information Processing Systems |
Abstract
We investigate neural network representations from a probabilistic perspective. Specifically, we leverage Bayesian nonparametrics to construct models of neural activations in Convolutional Neural Networks (CNNs) and latent representations in Variational Autoencoders (VAEs). This allows us to formulate a tractable complexity measure for distributions of neural activations and to explore global structure of latent spaces learned by VAEs. We use this machinery to uncover how memorization and two common forms of regularization, i.e. dropout and input augmentation, influence representational complexity in CNNs. We demonstrate that networks that can exploit patterns in data learn vastly less complex representations than networks forced to memorize. We also show marked differences between effects of input augmentation and dropout, with the latter strongly depending on network width. Next, we investigate latent representations learned by standard fi -VAEs and Maximum Mean Discrepancy (MMD) fi-VAEs. We show that aggregated posterior in standard VAEs quickly collapses to the diagonal prior when regularization strength increases. MMD-VAEs, on the other hand, learn more complex posterior distributions, even with strong regularization. While this gives a richer sample space, MMD-VAEs do not exhibit independence of latent dimensions. Finally, we leverage our probabilistic models as an effective sampling strategy for latent codes, improving quality of samples in VAEs with rich posteriors.