Szczegóły publikacji
Opis bibliograficzny
Monte Carlo winning tickets / Rafał GRZESZCZUK, Marcin KURDZIEL // W: Computational Science – ICCS 2021 : 21st international conference : Krakow, Poland, June 16–18, 2021 : proceedings, Pt. 2 / eds. Maciej Paszyński, [et al.]. — Cham : Springer Nature Switzerland, cop. 2021. — (Lecture Notes in Computer Science ; ISSN 0302-9743 ; LNCS 12743. Theoretical Computer Science and General Issues ; ISSN 0302-9743). — ISBN: 978-3-030-77963-4; e-ISBN: 978-3-030-77964-1. — S. 133–139. — Bibliogr., Abstr. — Publikacja dostępna online od: 2021-06-09
Autorzy (2)
Słowa kluczowe
Dane bibliometryczne
| ID BaDAP | 134717 |
|---|---|
| Data dodania do BaDAP | 2021-07-08 |
| DOI | 10.1007/978-3-030-77964-1_11 |
| Rok publikacji | 2021 |
| Typ publikacji | materiały konferencyjne (aut.) |
| Otwarty dostęp |  |
| Wydawca | Springer |
| Konferencja | 21st International Conference on Computational Science |
| Czasopisma/serie | Lecture Notes in Computer Science, Theoretical Computer Science and General Issues |
Abstract
Recent research on sparse neural networks demonstrates that densely-connected models contain sparse subnetworks that are trainable from a random initialization. Existence of these so called winning tickets suggests that we may possibly forego extensive training-and-pruning procedures, and train sparse neural networks from scratch. Unfortunately, winning tickets are data-derived models. That is, while they can be trained from scratch, their architecture is discovered via iterative pruning. In this work we propose Monte Carlo Winning Tickets (MCTWs) – random, sparse neural architectures that resemble winning tickets with respect to certain statistics over weights and activations. We show that MCTWs can match performance of standard winning tickets. This opens a route to constructing random but trainable sparse neural networks.