Szczegóły publikacji
Opis bibliograficzny
Grouped multi-layer echo state networks with self-normalizing activations / Robert Wcisło, Wojciech CZECH // W: Computational Science – ICCS 2021 : 21st international conference : Krakow, Poland, June 16–18, 2021 : proceedings, Pt. 1 / eds. Maciej Paszyński, [et al.]. — Cham : Springer Nature Switzerland, cop. 2021. — (Lecture Notes in Computer Science ; ISSN 0302-9743 ; LNCS 12742. Theoretical Computer Science and General Issues ; ISSN 0302-9743). — ISBN: 978-3-030-77960-3; e-ISBN: 978-3-030-77961-0. — S. 90–97. — Bibliogr., Abstr. — Publikacja dostępna online od: 2021-06-09
Autorzy (2)
Słowa kluczowe
Dane bibliometryczne
| ID BaDAP | 134697 |
|---|---|
| Data dodania do BaDAP | 2021-06-23 |
| DOI | 10.1007/978-3-030-77961-0_9 |
| Rok publikacji | 2021 |
| Typ publikacji | materiały konferencyjne (aut.) |
| Otwarty dostęp |  |
| Wydawca | Springer |
| Konferencja | 21st International Conference on Computational Science |
| Czasopisma/serie | Lecture Notes in Computer Science, Theoretical Computer Science and General Issues |
Abstract
We study prediction performance of Echo State Networks with multiple reservoirs built based on stacking and grouping. Grouping allows for developing independent subreservoir dynamics, which improves linear separability on readout layer. At the same time, stacking enables to capture multiple time-scales of an input signal by the hierarchy of non-linear mappings. Combining those two effects, together with a proper selection of model hyperparameters can boost ESN capabilities for benchmark time-series such as Mackey Glass System. Different strategies for determining subreservoir structure are compared along with the influence of activation function. In particular, we show that recently proposed non-linear self-normalizing activation function together with grouped deep reservoirs provide superior prediction performance on artificial and real-world datasets. Moreover, comparing to standard tangent hyperbolic models, the new models built using self-normalizing activation function are more feasible in terms of hyperparameter selection.