Szczegóły publikacji
Opis bibliograficzny
Training neural networks on high-dimensional data using random projection / Piotr Iwo WÓJCIK, Marcin KURDZIEL // Pattern Analysis and Applications ; ISSN 1433-7541. — 2019 — vol. 22 iss. 3, s. 1221–1231. — Bibliogr. s. 1230–1231, Abstr. — Publikacja dostępna online od: 2018-03-19
Autorzy (2)
Słowa kluczowe
Dane bibliometryczne
| ID BaDAP | 122861 |
|---|---|
| Data dodania do BaDAP | 2019-07-18 |
| Tekst źródłowy | URL |
| DOI | 10.1007/s10044-018-0697-0 |
| Rok publikacji | 2019 |
| Typ publikacji | artykuł w czasopiśmie |
| Otwarty dostęp |  |
| Creative Commons | |
| Czasopismo/seria | Pattern Analysis and Applications |
Abstract
Training deep neural networks (DNNs) on high-dimensional data with no spatial structure poses a major computational problem. It implies a network architecture with a huge input layer, which greatly increases the number of weights, often making the training infeasible. One solution to this problem is to reduce the dimensionality of the input space to a manageable size, and then train a deep network on a representation with fewer dimensions. Here, we focus on performing the dimensionality reduction step by randomly projecting the input data into a lower-dimensional space. Conceptually, this is equivalent to adding a random projection (RP) layer in front of the network. We study two variants of RP layers: one where the weights are fixed, and one where they are fine-tuned during network training. We evaluate the performance of DNNs with input layers constructed using several recently proposed RP schemes. These include: Gaussian, Achlioptas’, Li’s, subsampled randomized Hadamard transform (SRHT) and Count Sketch-based constructions. Our results demonstrate that DNNs with RP layer achieve competitive performance on high-dimensional real-world datasets. In particular, we show that SRHT and Count Sketch-based projections provide the best balance between the projection time and the network performance.