nikkostrom  |  NICO  |  Quite BASIC

Nikko Ström (1997): "Sparse Connection and Pruning in Large Dynamic Artificial Neural Networks," Proc. EUROSPEECH '97, pp. 2807-2810, Rhodes, Greece.

Sparse Connection and Pruning in Large Dynamic Artificial Neural Networks

Nikko Ström

Abstract -- This paper presents new methods for training large neural networks for phoneme probabilty estimation. A combination of of the time-delay architecture and the recurrent network architecture is used to capture the important dynamic information in the speech signal. Motivated by the fact that the number of connections in a fully connected recurrent network grows super-linear with the number of hidden units, schemes for sparse connection and connection pruning are explored. It is found that sparsely connected networks outperform their fully connected counterparts with equal or smaller number of connections. The networks are evaluated in a hybrid HMM/ANN system for phoneme recognition on the TIMIT database. The achieved phoneme error rate, 28.3%, for the standard 39 phoneme set on the core test-set of the TIMIT database is not far from the lowest reported. All training and simulation software used is made freely available by the author, making reproduction of the results feasible.