Self-organizing networks such as Neural Gas, Growing Neural Gas and many others have been adopted in actual applications for both dimensionality reduction and manifold learning. Typically, the goal in these applications is obtaining a good estimate of the topology of a completely unknown subspace that can be explored only through an unordered sample of input data points. In the approach presented here, the dimension of the input manifold is assumed to be known in advance. This prior assumption can be harnessed in the design of a new, growing self-organizing network that can adapt itself in a way that, under specific conditions, will guarantee the effective and stable recovery of the exact topological structure of the input manifold.
@cite_14: The ability to grow extra nodes is a potentially useful facility for a self-organising neural network. A network that can add nodes into its map space can approximate the input space more accurately, and often more parsimoniously, than a network with predefined structure and size, such as the Self-Organising Map. In addition, a growing network can deal with dynamic input distributions. Most of the growing networks that have been proposed in the literature add new nodes to support the node that has accumulated the highest error during previous iterations or to support topological structures. This usually means that new nodes are added only when the number of iterations is an integer multiple of some pre-defined constant, λ.This paper suggests a way in which the learning algorithm can add nodes whenever the network in its current state does not sufficiently match the input. In this way the network grows very quickly when new data is presented, but stops growing once the network has matched the data. This is particularly important when we consider dynamic data sets, where the distribution of inputs can change to a new regime after some time.We also demonstrate the preservation of neighbourhood relations in the data by the network. The new network is compared to an existing growing network, the Growing Neural Gas (GNG), on a artificial dataset, showing how the network deals with a change in input distribution after some time. Finally, the new network is applied to several novelty detection tasks and is compared with both the GNG and an unsupervised form of the Reduced Coulomb Energy network on a robotic inspection task and with a Support Vector Machine on two benchmark novelty detection tasks.
In the Grow-When-Required (GWR) algorithm @cite_14 , which is a development of GNG, the error variable associated to each GNG unit is replaced by a firing counter @math that decreases exponentially each time the unit is winner, i.e. closest to the input signal. When @math gets below a certain threshold @math , the unit is deemed and its behavior changes. The adaptation of unit being closest to the input takes place only if the distance is below a certain threshold @math , otherwise a new unit is created. This means that the network continues to grow until the input data sample is completely included in the union of balls of radius @math centered in each unit.