Adapting neural network
This is roughly an open problem subject to ongoing research with various different strategies and heuristics known. A key word is "neural network architecture". The most basic strategy is to iterate through various network topologies and retrain for each one. Another strategy is to start with a relatively larger network and prune connections that have low weight and retrain and look for improvements; years ago this was called "brain damage" in at least one paper [2]. Here are some example references. There are many others. There is also a possibility of using GA-like algorithms to determine the network structure [3]. Here is part of the abstract from [1]: 


  The use of ANNs requires some critical decisions on the part of the user, which may affect the accuracy of the resulting classification. In this study, determination of the optimum network structure, which is one of the most important attributes of a network, is investigated. The structure of the network has a direct effect on training time and classification accuracy. Although there is some discussion in the literature of the impact of network structure on the performance of the network, there is no certain method or approach to determine the best structure. Investigations of the relationship between the network structure and the accuracy of the classification are reported here, using a MATLAB tool-kit to take the advantage of scientific visualisation. 


[1] Determining Optimum Structure for Artificial Neural Networks by Taskin Kavzoglu

[2] Optimal brain damage Le Cun, Denker, Solla

[3] Finding Optimal Neural Network Architecture Using Genetic Algorithms Britos et al