Shortening the development time of a neural network I am developing an LSTM for sequence tagging. 
During the development, I do various changes in the system, for example, add new features, change the number of nodes in the hidden layers, etc. 
After each change, I check the accuracy using cross-validation on a development corpus. 

Currently, in each check, I use 100 iterations to train the system, which takes a lot of time. So I thought that maybe, during development, I can use only e.g. 20 iterations. Then, each check will be faster. After I find the best configuration, I can switch back to 100 iterations to get better accuracy.

My question is: is this consideration correct? I.e, if feature-set A is better than feature-set B with 20 training iterations, is it likely that A will be better than B also with 100 training iterations?

Alternatively, is there a better way to speed up the development process?
Your scenario is common.

The most straightforward approach is to subsample your data randomly. Unless your data or your model has strong bias, your performance to the smaller data set should be comparable. The accuracy might be lower, but the purpose is to do quick sanity check.