title: Suggested Neural Network for small, highly varying dataset?
body: <p>I am currently working with a small dataset of training values, no more than 20, and am getting large MSE.  The input data vectors themselves consist of 16 parameters, many of which are binary variables.  Across all the training values, a majority of the 16 parameters stay the same (but <strong>not all</strong>).  The remaining input variables, across all the exemplars, vary a lot amongst one another.  This is to say, two exemplars might appear to be the same except for two parameters in which they differ, one parameter being a binary variable, and another being a continuous variable, where the difference could be greater than a single standard deviation (for that variable's set of values).</p>
  
  <p>My single output variable (as of now) can either be a continuous variable, OR depending on the true difficulty of reducing the error in my situation, I can make this a classification problem instead, with 12 different forms for classification.</p>
  
  <p>I have long been researching different neural networks than my current implementation of a feed-forward MLP, as I have read into Stochastic NNs, Ladder NNs, and many forms of recurrent NNs.  I am stuck with which one I should investigate, as I do not have time to try every NN available.</p>
  
  <p>While my description may be vague, could anyone make a suggestion as to which network I should investigate to minimize my cost function (as of now, MSE) the most?</p>
  
  <p>If my current setup must be rendered implacable because of the sheer difficulty involved with predicting correct output for such a small set of highly variant training values, which network would best work, should my dataset be expanded to the order of thousands of exemplars (at the cost of having a significantly more redundant, seemingly homogenous set of input values)? </p>
  
  <p>Any help is most certainly appreciated.</p>