Correlation threshold for Neural Network features selection
Given non-linearity of neural networks, I believe correlation analysis isn't a good way to estimate importance of variables. For example, imagine that you have 2 input variables - x1 and x2 - and following conditions hold: 


cor(x2, y) = 1 if x1 = 1
cor(x2, y) = 0 otherwise
x1 = 1 in 10% of cases


That is, x2 is a very good predictor for y, but only given that x1 = 1, which is the case only in 10% of data. Taking into account correlations of x1 and x2 separately won't expose this dependency, and you will most likely drop out both variables. 

There are other ways to perform feature selection, however. Simplest one is to train your model with all possible sets of variables and check the best subset. This is pretty inefficient with many variables, though, so many ways to improve it exist. For a good introduction in best subset selection see chapter 6.1 of Introduction to Statistical Learning.