Using the GA R package to optimize the weights of a MLP neural network The neural network I am trying to evolve uses the tanh as an activation function in each neuron and has a topology of 1-5-1, so I need at least 5 weights. The solution of the GA is a real-number vector of length 5, which represents the weights of the network and each weight should take values between -5 to 5. I wrote an R function to use as a fitness function which returns the mean squared error (MSE) of the output data in comparison to the desired output. I want it to learn the cubic function. The input data I am using is

input&lt;-seq(-1,1,0.02)


and the output data is its cubic function

des_out&lt;-input^3


The evaluation function is the following

evalnn&lt;-function(x){

mse&lt;-0
for(i in 1:length(input)){

     nn.out &lt;- tanh(x[1]*input[i]) + tanh(x[2]*input[i]) + tanh(x[3]*input[i]) + tanh(x[4]*input[i]) + tanh(x[5]*input[i]) 
     mse &lt;- (des.out[i] - nn.out)^2 + mse

}
return(-(mse)/length(input))
}


I have set the return value to negative, because I want the smallest value to be thought as the best fit.

gann&lt;-ga(type="real-valued", fitness=evalnn,min=c(-5,-5,-5,-5,-5),max=c(5,5,5,5,5),popSize=100,maxiter=150,pmutation=0.01,pcrossover=0.8)


What I always get back from the GA are weights that make up a linear function although I have been experimenting with the ga's parameters quite a lot, i.e. I have tried all crossover methods and mutations. https://cran.r-project.org/web/packages/GA/GA.pdf.

The cubic function plotted together with my output function.

I have used a linear function as training data before the cubic with this implentation and worked. It has trouble with the non-linear.

If anybody could figure out why do I get this, is there something I have missed. 
Thank you
I finally got the results I was looking for.

Apparently a single hidden layer neural network cannot learn the cubic function. Instead of just evolving the weights, I included the topology of the neural network into the chromosome. My neural network can have maximum 5 hidden layers with maximum 5 neurons in each layer and the chromosome contains information for the structure of the network and the selection of the weights. 

When I ran the GA with population of 200 and for 100 iterations I got a full topology of:

1(input) - 4-2-3-2-2 - 1(output), a mean squared error of 0.0051 and the following plot

I would be glad if anyone has any comments or questions.