I've been interested in NNs for a while, just started playing with them. I liked the look of Keras, so I got started with some toycode to do some regression.

I tried the simplest set up I could:

* $500$ inputs drawn uniformly from an interval $0<x<2\*\pi$
* Targets generated by $3\times sin(x)+1+e$ for $e\sim \mathcal{N}(0, 0.5)$ (Sin curve with random error).
* Neural network therefore has one input ($x$) and one output ($y$).
* One fully connected hidden layer with three neurons, activation function of $tanh$, loss function of mean squared error, training
algorithm of stochastic gradient descent.

See self contained [code gist](https://gist.github.com/Ogaday/165f8d442877f3754cdb).

The only problem is that it seems no matter what I try, I can't fit the values of $x$ for $pi<x<2\*\pi$. I've tried different loss functions, Nesterov momentum, more epochs, more layers, more neurons, different learning rates, decay, momentum, different optimizers. Some made the fit slightly better, many changes made it worse.

Am I making a syntactical mistake somewhere that's affecting my results or is there a better way to build a simple MLP for this kind of problem? This is the first time I'm using Theano and Keras, so I don't know whether a fundamental mistake is plaguing me or if I need a new approach. I can't find any examples for regression using the Keras library.