title: Implemention of Neural Networks
body: <p>I'm trying to understand how to implement neural networks. So I made my own dataset. Xtrain is numpy.random floats. Ytrain is sign(sin(1/x^3).
  Try to implement neural networks gave me very poor results. 30%accuracy. Random Forest with 100 trees give 97%. But I heard that NN can approximate any function. What is wrong in my understanding?</p>
  
  <pre><code>import numpy as np
  import keras
  import math
  from sklearn.ensemble import RandomForestClassifier as RF
  train = np.random.rand(100000)
  test = np.random.rand(100000)
  def g(x):
      if math.sin(2*3.14*x) &gt; 0:
          if math.cos(2*3.14*x) &gt; 0:
              return 0
          else:
              return 1 
      else:
          if math.cos(2*3.14*x) &gt; 0:
              return 2
          else:
             return 3
  def f(x):
      x = (1/x) ** 3
      res = [0, 0, 0, 0]
      res[g(x)] = 1
      return res
  
  ytrain = np.array([f(x)  for x in train])
  ytest = np.array([f(x) for x in test])
  train = np.array([[x] for x in train])
  test = np.array([[x] for x in test])
  
  from keras.models import Sequential
  from keras.layers import Dense, Activation, Embedding, LSTM
  
  model = Sequential()
  model.add(Dense(100, input_dim=1))
  model.add(Activation('sigmoid'))
  model.add(Dense(100))
  model.add(Activation('sigmoid'))
  model.add(Dense(100))
  model.add(Activation('sigmoid'))
  model.add(Dense(4))
  model.add(Activation('softmax'))
  model.compile(optimizer='sgd',
            loss='categorical_crossentropy',
            metrics=['accuracy'])
  </code></pre>
  
  <p>P.S. I tried out many layers, activation functions, loss functions, optimizers, but never got more than 30% accuracy :(</p>