title: Multi-Layer Perceptrons in EmguCV
body: <p>I'm trying to implement Multi-Layer Perceptrons (MLP) neural networks using EmguCV 3.1 (a dot NET wrapper for OpenCV library) in C#(Windows Form). In order to practice with this library I decide to implement <code>OR</code> operation using MLP. </p>
  
  <p>I create MLP using "Initialize" method and learn it using "Train" method as below:</p>
  
  <pre><code>private void Initialize()
  {
      NETWORK.SetActivationFunction(
      ANN_MLP.AnnMlpActivationFunction.SigmoidSym);
  
      NETWORK.SetTrainMethod(ANN_MLP.AnnMlpTrainMethod.Backprop);
  
      Matrix&lt;double&gt; layers = new Matrix&lt;double&gt;(new Size(4, 1));
      layers[0, 0] = 2;
      layers[0, 1] = 2;
      layers[0, 2] = 2;
      layers[0, 3] = 1;
      NETWORK.SetLayerSizes(layers);
  }
  
  private void Train()
  {
      // providing data for input
  
      Matrix&lt;float&gt; input = new Matrix&lt;float&gt;(4, 2);
      input[0, 0] = MIN_ACTIVATION_FUNCTION; input[0, 1] = MIN_ACTIVATION_FUNCTION;
      input[1, 0] = MIN_ACTIVATION_FUNCTION; input[1, 1] = MAX_ACTIVATION_FUNCTION;
      input[2, 0] = MAX_ACTIVATION_FUNCTION; input[2, 1] = MIN_ACTIVATION_FUNCTION;
      input[3, 0] = MAX_ACTIVATION_FUNCTION; input[3, 1] = MAX_ACTIVATION_FUNCTION;
  
      //providing data for output
      Matrix&lt;float&gt; output = new Matrix&lt;float&gt;(4, 1);
      output[0, 0] = MIN_ACTIVATION_FUNCTION;
      output[1, 0] = MAX_ACTIVATION_FUNCTION;
      output[2, 0] = MAX_ACTIVATION_FUNCTION;
      output[3, 0] = MAX_ACTIVATION_FUNCTION;
  
  
      // mixing input and output for training
      TrainData mixedData = new TrainData(
          input,
          Emgu.CV.ML.MlEnum.DataLayoutType.RowSample,
          output);
  
      // stop condition = 1 million iterations
      NETWORK.TermCriteria = new MCvTermCriteria(1000000);
  
      // training
      NETWORK.Train(mixedData);
  }
  </code></pre>
  
  <p>Where <code>MIN_ACTIVATION_FUNCTION</code>, and <code>MAX_ACTIVATION_FUNCTION</code> are equal to -1.7159 and 1.7159, respectively (<a href="http://docs.opencv.org/3.1.0/dd/ded/group__ml.html#gade71470ec8814021728f8b31b09773b0" rel="nofollow noreferrer">according to OpenCV Documentation</a>). After 1000000 iterations of (as you see in my code in stop condition), I test my network for prediction using Predict method as below:</p>
  
  <pre><code>private void Predict()
  {
      Matrix&lt;float&gt; input = new Matrix&lt;float&gt;(1, 2);
      input[0, 0] = MIN_ACTIVATION_FUNCTION;
      input[0, 1] = MIN_ACTIVATION_FUNCTION;
  
      Matrix&lt;float&gt; output = new Matrix&lt;float&gt;(1, 1);
  
      NETWORK.Predict(input, output);
      MessageBox.Show(output[0, 0].ToString());
  
      //////////////////////////////////////////////
  
      input[0, 0] = MIN_ACTIVATION_FUNCTION;
      input[0, 1] = MAX_ACTIVATION_FUNCTION;
  
      NETWORK.Predict(input, output);
      MessageBox.Show(output[0, 0].ToString());
  
      //////////////////////////////////////////////
  
      input[0, 0] = MAX_ACTIVATION_FUNCTION;
      input[0, 1] = MIN_ACTIVATION_FUNCTION;
  
      NETWORK.Predict(input, output);
      MessageBox.Show(output[0, 0].ToString());
  
      ////////////////////////////////////////////////
  
      input[0, 0] = MAX_ACTIVATION_FUNCTION;
      input[0, 1] = MAX_ACTIVATION_FUNCTION;
  
      NETWORK.Predict(input, output);
      MessageBox.Show(output[0, 0].ToString());
  }
  </code></pre>
  
  <p>Here is a sample of what NETWORK predicts:<br/>
  -0.00734469<br/>
  -0.03184918<br/>
  0.02080269<br/>
  -0.006674092<br/><br/></p>
  
  <p>I expect be some thing like this:<br/>
  -1.7<br/>
  +1.7<br/>
  +1.7<br/>
  +1.7<br/>
  <br/><br/>
  What is wrong among my codes?</p>
  
  <p>Note that I also use 0, 1 for <code>MIN_ACTIVATION_FUNCTION</code> and <code>MAX_ACTIVATION_FUNCTION</code> values but I still do not any good results.</p>
  
  <p>Update 1:
  I edit my codes as first answer refers me (even I test my code with idea referenced in comments). Now I get <code>NaN</code> when call <code>predict</code> method.</p>