title: Continuous Perceptron Weights Change Zero
body: <p>I am working on 2 class simple perceptron problem. My project work getting user mouse click from GUI panel and make classification. Class 1 expected output: 1 and Class 2 expected output -1. My problem is discrete perceptron working fine but continuous perceptron after one point stop decrease error. I don't know what I am doing wrong. I look so much code and source.</p>
  <p>My formulas;</p>
  <ul>
  <li>E=1/2 Σ(d-o)^2</li>
  <li>f(net)=(2/(1+ⅇ^(-net)))-1</li>
  <li>ΔW=n(d-o)(1-o^2)y</li>
  </ul>
  <p>like this.</p>
  <p><code>d</code>: Expected output,</p>
  <p><code>net</code>: weight*input sum,</p>
  <p><code>y</code>: input matrix ([x1 x2 -1]) and</p>
  <p><code>o</code>: Actual output.</p>
  <p>Code for continuous perceptron below;</p>
  <pre><code>while (totalError &gt; Emax)
  {
     totalError = 0;
     for(i=0; i&lt;point.Count; i++)
     {
         double x1 = point[i].X1;
         double x2 = point[i].X2;
         double net = (x1 * w0) + (x2 * w1) + (x0 * w2);
         double o = (2 / (1 + Math.Exp(-net))) - 1;
         double error = Math.Pow(point[i].Class - o, 2);
         w0 += (x1 * c * (point[i].Class - o) * (1 - Math.Pow(o, 2))) / 2;
         w1 += (x2 * c * (point[i].Class - o) * (1 - Math.Pow(o, 2))) / 2;
         w2 += (x0 * c * (point[i].Class - o) * (1 - Math.Pow(o, 2))) / 2;
         totalError += error;
     }
     totalError = totalError / 2;
     ErrorShow(cycle, totalError);
     objGraphic.Clear(Color.White);
     DrawSeperationLine();
     cycle++;
  }
  </code></pre>
  <p>Emax=0.001 selected. Project working like this. You can see it not correct line location. Class 1 is blue and class 2 red.</p>
  <p>I think problem in for loop.</p>
  <p><a href="https://i.stack.imgur.com/sts3T.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/sts3T.png" alt="Project working like this. You can see it not correct shape." /></a></p>
  <p>Console Output of Code:</p>
  <p><a href="https://i.stack.imgur.com/snXOm.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/snXOm.png" alt="Console Output" /></a></p>
  <p><strong>Edit:</strong>
  After discuss with @TaW (Thanks for showing road), I find out my problem in output (activation function). It always return 1 or -1. After that in weight change function [<code>1-Math.Pow(o,2)</code>] part return 0 and that make weight change equal 0. So my question how can I solve this problem. Type casting not work.</p>