title: Sigmoid function returns 1 for large positive inputs
body: <p>I wrote the following function in Python to calculate sigmoid function of a scalar, vector or matrix.</p>
  
  <pre><code>def sigmoid(z):
      sig = 1.0/(1.0 + np.exp(-z))
      return sig
  </code></pre>
  
  <p>For relatively large positive values of z, <code>e^-z</code> returns a very small value close to zero (0) and hence the value of sig is rounded to 1. My final objective is to determine cost function of logistic regression algorithm. Since sigmoid returns exactly 1, <code>log(1-1)</code> return 'nan'. How can I solve the problem in such a way that my sigmoid() function will return the proper value and not round the <code>e^-z</code> to 0?</p>
  
  <p>When I normalized the input features using mean and standard deviation, it worked fine. But is there a way to make it work with larger values of z?</p>
  
  <p>Also, I tried the same on Matlab and it worked fine without normalization.</p>
  
  <pre><code>&gt;&gt;&gt; Z = np.array([[60, 100],[20, 80]])
  &gt;&gt;&gt; Z
  array([[ 60, 100],
         [ 20,  80]])
  &gt;&gt;&gt; np.exp(-Z)
  array([[8.75651076e-27, 3.72007598e-44],
         [2.06115362e-09, 1.80485139e-35]])
  &gt;&gt;&gt; 1.0/(1.0 + np.exp(-Z))
  array([[1., 1.],
         [1., 1.]])
  </code></pre>