title: Sigmoid function for large numbers in Python
body: <p>Im doing a Neural network task with <code>Sigmoid</code> activation function. My networks input is image(<code>MNIST</code> dataset) and because dimension of each image is <code>28*28</code>, When I have to convert them to a vector, i will have <code>N*784</code> matrix. Multiplication of this large matrix with weight matrix produce large positive and negative numbers for weights and i have to pass them to a <code>Sigmoid</code> function. I use <code>expit()</code> as sigmoid function and my problem is that:</p>
  
  <p>numbers until 30 resulting near 1 in <code>expit()</code>. for example <code>expit(28)</code> results <code>0.99999999</code> and <code>expit(29)</code> results <code>1.0</code> and upper <code>29</code> also gets <code>1</code>. But my new wights are upper <code>30</code> and because of that some of them gets <code>1</code> and some 0 in the first cycle of learning and in fact there is no any learning at all. </p>
  
  <p>What i have to do? <code>Sigmoid's</code> upper bound is <code>29</code>? i cant change it? I have to change my image dimension to overcome this? </p>