title: Theano scan for fast computations on an array
body: <p>I am trying to use Theano to speed up code that is already implemented in numpy that sums the elements in an array. In numpy, the function looks like below</p>
  
  <pre><code>import numpy as np
  
  def numpy_fn(k0, kN, x):
      output = np.zeros_like(x)
      for k in range(k0, kN+1):
          output += k*x
      return output
  </code></pre>
  
  <p>with a sample call</p>
  
  <pre><code>&gt;&gt;&gt; numpy_fn(1, 3, np.arange(10))
  array([  0.,   6.,  12.,  18.,  24.,  30.,  36.,  42.,  48.,  54.])
  </code></pre>
  
  <p>The theano equivalent of the above function is </p>
  
  <pre><code>import theano
  import theano.tensor as tt
  
  k  = tt.scalar('k')
  k0 = tt.scalar('k0')
  kN = tt.scalar('kN')
  x  = tt.vector('x')
  
  def fn(k, sumtodate):
      return sumtodate + k*x
  
  rslt, updt = theano.scan(fn=fn, 
                           outputs_info=tt.zeros_like(x),
                           sequences=tt.arange(k0, kN+1))
  theano_fn = theano.function(inputs=[k0, kN, x], 
                              outputs=rslt[-1])
  </code></pre>
  
  <p>When called, this gives the correct output</p>
  
  <pre><code>theano_fn(1, 3, np.arange(10))
  array([  0.,   6.,  12.,  18.,  24.,  30.,  36.,  42.,  48.,  54.])
  </code></pre>
  
  <p>However, when I benchmark the two, the numpy function surpasses theano in speed by a factor of three on my computer. </p>
  
  <pre><code>%timeit theano_fn(1, 1000, np.ones(10000))
  10 loops, best of 3: 21.5 ms per loop
  
  %timeit numpy_fn(1, 1000, np.ones(10000))
  100 loops, best of 3: 7.9 ms per loop
  </code></pre>
  
  <p>Since theano converts the outerloop into C, should it not be faster than Python? What can be done to speed up the theano code?</p>
  
  <p><strong>EDIT:</strong></p>
  
  <p>I am aware that the brute code in numpy can be optimized using summations, but the reason why I wanted to take the theano route was because I am interested in cases where the update to output can be any general function of <code>k</code> and <code>x</code>, say </p>
  
  <pre><code>output += x**k
  output += exp(k*x)
  output += (x-k)**2
  </code></pre>
  
  <p><code>output += k*x</code> was just one specific example to make the point. Using mathematical notation what I am trying to implement is a fast summation <code>\sum_{k=k0}^{kN} f(k, x)</code>, where <code>k0</code> and <code>kN</code> are integers, and <code>x</code> is a vector, and <code>f</code> can be any general function of <code>k</code> and <code>x</code> like the ones given above.</p>
  
  <pre><code>import numpy as np
  
  def f(k, x):
      return x**k
  
  def numpy_fn(k0, kN, x):
      output = np.zeros_like(x)
      for k in range(k0, kN+1):
          output += f(k, x)
      return output
  </code></pre>
  
  <p>I was hoping that by using theano, I would be able to optimize the outter loop, and would get a faster solution than the brute numpy solution.</p>