title: PyMC: Parameter estimation in a Markov system
body: <h1>A Simple Markov Chain</h1>
  <p>Let's say we want to estimate parameters of a system such that we can predict the state of the system at time step t+1 given the state at timestep t. PyMC should be able to deal with this easily.</p>
  <p>Let our toy system consist of a moving object in a 1D world. The state is the position of the object. We want to estimate the latent variable, the speed of the object. The next state depends on the previous state and the latent variable the speed.</p>
  <pre><code># define the system and the data
  true_vel = .2
  true_pos = 0
  true_positions = [.2 * step for step in range(100)]
  </code></pre>
  <p>We assume that we have some noise in our observation (but that does not matter here).</p>
  <p>The question is: how do I model the dependency of the next state on the current state. I could supply the transition function a parameter idx to access the position at time t and then predict the position at time t+1.</p>
  <pre><code>vel = pymc.Normal(&quot;pos&quot;, 0, 1/(.5**2))
  idx = pymc.DiscreteUniform(&quot;idx&quot;, 0, 100, value=range(100), observed=True)
  
  @pm.deterministic
  def transition(positions=true_positions, vel=vel, idx=idx):
      return positions[idx] + vel
  
  # observation with gaussian noise
  obs = pymc.Normal(&quot;obs&quot;, mu=transition, tau=1/(.5**2))
  </code></pre>
  <p>However, the index seems to be an array which is not suitable for indexing. There is probably a better way to access the previous state.</p>