title: Cell state in DA-RNN
body: <p>I want to implement the DA-RNN in this <a href="https://www.ijcai.org/proceedings/2017/0366.pdf" rel="nofollow noreferrer">paper</a>. The page 3 of the research paper talks about the encoder network. Specifically I want to implement the following equations : </p>
  
  <p><a href="https://i.stack.imgur.com/rgrc3.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/rgrc3.png" alt="enter image description here"></a></p>
  
  <p>I want to confirm that in the above equation I require cell state <code>s</code> at every timestep ? Usually in RNNs we just need the cell state from the previous timestep. So what I mean is if we have a batch of 5 timestamps we use the hidden state <code>h</code> at all timesteps from <code>1 to 5</code> and we use cell state <code>s</code> from only the 5th timestep (and not for all the timesteps). </p>
  
  <p>The <code>out,s = dynamic_rnn</code> function in tensorflow also gives me this output where <code>out</code> will have hidden states computed at each timestep and <code>s</code> will be a tuple of <code>c</code> and <code>h</code> at the last timestep. For example if my input is of the size <code>batch_size x max_timestep x num_features</code> and lets say <code>max_timestep=5</code> then <code>s.c</code> and <code>s.h</code> will consist of cell state and hidden state from the 5th timestep and not for timesteps <code>1,2,3,4</code>. However in the paper the general notation is cell state <code>s</code> at time <code>t-1</code>. So if we are talking about 5 timesteps we need cell state <code>s</code> at timesteps <code>1,2,3,4</code> which I can't obtain thru <code>dynamic_rnn</code> function. </p>
  
  <p>So to summarise</p>
  
  <ol>
  <li>is my assumption correct that I need cell state <code>s</code> for all the timesteps to implement above equation in the image ? </li>
  <li>If yes how can i do this in tensorflow ? Should I write my own LSTM wrapper ?  </li>
  </ol>
  
  <p>Update : </p>
  
  <p>This <a href="https://stackoverflow.com/questions/47745027/tensorflow-how-to-obtain-intermediate-cell-states-c-from-lstmcell-using-dynam">answer</a> resolved my issue. I turns out there is no direct function to obtain cell states at each timestep however we can wrap the RNN cell and just give it as input to the <code>dynamicRNN</code> function in tf. </p>