title: Time delay neural network in tensorflow and the meaning of convolutions
body: <p>I wondered if there was anyone who might spare a little time to help me with Time-delay neural networks?</p>
  
  <p>As I understand it, each neuron is sensitive to part of the input through a particular number of time-steps. In my example, I have a 2D array of 31 amino acids in a sequence (time if you will). Each entry is a sparse encoded bitfield, 21 wide, with a true if its a particular acid, and false in all other entries.</p>
  
  <p>As I understand it, I need to use a convolution. I need a kernel that is 1D in size. Taking the image analogy, I have a 1D image that, instead of having RGB channels, has 21 channels thus producing 21 feature maps. My convolution is 1 x time_window_size x 21. I.e, I have 21 neurons, each sensitive to a particular channel (in this case the presence of a particular amino acid).</p>
  
  <p>The source code is here on git:
  <a href="https://github.com/OniDaito/MRes/blob/master/nn01/nn01.py" rel="nofollow noreferrer">https://github.com/OniDaito/MRes/blob/master/nn01/nn01.py</a></p>
  
  <p>I'm aware that LSTMs, dense encoding and the like will be better - I'd just like to start somewhere known and progress from there. </p>
  
  <p>My question - is my assumption that a 1D convolution will result in a set of neurons with shared weights that are sensitive to order and a particular channel?</p>
  
  <p>Cheers</p>
  
  <p>Ben</p>