anchor.diff:  image::alt_neural2/rotations_L.png[rotations_of_filter, 600]
   {empty} +
   
  -Despite the setback, all \(90\circ\) are accurate and \(45\circ\) are only slightly skewed. Skew might be a blessing in disguise as it might endow the neuron to be _deform_ invariant as well (wishful thinking). In either case, the deformations are not too bad for \(3 \times \3\) filters, and it is atleast worth experiementing with vggnet which uses all \(3 \times \3\) conv filters.
  +Despite the setback, all \(90\circ\) are accurate and \(45\circ\) are only slightly skewed. Skew might be a blessing in disguise as it might endow the neuron to be _deform_ invariant as well (wishful thinking). In either case, the deformations are not too bad for \(3 \times \3\) filters, and it is at-least worth experimenting with vggnet which uses all \(3 \times \3\) conv filters.
   
  -The computations discussed so far apply to an individual neuron. For Convolutional layer, the output is genenerated as follows:
  +The computations discussed so far apply to an individual neuron. For Convolutional layer, the output is generated as follows:
   
   * Generate all 8 rotated filters from \(W\), \(W_{1}, \cdots, W_{8}\).
   * Compute output activation volumesfootnote:[If you are not familiar with the idea of activation volumes, I would recommend referring to link:http://cs231n.github.io/convolutional-networks/[CS231n Convolutional Neural Networks for Visual Recognition] for an outstanding explanation.] \(O_{1}, \cdots, O_{8}\) by convolving input with \(W_{1}, \cdots, W_{8}\).
  -* Across volumes, select \(x, y\) value corresponding to filter `(row, col)` that has the max value. This correspondins to selecting the best _aligned_ filter with input patch across the 8 rotations.
  +* Across volumes, select \(x, y\) value corresponding to filter `(row, col)` that has the max value. This corresponding to selecting the best _aligned_ filter with input patch across the 8 rotations.
   
  -These steps can be implemented as follows. Note that `W.shape = (rows, cols, nb_input_filters, nb_output_filters)`. You also need bleeding edge tensorflow from `master`; as of Nov 2016, `gather_nd`does nto have a gradient implementation.
  +These steps can be implemented as follows. Note that `W.shape = (rows, cols, nb_input_filters, nb_output_filters)`. You also need bleeding edge tensorflow from `master`; as of Nov 2016, `gather_nd` does not have a gradient implementation.
   
   [source,python]
   ----
  
positive.code:     return output
  ----
  
  
  
  == Experiments
  
  **Setup**
  
  * Mini vggnet comprising of \(3 \times 3\) convolutions with ReLU activation.
  * cifar10 dataset augmented with 10% random shifts along image rows/cols along with a 50% chance of horizontal flip.
  * `random_seed = 1337` for reproducability.
  
  The model has 1,250,858 parameters and trained for 50 epochs with a batch size of 32 using categorical crossentropy loss with Adam optimizer.
  
  [.text-center]
  .Test model
  
negative.code: {empty} +
  
  == Discussion
  
  **TODO: Add experimental results. WIP**
  
  Instead of trying all possible alignments, we could try to find \(\theta\) that _maximizes_ the dot product.
  
  