anchor.diff:      return output
   ----
   
  -
  -
   == Experiments
   
   **Setup**
   
   * Mini vggnet comprising of \(3 \times 3\) convolutions with ReLU activation.
   * cifar10 dataset augmented with 10% random shifts along image rows/cols along with a 50% chance of horizontal flip.
  -* `random_seed = 1337` for reproducability.
  +* `random_seed = 1337` for reproducibility.
   
  -The model has 1,250,858 parameters and trained for 50 epochs with a batch size of 32 using categorical crossentropy loss with Adam optimizer.
  +The model has 1,250,858 parameters and trained for 50 epochs with a batch size of 32 using categorical cross-entropy loss with Adam optimizer.
   
   [.text-center]
   .Test model
  
positive.code: image::alt_neural2/vertical_line_filter.png[vertical_line_filter, 300]
  {empty} +
  
  When a neuron tries to match this pattern in a \(3 \times 3\) input image patch, it will only output a high value if a vertical edge is found in the center of the image patch. i.e., dot product is not invariant to shifts. My first thought was to use link:https://en.wikipedia.org/wiki/Pearson_product-moment_correlation_coefficient[pearson coefficient] as it provides shift invariance. However, this not a problem in practice as the filter is slid across all possible \(3 \times 3\) patches within the input image (see fig2).
  
  [.text-center]
  .Sliding filter over input image to generate the output activation area (image borrowed from link:http://intellabs.github.io/RiverTrail/tutorial/[river trail documentation])
  
negative.code: = Baking rotational invariance into a neuron
  :hp-tags: deep learning
  
  This is a continuation from the first link:https://raghakot.github.io/2017/01/03/Exploring-alternative-neural-computational-models.html[post] on expiring alternative neural computational models. To recap quickly, a neuron is fundamentally acting as a pattern matching unit by computing the dot product \(W \cdot X\), which is equivalent to unnormalized cosine similarity between vectors \(W\) and \(X\).
  
  This model of pattern matching is very naive, in a sense that it is not invariant to translations or rotations. Consider the following weight vector template (conv filter) corresponding to vertical edge pattern `|`:
  
  