anchor.diff:  [.text-center]
   .Test model
   image::alt_neural1/model.png[test_model, 300]
  -{empty} +
   
   == Discussion
   
  
positive.code: [.text-center]
  .Computation model of a neuron
  image::alt_neural1/neuron_model.jpeg[neuron_model, 400]
  {empty} +
  
  As far as i am aware, all architectures build on this fundamental computational model. Let us examine what the computation is conceptually doing in a step by step fashion.
  
  
negative.code: [.text-center]
  .Similarity between vectors X and W
  image::alt_neural1/vector_similarity.png[vector_similarity, 300]
  
  If we view a neuron as fundamental template matching unit, then various magnitude (length) of \(X\) and \(W\) should yield the same output. If we constrain \(W\) to be a unit vector (\(|W| = 1\)), then backprop would only need to search along points on a hypersphere of unit radius. Although the hypersphere contains infinitely many points, with sufficient discretization, the search space intuitively _feels_ a lot smaller than the entire weight vector space.
  
  