title: TensorFlow CNN Tutorial: How to edit top layer to be locally connected?
body: <p>I have some background in machine learning and python, but I am just learning TensorFlow. I am going through the <a href="https://www.tensorflow.org/tutorials/deep_cnn/" rel="nofollow noreferrer">tutorial on deep convolutional neural nets</a> to teach myself how to use it for image classification. Along the way there is an exercise, which I am having trouble completing.</p>
  
  <p><strong>EXERCISE:</strong> The model architecture in inference() differs slightly from the CIFAR-10 model specified in cuda-convnet. In particular, the top layers of Alex's original model are locally connected and not fully connected. Try editing the architecture to exactly reproduce the locally connected architecture in the top layer.</p>
  
  <p>The exercise refers to the inference() function in the <a href="https://github.com/tensorflow/models/blob/master/tutorials/image/cifar10/cifar10.py" rel="nofollow noreferrer">cifar10.py model</a>. The 2nd to last layer (called local4) has a shape=[384, 192], and the top layer has a shape=[192, NUM_CLASSES], where NUM_CLASSES=10 of course. I think the code that we are asked to edit is somewhere in the code defining the top layer:</p>
  
  <pre><code>with tf.variable_scope('softmax_linear') as scope:
      weights = _variable_with_weight_decay('weights', [192, NUM_CLASSES],
                                        stddev=1/192.0, wd=0.0)
      biases = _variable_on_cpu('biases', [NUM_CLASSES],
                            tf.constant_initializer(0.0))
      softmax_linear = tf.add(tf.matmul(local4, weights), biases,name=scope.name
      _activation_summary(softmax_linear)
  </code></pre>
  
  <p>But I don't see any code that determines the probability of connecting between layers, so I don't know how we can change the model from fully connected to locally connected. Does somebody know how to do this?</p>