Neural Net for regression task on images only learning mean of training data
I took a deeper look at this problem with some example data (that the OP provided) and some simplifications. I just predicted brick height param as opposed to the 17 example params in the question. I used the Keras library in Python to explore a few different architectures.

Initially I replicated the problem, simple CNNs were predicting identical values for all heights, and the best loss (around 0.0053 on the sample data) was when this predicted the mean very precisely for all inputs. For some reason the network could not find the visual structure that is obviously in the data.

After some experimentation, I had a network that functioned as intended. It was surprisingly deep for such a simple problem. My best guess at the reason why is that the combined overlap of all the convolutions needs to be of the same size order as the features being detected.

Some important details:


Check that the order of the dimensions of the images when read with the image library you are using match the expected structure that the CNN is working on. If the dimensions are in a different order, this will not cause any error, but it scrambles the interpretation used by the CNN and prevents it learning anything but the mean.
Avoid using pooling layers, as they obscure pixel locations of features from further layers.
Check that your parameters are not ambiguous. The two "brick width" parameters seem interchangeable for instance. That makes it much harder for the network to predict them, and it is likely to output a rough mean for the example even if you avoid the problems with architecture.
The problem is harder to find a working architecture for than I first thought. I am not entirely sure why that is the case, but am guessing it is to do with the size of the features that need to be isolated in order for the network to "measure" them. Eventual working architectures were quite deep e.g. 10 convolutional layers, despite the pixel-level structures being quite obvious and clear in the training data.


Here is a gist of my solution in Python/Keras. Summary of architecture in the gist: 4 times Conv2d layers with 8 feature maps and 5x5 kernel. Dropout 25%. 4 more times Conv2d layers with 16 feature maps and 5x5 kernel. Dropout 50%. Fully connected layer with 256 outputs. Dropout 50%. Fully connected later with 128 outputs. Dropout 50%. Single output neuron. All layers use ReLU activation, except output which is linear.

The example re-sizes images to 50x50 for speed. However, using 100x100 images works fine. I added a couple of convolutional layers to it for that (one each for the 8 features and 16 features section), and get a reasonable loss of ~0.00025 on a validation set. It looks possible to get a lower loss still with more training examples, more epochs and perhaps more feature maps. Many of the predictions were very good, close to the actual value. 

Interestingly, the worst errors were for short bricks predicting close to a 2:1 or 3:2 ratio with the ground truth, implying that the network had found correct structure but was being confused by the periodic nature of the bricks. I think this is something that could be resolved with more training examples (and perhaps more features in higher layers) and it should be possible to get much lower loss metric on brick height.

I think that the corresponding tiny-cnn structure looks like this:

net &lt;&lt; convolutional_layer&lt;relu&gt;(100, 100, 5, 3, 8)
    &lt;&lt; convolutional_layer&lt;relu&gt;(96, 96, 5, 8, 8)
    &lt;&lt; convolutional_layer&lt;relu&gt;(92, 92, 5, 8, 8)
    &lt;&lt; convolutional_layer&lt;relu&gt;(88, 88, 5, 8, 8)
    &lt;&lt; convolutional_layer&lt;relu&gt;(84, 84, 5, 8, 8)
    &lt;&lt; dropout(80 * 80 * 8, 0.25)
    &lt;&lt; convolutional_layer&lt;relu&gt;(80, 80, 5, 8, 16)
    &lt;&lt; convolutional_layer&lt;relu&gt;(76, 76, 5, 16, 16)
    &lt;&lt; convolutional_layer&lt;relu&gt;(72, 72, 5, 16, 16)
    &lt;&lt; convolutional_layer&lt;relu&gt;(68, 68, 5, 16, 16)
    &lt;&lt; convolutional_layer&lt;relu&gt;(64, 64, 5, 16, 16)
    &lt;&lt; dropout(60 * 60 * 16, 0.5)
    &lt;&lt; fully_connected_layer&lt;relu&gt;(57600, 256)
    &lt;&lt; dropout(256, 0.5)
    &lt;&lt; fully_connected_layer&lt;relu&gt;(256, 128)
    &lt;&lt; dropout(128, 0.5)
    &lt;&lt; fully_connected_layer&lt;identity&gt;(128, 1)