Why does "classification+regression" work better than "classification" in my NN? I am working on a research problem where I got stuck on something which doesn't really make sense to me. To explain the issue I will give an example of a similar problem (face age detection). 

Suppose that I have many face images that I precisely know how many **days** old the person in each image. (I need precision in my problem). 

I train my model using CNN in three ways. 

In the first training, I create a neural network that gives an output of 75 classes (say the ages are 0-74) which tells me the correct age with 30-40% accuracy. Below is the final code piece after the CNN layers:

net1 = tf.reduce_mean(net, axis=(1, 2), name='gap')

# Fully-connected classifier 
net1 = slim.fully_connected(net1, 512, activation_fn=utils.lrelu, scope='cmi_ff1')
net1 = slim.dropout(net1, prob, scope='dropout1_net1')
net1 = slim.fully_connected(net1, 256, activation_fn=utils.lrelu, scope='cmi_ff2_net1')
net1 = slim.dropout(net1, prob, scope='dropout2_net1')

y_ = slim.fully_connected(net1, n_classes, activation_fn=tf.nn.softmax, scope='cmi_ffo_net1')
loss = tf.reduce_mean(tf.nn.sparse_softmax_cross_entropy_with_logits( logits=y_, labels=y))


n_classes=75 as the number of ages is 75.

I keep everything the same and change the output layer from classification to a regression which results in significantly lower accuracy that each person approx 37 years old. This tells me that my neural network is trying to minimize the error and doesn't learn anything. I keep everything the same except for the output layer and loss function.

y2_ = slim.fully_connected(net1, 1, activation_fn=None, scope='cmi_ffo_net2') 
loss2 = tf.reduce_mean(tf.abs(y2_ - tf.cast(y,tf.float32)))


Up to now, nothing sounds weird to me (although I am a newbie in DL). The weird thing is happening when I decide to use two heads instead of a single head. I split the network into two right after the last convolutional layer. 

net1 = tf.reduce_mean(net, axis=(1, 2), name='gap')
net2 = tf.reduce_mean(net, axis=(1, 2), name='gap')

# Fully-connected classifier 
net1 = slim.fully_connected(net1, 512, activation_fn=activation, scope='cmi_ff1')
net1 = slim.dropout(net1, prob, scope='dropout_net1')
net1 = slim.fully_connected(net1, 256, activation_fn=activation, scope='cmi_ff2_net1')
net1 = slim.dropout(net1, prob, scope='dropout_net1')

y_ = slim.fully_connected(net1, n_classes, activation_fn=tf.nn.softmax, scope='cmi_ffo_net1')
loss = tf.reduce_mean(tf.nn.sparse_softmax_cross_entropy_with_logits(logits=y_, labels=y))

# Fully-connected regressor
net2 = slim.fully_connected(net2, first_fc, activation_fn=activation, scope='cmi_ff1')
net2 = slim.dropout(net2, prob, scope='dropout_net1')
net2 = slim.fully_connected(net2, second_fc, activation_fn=activation, scope='cmi_ff2_net2')

y2_ = slim.fully_connected(net2, 1, activation_fn=None, scope='cmi_ffo_net2') 
loss2 = tf.reduce_mean(tf.abs(y2_ - tf.cast(y,tf.float32)))


Here, the accuracy of classification goes up to 65-70% and regression remains to the same (converges to the middle value). While regression seems to be useless in the entire process, somehow it gives an amazing push to the classification to improve accuracy that I can't even speculate about the reason. 

I am adding a figure that shows the accuracy and loss of the two-headed network. (I was trying to optimize some hyperparameters.) 


Explanation of the figure: 

subplot 1,1: Training accuracy for classification head
subplot 2,1: Validation accuracy for classification head
subplot 1,2: Loss for classification head
subplot 1,3: Loss for regression head
subplot 2,2: Total loss: 0.2* regression + classification


I wonder what could be the reason for this or is there something that I do wrong? Also, I know the code is not complete but I can't share the entire code. I just need some directions about what to think and where to look.
You are using accuracy to estimate the performance of your model, which is a bad idea. You can't understand the limitations and advantages of your model using accuracy as a evaluation metric. Consider the following situations 


Your model predicts 36 instead of 37
Your model predicts 106 instead of 37


Accuracy as a evaluation metric, treats both of them in the same way. I'd suggest you to go with MAE or RMSE for evaluation. You can train the network either using classification or regression, but make sure that you evaluate it using the right metric. 

Btw, the multihead idea is really good.