Extracting multiple softmax values from image I am currently in the pre-process of starting an image classification and extraction project which needs to output multiple softmax and absolute values from a single image like such:

{
 time: "20:20", 
 teams: [
   {
     red: { goals: 2},
     blue: { goals: 1},
   },
   {
     scored_by : [{
      john: 80%, 
      kyle: 51%, 
      darren: 20%
     }
   ]}
 ]
}


I can create multiple models which are responsible for different task such as reading the time from the image as well as the score and eventually combine both. I would however like to make sure I maximize on efficiency to make sure the process is as fast as possible.

Any pointers in the right direction would be greatly appreciated.

With kind regards,
Dennis
It is not unheard of to share network weights with multiple output layers. I have seen it on DeepMind's Asynchronous Deep Learning paper, and I have also used it here.

The idea is to share all the layers and just have multiple outputs. However, this might decrease the accuracy of your networks, as is the usual performance VS accuracy trade-off.



To optimize this, just calculate the loss of both outputs and sum them when feeding the optimizer, like this

            self.target_policy_fast_t = tf.placeholder('float32', [None, a_size], name='target_policy_fast_t')
            self.loss_policy_fast = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits=self.network.policy_fast_before_softmax, labels=self.loss_policy_fast_t))

            self.target_policy_slow_t = tf.placeholder('float32', [None, a_size], name='target_policy_slow_t')
            self.loss_policy_slow = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits=self.network.policy_slow_before_softmax, labels=self.target_policy_slow_t))

            self.loss = tf.reduce_mean(self.loss_policy_fast + self.loss_policy_slow, name='loss')