How to adapt the softmax layer for multiple labels? Question: A image has multiple labels. Given a set of image with labels, how to adapt the softmax layers?

My idea:


Encode multiple labels to 0-1 variables. Use logistics regression as the output layer.
Choose the top X labels in softmax output. But I do not know 


how to determine the threshold.
how to modify the code in a pretrained net. Let's say 




{
      "op": "null",
      "param": {},
      "name": "softmax_label",
      "inputs": [],
      "backward_source_id": -1
    },
    {
      "op": "Softmax",
      "param": {"grad_scale": "1"},
      "name": "softmax",
      "inputs": [[510, 0], [511, 0]],
      "backward_source_id": -1
    }



Any recommendations on tutorials are appreciated. I have some theory background from the book ESL.
I don't know what's going on in your code, but you seem to be close: to get multiple labels, simply replace the softmax output layer with a logistic layer (or something else that maps a real number to a probability), then optimize the cross entropy. That way you will have a probability associated with each label such that their sum across labels no longer need to add to unity.