Avoiding trivial solution with sparse output I have a network whose output is a large vector ( ~ 2000 let's say ). The ground truth output is very sparse and binary- that is out of the 2000 most of the values will be 0 and just a handful will be 1. Just to clarify, each sample will have 1's at different locations with some uneven likelihood of there being a 1 at any given neuron. 

I am pretty sure that if I run this network it will quickly find the trivial solutions of outputting all 0 and just stay there. 

How might you suggest to nudge the network to avoid this trivial solution? 

I thought of somehow weighting the samples in such a way that each sample will give a high weight to the neurons that are equal to 1. This seems very hard to implement efficiently. How can I easily implement this in tensorflow/keras efficiently?

Another Idea is to set all the 0's to random centered at .5 and slowly decrease them towards zero as the network trains. 

Other ideas?
Some interesting suggestions I got from others: 


split the network so that each subnetwork is in charge of a subset of the classes.
use the Dice loss or jaccard loss which are pretty good for this situation.
pre-train with the less prevalent classes only and add classes over epochs. This needs to be done very carefully.