How to choose our data set wisely?
You have got nothing to lose by framing it as a multi-label detection task. All you have to do is remove the final Softmax layer and format your ground truth as multi-hot, and you can regress multiple labels directly. Mean Squared Error and Sum of Absolute Differences are neutrally biased with respect to false positives and false negatives if the data are balanced, whereas categorical cross-entropy is biased for higher recall and lower precision and usually does not work as well once the Softmax is removed, so choose your loss function accordingly.

Definitely include more objects in your training set that you don't want to detect--this will reduce overfitting by forcing the detection template(s) to improve their specificity.