Are absence of labels for classes of interest in a vision dataset a big problem?
So depends on sizes— if the second dataset has a lot of traffic lights without labels and you in a basic setup train on it, it will cause large performance decreases on that class. If it’s small, then it may just be “noise” to the model and it’ll filter it out, and still perform well. My recommendation is to adjust the network slightly and add a flag input saying which dataset the input was from: if from first, train normally. if from the second then ignore all losses pertain to boxes/classes that the network thinks a traffic light was in: that way your labels don’t try to tell the mode l “there’s no traffic light there”