When training an object detection network for one class, should I include empty images in the dataset?
It is correct that to eliminate the false positives, the training must include images that reflect true negative cases. The objective of training is not this.


  Find the objects of class $O$ in scene $X$.


It is this.


  Find any objects of class $O$ in scene $X$.


Training is effective when sample $S$ is drawn according to the same distribution as will be used for drawing at test time or during actual use.

It is not necessary to annotate the background if the loss function is chosen so that finding nothing when nothing there is indicated as a zero loss and a proper distribution of those cases exist in the training sample.