What is an object detection problem with only one class called?
If you are talking about "two-stage" obejct detectors like Faster R-CNN, note that the second phase is not only for classification, but to obtain more accurate results (https://stackoverflow.com/a/61965140/4762996).

in addition, I guess training a detector with many classes acts like a regularizer, which results in much better accuracies.

The only benefit of explicitly training with one class could be the reduzed model size (and a corresponding speedup).

Note also that there are one stage object detectors (CornerNet: https://arxiv.org/abs/1808.01244, YOLOv3: https://pjreddie.com/media/files/papers/YOLOv3.pdf, DETR: https://arxiv.org/abs/2005.12872, and many more). I just wanted to stress that just because you only have one class, it does not make sense to take a two-stage detection architecture and skip the second stage.

Finally, have a look at: https://stats.stackexchange.com/q/351079

An interesting direction would be to incorporate one-class classification approaches, e.g. as mentioned here: https://stackoverflow.com/a/61965358/4762996.