Trained CNN individually on multiple images to classify them, how can I now classify a related "set" of these images that correspond to one object?
As far as I understand your problem is related to ensemble learning: You ensemble is a set of models trained on detecting the class of an object on images with the same angle on the object, and now you want to classify an object and choose the model voting of the model trained on the correct angle?
In that case, I would choose the prediction of the model which has the most unambiguous output before outputting a binary label.
So it's probably related to these topics:

Majority Voting in Ensemble Learning
Ensemble Learning in Machine Learning | Getting Started