How to perform polynomial landmark detection with deep learning
Landmarks are nice when you have a fixed amount for every image but I don't think it is the right approach for your problem. Instead I think you should look into models that segment images by applying a mask. A place to start could be to look into Mask R-CNN.

Here is the paper: Mask R-CNN

Here is how to train a ready made Mask R-CNN implementation with your own dataset using Keras: Tutorial