Which deep learning models are suitable for image-to-image mapping?
Since you have already tried U-Net. You may look into Siamese Networks (with CNNs for images), they are very well known for computing similarity via deep learning. This is a central idea and can be performed with both text and images. As a tip, you may be able to leverage a lot of architecture from U-Net to Siamese. 

Hope it helps, Some useful links to start with :


https://medium.com/@prabhnoor0212/siamese-network-keras-31a3a8f37d04
https://www.aclweb.org/anthology/W16-1617.pdf