Multiple-input multiple-output CNN with custom loss function
Looks correct. When u use same branches in NN architecture - this name siamese-type nn. U may see there how to implement it in PyPorch. In my opinion, PyTorch must easiest framework for this task. Also implementation of encoder-decoder network u can find there.

This is frequently using technique when u need to compute, as example, distance between few inputs. But i'm had doubts about concatenation part. Witch dimension u need for outputs? If same as input, then all looks fine. If different dim, then just add fully connected layers after last convolution.