Constructing an image from a dense layer output
Concatenating the two output layers is not such a pain as you might think. For starters, there are a number of things that can possibly go wrong with the model. Let's look at them one by one:

1) If the loss curve is unstable or is not coverging, try using batchnorm in the intermediate dense layers. Since there a lot of dense layers, it can lead to overfitting as well so try using dropout too. Experiment with batchnorm and dropout separately as well as together to see which one helps.

2) Check the range of the input and output of your decoder network: Normalise the data to be within 0 and 1 by simply multiplying the RGB values with 255. Use this normalised value as the true target value if your output image is same as input image or else use the normalised target image as your target output.  Similarly, to obtain the output value between 0 and 1, simply clip the output from relu to 1 and store it in a float variable as its a fraction. Use this float variable for computing the loss. Multiply it with 255 and store it in integer variable for visualisation purpose.

If these don't work you might want to look at other upsampling methods out there like: Deconvolution layers and Pixel Shuffle layers. You might also have a look at the decoder of generative models like GANs, specifically the CGAN (conditional GAN) which do a similar thing of merging information from an image with some known prior.