title: What layers to use when comparing 2 arrays of 2 image
body: <p>I have 2 arrays of image, one of a degraded picture, the other one of the same picture but in a clean state. Their <code>shape=(576, 720, 3)</code>, since these images are <code>720*576</code>, and have <code>3</code> channels (RGB). I am currently trying to train my model with the degraded picture array as input, and the clean picture array as output. It is working perfectly fine, however I do precisely know what layer to add to my model, nor how to improve the accuracy I currently have.</p>
  <p>Here's my current model ( which is <strong>NOT</strong> good, I basically just randomly put layers here ):</p>
  <pre class="lang-py prettyprint-override"><code>model = models.Sequential()
  
  model.add(layers.Conv2D(32, 5, activation='relu', padding='same', input_shape=(576, 720, 3)))
  model.add(layers.AveragePooling2D(pool_size=(2, 2), strides=(2, 2)))
  
  model.add(layers.Conv2D(64, 3, activation='relu', padding='same'))
  model.add(layers.AveragePooling2D(pool_size=(2, 2)))
  
  model.add(layers.Conv2D(3, 3, activation='relu', padding='same'))
  model.add(layers.UpSampling2D((4, 4)))
  </code></pre>
  <p><a href="https://i.stack.imgur.com/DYjwU.png" rel="nofollow noreferrer"><img src="https://i.stack.imgur.com/DYjwU.png" alt="Model Description" /></a></p>
  <p>And here's how I compile it:</p>
  <pre class="lang-py prettyprint-override"><code>model.compile(optimizer='rmsprop', loss='mean_absolute_error', metrics=[&quot;accuracy&quot;])
  </code></pre>
  <p>I currently have a 5000 pair of degraded / clean image, and I manage to get to ~75% accuracy and ~20 loss, but can't get to improve the model.</p>
  <p>What I would like is to understand what I am doing here, since I can't get to find anything on the Internet apart from Image Classification, which is not what I am doing. I would like to know what layer can be useful, and why. I know Conv2D is useful since I want to find flaws in images and correct them, and I know LSTM is useful if I'm working with a video, but apart from that, I do not know the correct layer Sequence to setup, nor the correct layers.</p>
  <p>Any help would be much appreciated, thank you.</p>