Training 128x128 autoencoders on 512x512 images, produces strange gridline after recombining So I'm training an autoencoder that can recreate 128x128 images, so it can recreate any images by splitting them into 128x128 patches first, running it through the autoencoder, and having them combined with each other to form the original image. 

I should be using other dimensions too but right now I'm testing this with 512x512 images. So :

   x = next_train_batch(25) 


    for xx in range(0,4):
        x_index = xx*128
        for y in range(0,4):
            y_index = y*128

            this_image = x[:, x_index:x_index+128, y_index:y_index+128, :]
            sess.run(training, feed_dict={x_in: this_image, step_iter_global:step_iter})


This is how I'm doing it now. But this keeps happening:


True, this is just in the beginning of the training process. But I guess I wonder why the autoencoder is having a hard time learning the edges of the patches. These gridlines actually are still present in visualizations of certain layers in the autoencoder (the reason I'm doing this is because I'm trying new layers that could potentially improve autoencoders) even if the output image doesn't have it.
The problem is that the model has no idea what the image should look like outside of the 128x128 crop you feed it with, so it fails to extrapolate appropriately. One simple fix is to train the model on bigger (and overlapping!) crops, like 144x144, and then crop out 8 pixels from each side.

However, even this approach does not guarantee absence of any visible edges, as it's hard for the model to enforce this "boundary consistency" without direct access to its output from the previous patch. This could be approached with more powerful decoders, like PixelRNN / PixelCNN