CNN Pooling layers unhelpful when location important?
Pooling doesn't completely remove information about location of features within your image. If you don't want to use pooling but want to reduce the size of your neural net layers you should try a value of stride greater than 1 in your convolutional layers. e.g:

keras.layers.Conv2D(filters, kernel_size, strides=(1, 1), padding='valid', data_format=None, dilation_rate=(1, 1), activation=None, use_bias=True, kernel_initializer='glorot_uniform', bias_initializer='zeros', kernel_regularizer=None, bias_regularizer=None, activity_regularizer=None, kernel_constraint=None, bias_constraint=None)