Confusion about adjoint of convolution
Complex conjugation in the discrete Fourier domain should result in spatial domain horizontal and vertical mirroring around the origin. The origin happens to be at row 1 column 1, as is evident from MATLAB's definition of FFT, with Y = fft(X):

$$Y(k)= \sum^n_{j=1} X(j) e^{−2πi(j−1)​(k−1)/n}$$

For $j = 1$ the phase is zero for all $k$ so anything on column 1 or on row 1 will stay there through complex conjugation. Column 11 and row 11 are each 10 steps from the origin horizontally or vertically, as are column 12 and row 12 if you measure the distance across the border. So the reversal swaps the data between columns 11 and 12, and between rows 11 and 12. Figure 1 below illustrates the mirroring.

Figure 1. Row or column indices and the corresponding phases of frequency $k = 2$ plotted as angles on the circle.

If the dimensions of your data were even rather than odd as they are now, there would be a middle row and a middle column that would be at equal distance from the origin both directly and across the border.