The line of symmetry move when I adjust the size
The easiest way to fix weird mirroring is to create a new empty and use that as your line of symmetry by selecting it under Mirror Object



If it doesn't work as you expect this is usually faster than messing around with whatever it is.