OK to replace images on server?
I've never had any problems just replacing images.

If there are any images that have image styles applied then don't forget to clear the image styles caches/files so they get regenerated as well.  You can do them all at once with Image style flush

tl;dr:

Should be fine so long as the filenames match.