Very slow screen refresh on server screen share
After much ado, I figured out that the reason the performance is so terrible in the remote Mac Mini is because it is head-less and the GPU is not being engaged (according to this answer). I ended up ordering the headless adaptor for $15 and problem solved.