VGA Image Refreshing
VGA isn't a protocol that is talked to some memory. The intensity values are used directly to modulate the intensity of an electron beam hitting the fosphor in your screen. So yes, you need all these signals to keep the electron beam travelling to the right places in your CRT. LCD monitors that have an VGA input are just emulating that behaviour, so yes, they will need all that too.