Video RAM: How to create a shared bus for processor and VGA controller
Actually, that's TOO simple. You haven't provided any way for the CPU's address bus to get to the RAM chips for writing data. Each of the RAM chips needs an address multiplexer in addition to the data buffers that you have. The good news is that the same muxes can probably be used to steer the CPU's write enable to the correct chip as well.

There's another problem as well. I can see why you didn't connect the three LSBs of Pixel, assuming you're only using 1 bit per pixel, but you can't do the same thing with Line. You need ALL of the Line addresses, which means that your RAMs need to be 8&times; the size &mdash; 16 address lines instead of 13, giving you 65 kB of storage (up to 512k pixels). Standard VGA requires 480 &times; 640 = 300k pixels.