Video generation with Shift Register
In NTSC, the active portion of a horizontal line is 52.6 us.  To pack 640 pixels in there would take a 12.17 MHz shift clock and a 1.52 MHz load rate.  I don't think an AVR can generate a read address, read the data, push it out to the PISO shift register, and calculate the next RAM address fast enough.

Going back in time to microprocessor-based video terminals, they used the micro's for the user interface and writing to the (dual-port) video RAM.  Scanning the RAM and turning its contents into video was all random hardware.  Today that hardware certainly would fit in one CPLD or FPGA, but functionally it still would be a non-firmware engine.