1. A latency tolerant system for executing video processing operations, comprising: a host interface for implementing communication between a video processor and a host CPU; a scalar execution unit coupled to the host interface and configured to execute scalar video processing operations; a vector execution unit coupled to the host interface and configured to execute vector video processing operations; a command FIFO for enabling the vector execution unit to operate on a demand driven basis by accessing the command FIFO, wherein the vector execution unit is operable to adjust the performance level of the vector execution unit based on the demand driven basis, and wherein the demand driven basis is configured to hide a latency of a data transfer from the different memory locations to the command FIFO of the vector execution unit; a memory interface for implementing communication between the video processor and a frame buffer memory; and a DMA engine built into the memory interface for implementing DMA transfers between a plurality of different memory locations and for loading a datastore memory and an instruction cache with data and instructions for the vector execution unit.