1. A computer program product for implementing selectively powering up a cache set of a multi-set associative cache, the computer program product comprising: a tangible storage medium readable by a processing circuit and storing instructions for execution by the processing circuit for performing a method comprising: receiving an instruction fetch address; based on determining that the instruction fetch address corresponds to one of a plurality of entries of a content addressable memory, identifying, by a computer, a cache set of the multi-set associative cache that contains a cache line referenced by the instruction fetch address, wherein only a subset of cache sets are powered up; based on the identified cache set not being powered up, selectively powering up the identified cache set of the multi-set associative cache; transmitting one or more instructions stored in the cache line referenced by the instruction fetch address to a processor, wherein each of the plurality of entries of the content addressable memory correspond to a recently accessed cache line and comprise an instruction address field that includes the address of the recently accessed instruction cache line and a set field that includes the cache set within the multi-set associative cache that contains the recently accessed instruction cache line, and based on determining that the instruction fetch address does not correspond to one of the plurality of recently accessed cache lines stored in the content addressable memory: powering up each of the cache sets of the multi-set associative cache in parallel; reading the cache lines of each of the cache sets referenced by the instruction fetch address; selecting the cache line of a cache set based on the instruction fetch address; and transmitting a data stored in the cache line referenced by the instruction fetch address to a processor.