1. A chip-multiprocessing system with scalable architecture, comprising on a single chip: a plurality of processor cores; a two-level cache hierarchy including a pair of instruction and data caches for, and private to, each processor core, the pair being first level caches, and a second level cache with a relaxed inclusion property, the second-level cache being logically shared by the plurality of processor cores, the second level cache being modular with a plurality of interleaved modules; one or more memory controllers capable of operatively communicating with the two-level cache hierarchy and with an off-chip memory; a cache coherence protocol; one or more coherence protocol engines; an intra-chip switch; and an interconnect subsystem.