In most colleges, gowns are worn to Formal Hall (formal dinner, held every night in some colleges, once or twice a term in a few others) and to Chapel.
On special occasions, fuller academic dress is used, including hoods.