Due to its location in Melbourne's former justice precinct, the college originally bordered the Melbourne Gaol, City Watch-House (now Building 19) and Supreme Court (later the Magistrates' Court and now Building 20).
It is bounded by La Trobe Street to the south, Lygon Street and Russell Street to the east, Queensberry Street and Victoria Street to the north, Elizabeth Street and Swanston Street to the west.