The city stands on the hills and valleys surrounding the head of Otago Harbour . The city suburbs extend out into the surrounding valleys and hills , onto the isthmus of the Otago Peninsula , and along the shores of the Otago Harbour and the Pacific Ocean .