 When first built in 1869 , Sri Temasek was the residence of the Colonial Secretary of the Straits Settlements ( later Colonial Secretary / Chief Secretary of Singapore ) . All holders of the position had resided in Sri Temasek from 1869 to 1959 . It is currently designated as the official residence of the Prime Minister of Singapore since 1959 , though none of the past and present holders of the post , Lee Kuan Yew , Goh Chok Tong and Lee Hsien Loong , have lived in it as a family home . In the 1998 book Lee Kuan Yew : The Man and His Ideas , Lee Kuan Yew said that when he became Prime Minister in 1959 he and his wife Kwa Geok Choo decided not to move into Sri Temasek with their three children , who were then aged seven , five and two , because the couple " did not want them to grow up in such grand surroundings with butlers and orderlies to fuss over their needs " . The family did stay there in 1965 for a short time for security reasons when Singapore separated from Malaysia . Apart from that , according to a 2000 Straits Times interview with Kwa , the Lee children often played in the grounds of the house in the evening while their father played golf or made use of the practice tee and putting green . 
