 Although some in the Conservative party agreed with Wilson ( and Jo Grimond , the Liberal leader ) that the Conservatives would have won the 1964 election if Butler had been Prime Minister , The Times observed , " it should not be overlooked that in October 1963 Home took over a Government whose morale was shattered and whose standing in the opinion polls was abysmal . A year later Labour won the general election , with an overall majority of only four seats . That [ Home ] recovered so much ground in so short a time was in itself an achievement . " However , looking back across Home 's career , The Times considered that his reputation rested not on his brief premiership , but on his two spells as Foreign Secretary : " He brought to the office ... his capacity for straight talking , for toughness towards the Soviet Union and for firmness ( sometimes interpreted as a lack of sympathy ) towards the countries of Africa and Asia . But he brought something else as well : an unusual degree of international respect . " 
