adams 's rise to power had begun with his appointment as secretary to robert walpole-and grey saw , as clearly as though the scene was taking place before him on the drive , the picture of the tall , ailing prime minister , leaning heavily on his secretary-his irish secretary-coming down the path to visit the widow of the late duke of pardloe .