Ashley was private secretary to Lord Palmerston from 1858 to 1865 and worked as a barrister on the Oxford Circuit from 1865 to 1874.
He later stood unsuccessfully for the Liberal Unionist Party in the Glasgow Brdigeton by-election in 1887.