As soon as the Federation happened , Sir Edmund Barton was put in charge as Prime Minister of Australia until the first Australian elections were held . Barton won the election and continued to be the Prime Minister . Sir Edmund Barton was the caretaker Prime Minister of Australia at the inaugural Federal election in 1901 , at which he retained his office .