 After leaving Larkhall Academy, Miller started playing for local side Royal Albert, where he would stay until 1959.He was signed by Dunfermline Athletic before the start of the 1959–60 season, instantly becoming a first-team regular as the "Pars" battled against relegation.The arrival of Jock Stein in 1960 transformed the side however, and against the odds they reached 1961 Scottish Cup final against Celtic.