 Baillie, a central defender, made his Hibernian debut during the 2003–04 season when manager Bobby Williamson drafted him into defence for a League Cup quarter-final match against Celtic in December 2003, which the 18-year-old Baillie helped his side win 2–1.Injuries meant that he didn't get a prolonged spell in the first team.He had to wait for selection until another match against Celtic in the Scottish Premier League three months later, but Hibs were beaten 4–0 and Baillie himself was sent-off.