Rinne was developed in Kärpät playing predominantly for their junior team between 2000 and 2003 and, after that, for their men's team in championship-winning years 2004 and 2005, mostly as a backup to Niklas Bäckström.
Following Dan Ellis's stellar performance for Nashville in the 2007-08 campaign, starter Chris Mason was traded to the St. Louis Blues and Rinne was called up to serve as backup to Ellis for the subsequent 2008-09 season.