Sunderland, helping them to finish third in the Championship, after which they lost 5–4 on penalties to Crystal Palace in the play-off semi-finals. For the second consecutive season Smith scored the winning goal in an FA Cup quarter final, and this time he was selected to play in the semi-final, which Sunderland lost 1–0 to Millwall. Smith bought a house in the Sunderland area and anticipated discussing a two-year contract at the end of the 2003–04 season. However, talks stalled and he chose to move to another Championship club, Derby County. Derby County Smith made his Derby debut in a league game against Leeds United. He scored his first goal for the club in a 2–0 victory over Queens Park Rangers. Derby reached the Championship play-offs in Smith's first season at Pride Park. However, they were defeated 2–0 at Preston in the first leg of the semi finals. In the second leg, at Pride Park, Smith was substituted at half time. The game finished goalless, meaning that Preston progressed to the final. Smith came second in the voting for Player of the year behind Iñigo Idiakez, but did win the inaugural Derbyshire Fair Play award. Derby's form deteriorated in 2005–06, with the club finishing 20th in the table, and eliminated early from both cup competitions. However, this time Smith became the club's Player of the year. At the end of the season, Billy Davies was appointed Derby manager charged with the task of turning the club around and Smith believed that Davies "wanted to bring his own players in." He was close to signing for Sheffield United, before he was the subject of an approach from former club Watford, who had just won promotion to the Premier League. On 30 August 2006, Smith rejoined Watford for a fee of £500k, signing a four-year contract. Return to Watford Smith scored his first league goal of his second spell at Watford on 26 December 2006, against Arsenal at Vicarage Road. However, this wasn't enough to prevent a 2–1 defeat. Watford reached the last four of the FA Cup in 2007, and in contrast to 2003, Smith was selected to start for Watford in the semi-final. However, for the second time in his career Smith was on the losing side, this time 4–1 to Manchester United. At the end of the 2006–07 season Smith finished sixth in the voting for the club's Player