Wolverhampton Wanderers on 9 May 2004. Caldwell appeared in the last two matches of the 2003–04 season, as Newcastle United failed to qualify for the UEFA Champions League next season. In total at the club, he made a total of seven appearances in all competitions. On 2 February 2004, Caldwell joined fellow Premiership club Leeds United on loan as part of a loan swap for Michael Bridges who went in the opposite direction. He made his debut for the club, starting the whole game, in a 2–0 loss against Aston Villa five days later on 7 February 2004. Since joining Leeds United, Caldwell became a first team regular, playing in the centre–back position, as he attempt to help the club avoid relegation once again. It wasn't until on 10 April 2004 when Caldwell scored his first goal for Leeds United, in a 2–1 win against Blackburn Rovers. However, he was recalled by his parent club on 5 May 2004. By the time Caldwell was recalled, he made thirteen appearances and scoring once in all competitions. Following his absent, Leeds United was eventually relegated to the Championship. Once the 2003–04 season was concluded, Caldwell was released by Newcastle United and by the time he left the club, he made thirty–seven appearances and scoring two times in all competitions. Sunderland Throughout the 2003–04 season, Sunderland wanted to sign Caldwell after their efforts to sign him, both on a free transfer and on loan failed. The club opted to wait to sign him on a free transfer at the end of the 2003–04 season. In May 2004, he agreed a move to Sunderland after leaving Newcastle United. The move was later confirmed on 27 June 2004, with the player signed a three–year contract. After missing out the first match of the 2004–05 season, Caldwell made his debut for the club, starting the whole game, in a 3–1 win against Crewe Alexandra on 10 August 2004. This was followed up by scoring his first goal for Sunderland, in a 3–3 draw against Queens Park Rangers. Since making his debut for the club, Caldwell became was known for his whole-hearted approach to the game and the solid partnership he formed with Gary Breen. Caldwell said that his aim was to help Sunderland reach the promotion to the Premiership, in which Manager McCarthy praised Caldwell for his effort. However, during a 1–1 draw against Wigan Athletic