1950–51 season he scored 20 goals in 36 league games, but Southampton's defence leaked too many goals and they finished in mid-table. The following season started in similar vein, and Brown maintained his scoring ratio with 12 goals in 21 games, until injury meant his season – and his Saints career – came to an end in January 1952. Brown had failed to settle at Southampton, despite scoring 34 goals in 59 starts while at the club, and in March 1952, having lost his place to Walter Judd, he was granted a transfer to Coventry City of the Third Division (South), where he continued to score goals at an impressive rate. Birmingham City In October 1954, following a run of five games without a win, Coventry sold him to Birmingham City of the Second Division for £9,000, a decision which provoked the resignation of Coventry's manager Jack Fairbrother. Brown's career at Birmingham coincided with probably the best period in the club's history. He arrived in mid-October 1954, and in that first part-season scored 14 goals in 28 League games, including a hat-trick in a 9–1 demolition of Liverpool which remains their record defeat. His goals helped Birmingham to the 1954–55 Second Division championship. The following season, 1955–56, they achieved their highest ever finishing position, sixth in the First Division; Brown, playing alongside Peter "Spud" Murphy and Welsh international Noel Kinsey and with Alex Govan and England international Gordon Astall on the wings, finished top scorer with 21 League goals. He scored another seven in the run which took the club to their second ever FA Cup Final, only to lose 3–1 to a Manchester City side inspired by Don Revie. This was the match best remembered for Manchester City's goalkeeper Bert Trautmann breaking a bone in his neck and still finishing the game. In 1956–57 Brown scored 20 goals in all competitions and played in the semi-final of the FA Cup, losing to Manchester United's Busby Babes. He was also a pioneer of European competition, as part of the Birmingham side which reached the semi-final of the Inter-Cities Fairs Cup 1955–58, where he scored two goals in the 4–3 home leg win over Barcelona before Birmingham eventually lost out in a replay. His last full season at Birmingham, 1957–58, produced another 15 League goals. Later career He moved on to Second Division Leyton Orient in December 1958, where despite arriving