Mr Garner will join Nationwide in spring 2016 to replace current chief executive Graham Beale, who is retiring.
He will leave Openreach, the fixed-line network provider, after almost two years in charge.
Mr Garner said the Nationwide job was "too good to turn down".
He said: "I've greatly enjoyed my time at Openreach and deciding to leave has been an extremely hard decision, but Nationwide is a firm I long-admired during my time in financial services."
Mr Garner was previously head of HSBC UK, where he worked with banking clients including First Direct and Marks and Spencer Bank.
Nationwide chairman David Roberts said: "Joe is an exceptional leader and is exactly the right person to lead the society through the next chapter of its journey."
Gavin Patterson, the BT chief executive, said: "I am sad to see Joe leave but he is moving to a terrific job and I wish him well."
His departure comes as communications regulator Ofcom investigates whether Openreach should be split off from BT.
Rival internet providers argue BT has an unfair competitive advantage from owning the telecoms infrastructure business.
Most internet service providers offer services through the pipes and cables run by Openreach.
Nationwide building society has appointed Joe Garner, currently boss of BT's Openreach unit, as its new chief executive.