Mr Roache, who is secretary of the union's Yorkshire region, won 56.7% of the vote, while the only other candidate, Paul McCarthy, from the North West region, had 43.3%.
The union is the third largest in the UK with more than 600,000 members.
Current general secretary Paul Kenny announced in the summer he was standing down after almost a decade in the job.
Mr Roache has 35 years experience at the GMB and led the Leeds City Council 13-week refuse and street cleaning strike in 2009 - the longest in the union's history.
He said he was "proud and humbled" to have been elected.
"I will repay GMB members' faith in me by leading a 21st Century union that fights for our members, their families and communities, every hour of every day."
Mr Roache also paid tribute to Mr Kenny for his work "that has made GMB the envy of the union movement".
Details of the handover date have yet to be agreed.
The GMB is one of the three largest affiliates to the Labour Party and is a significant financial contributor to the party locally and nationally.
Regional official Tim Roache has been elected to become the new general secretary of the GMB union.