The top seed claimed the first set 6-3, but the Austrian saved two match points in winning a tie-break 9-7 and then took the decider 6-4.
Thiem, who beat Federer on clay last month in Rome, will play for his fourth title of the season on Sunday.
He will face either Argentina's Juan Martin del Potro or Germany's Philipp Kohlschreiber.
Thiem, 22, moved into the world's top 10 for the first time this week and wore down his 34-year-old opponent in a gripping grass-court battle, which also included two rain breaks.
Federer, who had tied with Ivan Lendl for the most wins in the open era (1,071) after his second round win against American Taylor Fritz, had missed the French Open last month with a back problem.
The defeat will be a blow to Federer as he aims to win a record eighth Wimbledon singles title later this month.
Subscribe to the BBC Sport newsletter to get our pick of news, features and video sent to your inbox.
Roger Federer was beaten by French Open semi-finalist Dominic Thiem in the Mercedes Cup semi-finals at Stuttgart.