In 657, following Empress Wu's directions, Xu and Li Yifu accused Han and Lai (who were still chancellors at this point but had basically lost power) of conspiring with Chu to rebel.
In 662, Emperor Gaozong made Xu an advisor to the new crown prince as well as de facto chancellor of the first class, while continuing to exercise actual authority over the legislative bureau.