Dave Hartnett, who was head of tax at HM Revenue & Customs, said: "I'd always expected there to be be more."
Mr Hartnett, who left HMRC in 2012, was giving evidence to the Commons Public Accounts Committee.
HSBC has come under fire over what it knew about some clients' tax affairs.
Details of some 30,000 accounts at HSBC's Geneva-based private bank were leaked to the French authorities by a whistleblower, Herve Falciani, in 2007.
Information about 3,600 UK account holders were passed to HMRC.
Both HSBC and HMRC have been accused of not doing enough to investigate if these accounts were held for the purposes of tax evasion rather than legitimate avoidance.
Mr Hartnett told the committee: "I'd like to understand why there weren't more criminal prosecutions. I'd always expected there to be more. I'd have liked to understand what would have happened if  more resources were diverted to [investigating the leaked details]."
He said there were discussions with the French authorities before the leaked information was passed to the UK, and there were also discussions with HSBC. But he never got involved in the detail, "to pressure the integrity of the investigations taking place".
But HMRC tax assurance commissioner Edward Troup insisted that his department had been "diligent" in pursuing the information received from the French.
"I don't think there is anything... which suggests that we have not been diligent," he told the committee. "We have collected Â£135m. We believe we have collected all the tax, all the interest, and a significant amount of penalties."
Mr Hartnett now works for HSBC. He said that he had followed the rules for former senior officials on taking appointments outside the civil service "to the letter".
He said: "Colleagues checked it, the Cabinet Office checked it, the committee checked it, it went to the Prime Minister, and the Financial Conduct Authority also had to approve my appointment."
A former senior tax official has expressed "surprise" that just one person has been prosecuted in the UK over alleged tax evasion by clients of HSBC's Swiss banking division.