Banking giant HSBC must deal with the "legacy of the past", George Osborne said as he indicated new laws could be needed to deal with tax dodgers.
The Chancellor said he had been unaware of allegations that the bank's Swiss arm had colluded in tax evasion at the time its former chairman Lord Green was appointed as a trade minister.
He defended Lord Green's government work, saying he had been a "very effective trade minister" and the appointment "went through the proper procedure".
Asked why just one prosecution had resulted from the files passed to HM Revenue and Customs (HMRC) about the tax affairs of people with accounts at the bank's Swiss subsidiary, Mr Osborne said: "I trust the judgments of our prosecuting authorities."
He told BBC Radio 4's Today programme: "If there are deficiencies in the law, if it's difficult to bring these prosecutions, then we should change the law."
The Chancellor said the Government was "looking at further changes to the law to make sure that we have a tough regime".
But he stressed it was an important constitutional principle that as Chancellor of the Exchequer he did not know about the tax affairs of individuals or interfere with the work of HMRC and other prosecuting authorities.
"I would say the allegations at HSBC, of course, are serious, they need to be thoroughly investigated, but that has to be done independently of the person who is the Chancellor of the Exchequer."
Asked about the appointment of Lord Green, who was trade minister from 2011 to 2013, Mr Osborne said: "The only thing that I was aware of was what the public was aware of.
"We were aware that HMRC had received from the French government, before we came to office, this information. But this was handled by our independent tax-collecting authorities."
Mr Osborne said the allegations about HSBC's role had only recently come to light, following the 2009 leak of account data by a whistle-blower.