The taxman could owe British workers £3bn in overpaid taxes dating back to 2007-08 and earlier, according to a report.
And a backlog of underpaid taxes from four or more years ago could lead to HM Revenue and Customs demands for an extra £1.4bn from taxpayers, said spending watchdog the National Audit Office.
The problem could affect as many as 15 million people, many of whom may not know that they are facing additional tax demands or due repayments, which could run into thousands of pounds.
The NAO said the backlog was caused by the troubled introduction of a new computer system which was intended to combine National Insurance and PAYE income tax payments into one record for the first time.
By the time of its implementation in June 2009, the system had been delayed twice, leading to a backlog of 18.2 million unreconciled cases involving 15 million people, the NAO report said yesterday
The NAO found that the delays were "justified" as the system was not ready to go live earlier. But the slippage resulted in an additional cost of £33m, and the loss of £55m in planned efficiency savings. Changes to system requirements added a further £78m, bringing the total cost of the project - expected to save £532m over five years - to £389m.
The report said that the task of clearing the backlog remains "a very significant challenge".