G4S is waiting to hear whether fraud investigators are to launch a criminal inquiry into the security giant after it refused to co-operate with the Government over the tagging contract scandal.
The company - well-known for its botched handling of its Olympics security contract - and rival security firm Serco overcharged the Government by tens of millions of pounds, including for monitoring dead offenders.
Serco, meanwhile, has agreed to take part in the independent audit which will look at, among other areas, internal email trails between executives to establish what happened.
Auditors discovered the firms had charged the Government for tagging offenders who were back in prison, had had their tags removed, had left the country or had never been tagged in the first place but had been returned to court.
Justice Secretary Chris Grayling told the House of Commons: "The billing practices in question were clearly unacceptable and the Government will take all necessary steps to secure a refund for the taxpayer."
He added: "This is a wholly indefensible and unacceptable state of affairs. The House will share my astonishment that two of the Government's biggest suppliers would seek to charge in this way.
"The House will also be surprised and disappointed to learn that staff in the Ministry of Justice were aware of a potential problem and yet did not take adequate steps to address it."
Mr Grayling admitted that contract managers in the Ministry of Justice discovered issues with the contract as far back as 2008 - but did nothing to tackle the problem.
A Government-wide review of all contracts held by Serco and G4S, which both saw their shares plunge on London's leading FTSE 100 index on Thursday, will take place following the shock revelations.