America's National Security Agency reported its own breaches of surveillance rules to an intelligence court and repeatedly promised extra measures to prevent similar incidents, newly-declassified phone-tap files have revealed.
More than 1,000 pages detail the US government's controversial programme of collecting every citizen's phone records during the past seven years.
According to court records from 2009, after repeated assurances the NSA would obey the court's rules, it acknowledged that it had collected material improperly.
In one instance, the government said its violations were caused by "poor management, lack of involvement by compliance officials and lack of internal verification procedures, not by bad faith". In another case, the NSA said it improperly collected information due to a typographical error.
The intelligence court judge, US District Judge John Bates, said in the 2009 case that since the government had repeatedly offered so many assurances despite the problems continuing, "those responsible for conducting oversight at the NSA had failed to do so effectively". He called his conclusion "the most charitable interpretation possible".
The Obama administration published the heavily-censored files today as part of a continuing civil liberties lawsuit challenging the constitutionality of the government's collection of phone records, which the White House has said is important to countering terrorism. The files were so heavily censored that one of the two justifications for the government to search through Americans' phone records was blacked out.
The latest release reflects the administration's strenuous efforts to maintain its legal authority to collect Americans' phone records amid opposition in Congress.
Meanwhile, in a legal victory for the administration, the Supreme Court refused to intervene in the NSA controversy.
It rejected a call from a privacy group to stop the agency from collecting the telephone records of millions of Verizon customers in the United States. While the justices yesterday declined to get involved in the issue, other lawsuits on the topic are making their way through the lower courts around the US.
In the new disclosures, some files were declassified ostensibly to show that even when NSA employees collected records improperly or improperly shared material among themselves, those problems were reported to the intelligence court and new procedures were put in place to prevent them from happening again.
Similar documents about the US collecting phone records were previously declassified and published in response to a lawsuit filed by the Electronic Frontier Foundation, and the Obama administration has revealed others to convince Congress to allow it to continue collecting the phone records.
After the NSA began the bulk collection program in 2006, one NSA inspector general's report said rules already in place were "adequate" and "in several aspects exceed the terms" of what the Foreign Intelligence Surveillance Court had required.
But it recommended three additional practices be formally adopted. These included such obvious ideas as not allowing analysts who searched phone records in the terror database also to approve which numbers can be searched, and periodically checking the phone numbers that analysts searched to make sure they had actually been approved.
Despite the assurances in 2006 that rules were adequate, problems surfaced in 2009 that were so serious that the intelligence court temporarily shut down the surveillance programme.
One of the newly-disclosed files was a swipe - intentionally or otherwise - at Senator Patrick Leahy, chairman of the Senate Judiciary Committee who is leading the fight on Capitol Hill to rein in the government's phone records collection.
In a ruling by the then-chief judge of the intelligence court, Colleen Kollar-Kotelly, to justify the programme, she quoted a 2001 floor speech by Mr Leahy to explain that Congress believed that phone records could be collected under US laws.
Mr Leahy has proposed ending the NSA's sweep of phone records, allowing the government to seek only records related to ongoing terror investigations.
The documents included training materials for NSA analysts, who were warned that they should search only the database of all phone records for numbers they suspected were associated with terrorists: "Analysts are NOT free to use a telephone selector based on a hunch or guess," according to a 2007 training presentation. It added that the NSA's legal standard for picking a phone number for a terror suspect required "some minimal level of objective justification".
The training slides noted that the government should not snoop on the phone records of Americans whose only suspicious behaviours were protected by the First Amendment, such as speaking or writing in opposition to the US government, worshipping at a mosque or working as a journalist.
"A telephone selector believed to be used by a US person shall not be regarded as associated with (censored) solely on the basis of activities protected by the First Amendment," it said.
The administration has been under pressure to reveal more details about the government's domestic surveillance programme since former intelligence contractor Edward Snowden released documents showing massive trawling of domestic data by the National Security Agency.