Early plans to create a giant "Big Brother" database holding information about every phone call, email and internet visit made in the UK were last night condemned by the Government's own terrorism watchdog.
Lord Carlile of Berriew QC, the independent reviewer of anti-terrorist laws, said the "raw idea" of the database was "awful" and called for controls to stop government agencies using it to conduct fishing expeditions into the private lives of the public.
Today the Home Secretary, Jacqui Smith, is expected to signal the Government's intention to press ahead with proposals to collect more details about people's phone, email and web-browsing habits as she warns that the terrorist threat to Britain is growing.
The controversial measure will be included as a way of combating terrorism in the Data Communications Bill, which is to be introduced in the Queen's Speech in December. Ministers are known to be considering the creation of a single database holding all the information, which would include phone numbers dialled and addresses to which emails are sent but not details of phone conversations or the contents of emails.
An increasing number of influential figures from across the political spectrum have expressed growing alarm over the scale of the proposals that would give the state unprecedented access into the lives of its citizens.
Lord Carlile described the government's recent track record on handling public data as an "unhappy one", and said that searches of a new database should only be carried out with the authority of a court warrant.
He told The Independent: "As a raw idea it is awful. However it is a question of degrees and how it is developed. Searches should be made on a case-by-case basis with appropriate reviewing measures so that they can't be done willy-nilly by government."
Under the proposal, internet service providers and telecoms companies would hand over millions of phone and internet records to the Home Office, which would store them for at least 12 months so that the police and security services could access them. It is understood that more than £1bn has been earmarked for the database.
Richard Thomas, the Information Commissioner, has described the plans as "a step too far for the British way of life". Yesterday his office added: "It is clear that more needs to be done to protect people's personal information, but creating big databases... means you can never eliminate the risk that the data will fall into the wrong hands."
Shami Chakrabarti, director of the human rights group Liberty, said: "This is another example of the Government's obsession with gathering as much information on each of us as possible in case it might prove useful in the future. Like the discredited ID card scheme this will have a massive impact on our privacy but will do nothing to make us safer."
Lord Carlile acknowledged the value of using phone and internet intelligence in fighting crime, but he said it would be wrong to go as far as the US Patriot Acts. "[They] go much further so that they [US data searches] include everyone who has made contact with a terror suspect... There must be codes of practice... In counter-terrorism collation is everything but raw data only has a limited use."
Dominic Grieve, the Shadow Home Secretary, said: "The Government must justify the case for any such massive increase in state acquisition, sharing and retention of data, spell out the safeguards to prevent abuse and – given its appalling record – explain how it will protect the integrity of any database holding sensitive personal data."
Chris Huhne, the Liberal Democrat home affairs spokesman, said: "Ministers simply can't be trusted with confidential data of this sort, as it has shown again and again."
Plans for the giant database were first announced by the Prime Minister in February under the unprepossessing title of the Intercept Modernisation Programme. It is not clear where the database will be held but GCHQ, the government eavesdropping centre, may eventually be the home for the project.
The proposal emerged as part of plans to implement an EU directive developed after the 7 July bombings to bring uniformity to record-keeping. Since last October telecoms companies have been required to keep records of phone calls and text messages for 12 months. That requirement is to be extended to internet, email and voice-over-internet use and included in a Communications Data Bill.
* Lord West of Spithead, the Counter-terrorism minister, last night said "another great plot" was being investigated by police and security services.
How the Government wants to watch over us
What is the Communications Data Bill?
It will allow the authorities to collect and retain details of every phone number we have called or texted, as well as every address to which we have sent emails and internet site we have accessed. The Government is preparing to announce its inclusion in the Queen's Speech legislative programme to be set out on 3 December.
What happens at the moment?
Under the Regulation of Investigatory Powers Act (RIPA), some 650 public bodies – including police and local authorities – can require internet service providers or mobile-phone companies to hand over details of their customers' phone, email and internet habits. About 500,000 requests were made last year. The rules say public bodies can only access records if it is for a legitimate purpose and proportionate – criteria which critics complain are too vague. Under a voluntary agreement, the internet service providers and phone companies store such records for a year. But they have protested about the huge pressure they are under to store such vast amounts of material.
How does the Bill change this?
It puts the retention of data on a statutory – rather than voluntary – basis and, crucially, paves the way for the information being transferred to a giant government database. The Bill also turns into British law an EU directive requiring companies to keep communications data for up to two years.
How do ministers justify the plans?
They say police and security services need more up-to-date tools in tackling terrorist and criminal conspiracies that are more international in nature and rely on high-tech communications. It is in the security services' interests to have a single point of access to track phone and internet records of suspects. It is harder to trace links between conspirators whose records are held by different companies.
Would contents of emails and phone calls be included?
No. Investigators can only intercept emails and tap phones under warrants approved by the Home Secretary. The aim is to monitor patterns of behaviour and establish links between conspirators.
What information might they recover?
Police or the security services could establish when a call was made and its length, as well as the number that was dialled. It is thought information could also be gathered as to the location of a mobile phone when a call was made. Information could be retrieved about when emails were sent and who the recipient was, as well as a full picture of internet sites visited.
How would access to such a database be governed?
The details are likely to be spelt out later this year by the Home Secretary. The Government promises there will be "strict safeguards" to "strike the proper balance between privacy and protecting the public". The likelihood is that the RIPA rules requiring requests for information to be approved by senior officers within public bodies will continue to apply.
How much electronic communication is there?
About one trillion emails and more than 60 billion text messages will be sent in Britain this year. Most homes and offices now have a computer; there are an estimated 20 million broadband connections.
How does this information help solve crime?
Email and telephone data has proved vital in the fight against al-Qa'ida. Mobile-phone location data was used in January in the conviction of Colm Murphy for his part in the Omagh bombing. Telephone records helped to send the serial killer Harold Shipman to jail.