Gordon Brown insisted he would carry on as Prime Minister tonight, adding: "If I didn't think I was the right person leading the right team ... I would not be standing here."
He admitted Labour had plunged to "a painful defeat" in yesterday's elections.
But he added: "I will not waver. I will not walk away. I will get on with the job."
He said the current political crisis, fuelled by the Westminster expenses scandal "is a test of everyone's nerve - mine, the Government's the country's".
The premier was speaking at a Downing Street news conference after completing a Cabinet re-shuffle.
He vowed to fight on, saying: "If I didn't think I was the right person to lead these challenges I would not be standing here.
"I have faith in doing my duty ... I believe in never walking away in difficult times."
Mr Brown emphasised his determination to carry on despite the calls from within the Labour Party for him to stand down.
"Our party cannot lead or succeed by heeding the empty and expedient reactions of the hour," he said.
"Yes it was a tough night but we fight on for what we believe."
Mr Brown said: "I accept my responsibility. I'm not going to walk away from my duty to the country."
He insisted the Government needed to "clean up politics" and "take the country through the economic recession".
"I'm not complacent," he went on. "I have never been arrogant. I'm not doubting we have suffered a big defeat. But the task facing this country is so great we have got to get on with the job.
"I think people will see we are ready to deal with the problems, not shirking these responsibilities and getting on with the job."
Mr Brown acknowledged that it was a "testing time" for the Government in the face of the twin crises of the economy and parliamentary expenses.
He said it was now the Government's task to clean up politics, push forward with economic recovery, and carry forward with public service reform.
Three new ministerial councils - the Democratic Renewal Council, the Domestic Policy Council and an enhanced National Economic Council - would report weekly to the Cabinet.
He promised an extensive public consultation on political reform with legislation for a new independent regulator for MPs and tough legally binding code of conduct.
On the economy, he said there would be a new infrastructure plan, a plan for green jobs, more action on housing.
And he said ministers would be bringing forward new plans on education, health and policing in the next few days.
The Prime Minister acknowledged: "Over the expenses, Parliament should have acted earlier, all of us should have acted earlier. I accept, and apologise for, our failure to do so."
He went on: "I'm candid because I admit that there have been mistakes. I accept full responsibility, but I'm going to get on with the job. Nobody walks away when you are in difficulty.
"You do your duty by your country."
Asked if he had ever considered quitting, Mr Brown replied: "No. But I'm not arrogant and I'm never complacent.
"There's a job to do and I'm going to keep on doing it.
"I have always been brought up to believe that you have got to act with integrity at all times.
"When I look at these expenses claims, I remember the words of my father - always be honest, always sort things out if things are wrong."
Mr Brown defended Chancellor Alistair Darling as a "very good personal friend" and said the idea that the pair were split over the economy was "ridiculous".
He said: "Alistair Darling, as I said in the House of Commons only two days ago, has served the country not only well domestically but is internationally acclaimed for much of the work he did in relation to the G20.
"I was asked this in the Commons only on Wednesday and I paid tribute not only to his work in the past, but his work in the present, and the work that he is doing to bring about the economic recovery."
He added: "Alistair Darling is not only a very good personal friend of mine and I have known him for many years... but he has also been a great Chancellor.
"Look, we are going through a great economic crisis. It is unprecedented. The idea that we were to split over this issue of working through the economic problems and getting solutions is just ridiculous.
"Alistair and I are not only friends, colleagues, but have been working in the G20 to get answers. The respect that he has in the rest of the world is something that I hope the rest of the country will soon be able to acknowledge."
Mr Brown defended his Cabinet changes and said there were seven women who would be attending Cabinet who were all "very talented".
He added: "Lord Adonis has proved himself to be an excellent minister of state and... equally Bob Ainsworth has been a loyal minister of state at the Department of Defence. He is highly regarded by the Generals and all the commanding officers and will make an excellent Secretary of State for Defence.
"Look, you make changes as things develop, but you make changes for a purpose. The changes I have made are for these three purposes.
"We have got a new Constitutional Renewal Council that is going to deal with the problems of cleaning up politics and democratic reform.
"We have got a new enhancement of the Economic Council by what we are doing with the new departments... and of course we have a new national Domestic Policy Council.
"This is focused, it's determination, it's energy, and I do think that, when you look at the new people who join the Cabinet, you will see we have chosen very talented people who have got a determination to serve the country."
Full list of the new Cabinet:
Prime Minister, First Lord of the Treasury and Minister for the Civil Service
The Rt Hon Gordon Brown MP
Leader of the House of Commons and Lord Privy Seal; Minister for Women and Equality (and deputising for the Prime Minister at PMQs)
The Rt Hon Harriet Harman QC MP
First Secretary of State, Secretary of State for Business, Innovation and Skills and Lord President of the Council
The Rt Hon Lord Mandelson
Chancellor of the Exchequer
The Rt Hon Alistair Darling MP
Secretary of State for Foreign and Commonwealth Affairs
The Rt Hon David Miliband MP
Secretary of State for Justice and Lord Chancellor
The Rt Hon Jack Straw MP
Secretary of State for the Home Department
The Rt Hon Alan Johnson MP
Secretary of State for Environment, Food and Rural Affairs
The Rt Hon Hilary Benn MP
Secretary of State for International Development
The Rt Hon Douglas Alexander MP
Secretary of State for Communities and Local Government
The Rt Hon John Denham MP
Secretary of State for Children, Schools and Families
The Rt Hon Ed Balls MP
Secretary of State for Energy and Climate Change
The Rt Hon Ed Miliband MP
Secretary of State for Health
The Rt Hon Andy Burnham MP
Secretary of State for Northern Ireland
The Rt Hon Shaun Woodward MP
Leader of the House of Lords and Chancellor of the Duchy of Lancaster
The Rt Hon Baroness Royall of Blaisdon
Minister for the Cabinet Office, and for the Olympics and Paymaster General
The Rt Hon Tessa Jowell MP
Secretary of State for Scotland
The Rt Hon Jim Murphy MP
Secretary of State for Work and Pensions
The Rt Hon Yvette Cooper MP
Chief Secretary to the Treasury
The Rt Hon Liam Byrne MP
Secretary of State for Wales
The Rt Hon Peter Hain MP
Secretary of State for Defence
The Rt Hon Bob Ainsworth MP
Secretary of State for Transport
Secretary of State for Culture, Media and Sport
Ben Bradshaw MP
Other Cabinet attendees
Chief Whip (Parliamentary Secretary to the Treasury)
The Rt Hon Nick Brown MP
Minister of State, Foreign and Commonwealth Office
The Rt Hon Lord Malloch-Brown
Minister of State (Housing), Department for Communities and Local Government
The Rt Hon John Healey MP
Minister of State (Business), Department for Business, Innovation and Skills
The Rt Hon Pat McFadden MP
Minister of State (Science and Innovation), Department for Business, Innovation and Skills
The Rt Hon Lord Drayson
Attend Cabinet when their Ministerial responsibilities are on the agenda
The Rt Hon Baroness Scotland of Asthal QC
Minister of State (Children), Department for Children, Schools and Families
The Rt Hon Dawn Primarolo MP
Minister of State (Employment), Department for Work and Pensions
The Rt Hon Jim Knight MP
Minister of State (Regional Economic Development and Co-ordination) Department for Business, Innovation and Skills
The Rt Hon Rosie Winterton MP
The Queen has accepted the following resignations:
The Rt Hon Hazel Blears MP
The Rt Hon Geoff Hoon MP
The Rt Hon John Hutton MP
The Rt Hon Paul Murphy MP
The Rt Hon James Purnell MP
The Rt Hon Jacqui Smith MP
The Rt Hon Tony McNulty MP
The Rt Hon Margaret Beckett MP
The Rt Hon Beverley Hughes MP