Nick Clegg faces being pulled away from the centre-ground of British politics by a Liberal Democrat "shopping trolley that defaults to the left", a minster he sacked has warned.
But he warned they should be fighting to win back popular support after years of dismal polls ratings rather than focusing on existing supporters.
"I was a very early and enthusiastic supporter of Nick Clegg," he said.
"I was enthusiastic about him taking us into Government. It was an act of leadership for which he gets insufficient credit. So it's disorientating to receive that phone call and I suppose a bit puzzling. A bit painful."
Mr Browne's surprise departure from Government prompted speculation that he could defect to the Conservatives but he told The Times he rejected an invitation for a meeting with Tory chairman Grant Shapps - "one, because I had no intention or desire to defect to the Conservatives... and secondly, that it could and probably would be misconstrued if I had a meeting at all".
He added: I'd say my ambition for the Lib Dems is to attract the small 'l' Liberals in the Conservative and Labour parties.
"I regard it as a disappointment that we have not attracted more people of a liberal disposition in other political parties."
Mr Browne said he viewed his role as "doing everything I can to accelerate the Lib Dems' journey from a party of protest to a party of government".
"We must avoid facing inwards to try to reassure the 9% rather than facing outwards and trying to talk to the 91%," he told The Times.
"If the Lib Dems look ambivalent about being in government, we can hardly then complain if people assume that its successes must be down to the party that doesn't look ambivalent.
"You can't be half in and half out of government. We have to try to avoid the trap of looking like a party that is a reluctant party of government and looking uncomfortable and that we'd be grateful to be relieved of our collective responsibilities."