Prime Minister Theresa May has been warned against a "hokey cokey Brexit" as Tory former ministers outlined their demands for the forthcoming negotiations.
Peter Lilley suggested it would be "foolish" to try to retain parts of the existing EU arrangements while also seeking to withdraw from others, claiming this would "tie our hands" when negotiating with the rest of the world.
He spoke at an event hosted by Leave Means Leave, a group which insists no deal with the EU is better than a "bad" deal.
Elsewhere, ex-Tory frontbencher Theresa Villiers said there will be "some difficult trade-offs" for Britain's financial services although they can remain the "biggest and best".
Free movement in the EU has to end in its current form, Ms Villiers added, as she pressed for action on immigration.
Their views add to the growing list of demands faced by Mrs May as she builds up to the start of the formal negotiating process to leave the EU, which is due to begin by the end of March 2017.
Speaking at a Conservative Party conference fringe event, Mr Lilley said: " The one thing we must get over is that leaving cleanly and clearly is not a hard Brexit but a simple, complete Brexit.
"Trying not really to leave, and retain one foot in and one foot out, is partial Brexit, it's a hokey cokey Brexit, and that's not in Britain's interests, it's not in the interest of democracy, where we have to show we've responded to the interests of the British people.
"It'd be a very foolish thing to do and it'd tie our hands as far as negotiations with the rest of the world are concerned.
"So I'm very optimistic that we'll get out with either a good deal or no deal, but either way it'd be better than the past."
Ms Villiers said Brexit was an opportunity to cut legislation and red tape regarding agriculture and fisheries, but warned protecting the financial centre of the City would pose one of the biggest challenges.
She told a separate fringe event: "It'll take a while for us to work out what our priorities are, but I think we need to look soon and with some urgency at farming and fisheries.
"Because even where we share some of the goals with current EU regulation - of course we want to keep our food safe, of course we want high standards of animal welfare, of course we want financial support for farmers, of course we want sustainable fishing practices - but the sheer volume and complexity of the regulation of those sectors.
"I'm sure we can come up with a system that delivers our regulatory goals in a much simpler way.
"I think one of the most challenging tasks ahead is the future of the City. I think the City has reinvented itself and many times adapted to change, and I believe it will continue to be the biggest and best financial centre not just in Europe but in the world.
"But there are some difficult trade-offs that we may have to consider."
She also told the Press Association: "I think on immigration the Prime Minister has made it pretty clear that this is a red line for her and I think she is right to do that.
"The referendum and the Leave vote was absolutely not just driven by immigration, but it was clear that part of the reason why a number of people were voting leave was because they believed we should restore national control over our immigration system.
"So if we are going to implement the result and abide by the wishes of the electorate, then I think we do have to see free movement end in its current form.
"That doesn't mean an end to immigration from Europe, of course EU citizens will still come and live and work in the UK and be very welcome to do so, but we do need to have a system which allows the final say on overall numbers to be made in this country."