Criminalising paying for sex in Northern Ireland could threaten the safety of prostitutes, Justice Minister David Ford said.
The clause in Lord Maurice Morrow's draft Stormont legislation would be difficult to enforce and the police do not support it, the minister added.
If a prosecution was undertaken the woman risked being intimidated by the purchaser of sex for going to the authorities.
Lord Morrow has insisted a new prostitution law is vital to effectively tackle human trafficking, arguing that outlawing payment would simplify the law and make it easier to secure convictions that send a clear message to offenders.
Mr Ford told the Assembly: "I am not prepared to support a proposal which might not work and which might take risks with the lives and well-being of other vulnerable individuals caught up in prostitution."
Since April this year, 13 potential human trafficking victims have been referred for specialist support.
Lord Morrow has argued that his proposed legislation seeks to reduce demand for sexual services, a major driver for trafficking.
Mr Ford said: "The prosecution service sees difficulties in evidencing successful cases. Many of the key voluntary groups also have reservations, spelt out to me in meetings over the summer. As Justice Minister I cannot ignore these voices."
He said the clause may take risks around the lives and well-being of other vulnerable individuals caught up in prostitution.
Mr Ford also expressed concern that part of the draft law setting a two-year minimum sentence for trafficking and slavery offences could fetter the discretion of judges, who he said were best placed to consider the broad scope of circumstances in any particular case.
"I believe that sentencing guidance - which is already in place in respect of cases of trafficking for sexual exploitation - provides a more appropriate vehicle than primary legislation, to respond flexibly to case law as it emerges," he added.
According to Lord Morrow, under the draft law a child victim would not be punished for crimes which were the direct consequence of trafficking. A child trafficking guardian would be established to speak up for victims.
Lord Morrow's proposed legislation, the Human Trafficking and Exploitation Bill, contains 19 clauses updating Northern Ireland's laws on prostitution and trafficking.
It will be considered by an Assembly committee.