Britain and Nigeria signed an agreement to repatriate convicted criminals, but activists said Nigerian inmates returning home will face inhumane prison conditions.
The agreement is in line with Prime Minister David Cameron's promise that UK taxpayers will stop paying for foreign criminals.
A British High Commission statement said prisoner repatriation would begin later this year under the "compulsory prisoner transfer agreement".
There are 521 Nigerians in British jails and just one Briton jailed in Nigeria, officials said.
British justice minister Jeremy Wright said at a ceremony yesterday that his country is committed to supporting prison reform in Nigeria. Britain has promised £1 million to help upgrade Nigerian jails.
"Both the United Kingdom and Nigeria have prisoner transfer arrangements with other countries and it is right that our two countries, which have such strong ties and shared interests, should develop our own arrangements," Mr Wright said.
Amnesty International says Nigerian prisons are "unliveable" with little food and inadequate sanitation.
Human rights lawyer Chino Obiagwu said any prison transfer would additionally burden an already overstretched system.
He said some prisons are more than double their capacity, few have adequate medical facilities, prisoners sleep on the floor and many die of preventable diseases.
Mr Obiagwu, director of the Legal Defence and Assistance Projects, said Nigeria's government has made no clear commitment to improving conditions.
There are between 60,000 and 65,000 people in Nigeria's prisons, which are overcrowded because thousands of suspects wait six and seven years to go to trial.
"Nigeria must start by improving its justice system if it wants to improve prison conditions," Mr Obiagwu said.
Amnesty International says the prison system is corrupt and prisoners who can afford it are forced to bribe wardens to allow relatives to bring them food and to have family visits.