The head of the Army has admitted it was "extraordinarily difficult" to control the base where five Libyan soldiers accused of sex crimes were stationed.
General Sir Nicholas Carter said the alleged attacks were " completely beyond the pale" and "absolutely regrettable" as he appeared before the Commons defence select committee.
More than 300 members of the Libyan armed forces training at Bassingbourn Barracks in Cambridgeshire are being sent home following the allegations.
The Chief of the General Staff said he knew that some of the Libyans had sought political asylum.
Asked if thought that was appropriate, he replied: "No, probably not."
Members of Libya's armed forces have been based at the barracks since July as part of the UK's pledge to support the Libyan government.
The group of soldiers was expected to return home at the end of the month but a ll will have gone back "within days", Gen Carter said.
He told MPs : "I absolutely agree with you that those who have gone off and are alleged to have done what they have done is completely beyond the pale.
"The Bassingbourn site is not a prison camp.
"It is extraordinarily difficult to control it, in that sense.
"From our perspective, we've done everything that we have tried to do to motivate and to be focused entirely on training.
"Indeed, we have run an extremely tough walking out policy in conjunction with the Home Office who have helped in all of this and the upshot of it is that it's absolutely regrettable that this has occurred."
He added: "As far as we are concerned we have done as much training as we can in the circumstances and we are sending people back to Libya who are better soldiers than they were when they started the training."
Gen Carter appeared to distance himself from the decision to send Libyan troops to the Cambridgeshire base.
"I think to be fair to me as the service chief who provides the trainers, I was not involved in the making of the policy that suggested that Bassingbourn was the right solution to all of that," he said.
British forces have found training the Libyan recruits "challenging", the committee was told.
Gen Carter said: " For them, seeing their country in the state it now is, and of course it has got worse over the course of the last two or three months, has been quite destabilising.
"So, I think that trying to control them to focus entirely on training and what we're asking of them has been quite challenging.
"So, I think that none of us are particularly surprised that a few of them have, if you like, found it really difficult to do it."
Libyan cadets Ibrahim Naji El Maarfi, 20, and Mohammed Abdalsalam, 27, appeared at Cambridge Magistrates' Court last week and admitted two counts of sexual assault.
They are awaiting sentencing.
Khaled El Azibi, 18, has been charged with three counts of sexual assault linked to the same incident but has yet to enter a plea.
Moktar Ali Saad Mahmoud, 33, and Ibrahim Abogutila, 22, have also been charged with rape.
They appeared before the city's magistrates yesterday morning where their case was adjourned for them to appear at Cambridge Crown Court next Tuesday.
They were remanded in custody.
Libyan soldier Omar Al-Mukhtar, not one of the accused, told the BBC that his colleagues think the men concerned were badly treated.
He said the Libyan cadets were allowed out for only three hours a week and were always accompanied by British soldiers when they left the barracks.
He added that when soldiers left the base they had been offered drugs, alcohol and sex for money.
He said there had been "several problems" between the Libyan cadets and British soldiers.
Mr Al-Mukhtar said he and other Libyans were proud to have graduated from the training course at Bassingbourn, and they were very happy with the training they had received.
He described it as being of a high standard, and said: "Even the generals here say we did really well."
However, the cadets were unhappy with the way they had been treated by the British Government, which he said had "not offered a comfortable way of living here [at the barracks]", and that some people were "trying to ruin the reputation of the Libyan Army".
Asked why the cadets were being flown home two weeks before the end of their course, he said: "I feel, like all the rest of us, there is no problem.
"It was the British from the beginning. They should have sought a solution and finished the training well.
"They didn't tell us about British law and what's the difference between right and wrong here."