Hundreds of children in custody were already among the "most vulnerable individuals in our society" before they were jailed, and have been continually let down by agencies whose job it is to care for them, doctors have warned.
The Government needs to review the youth prison estate with a view to exploring alternatives to custody, the British Medical Association (BMA), which represents doctors and medical students in the UK, said.
The BMA also called for an end to holding children and young people aged 17 and under overnight in police cells.
But failure by individuals and agencies to protect children makes it "inevitable" some will end up in custody, the BMA added.
A new BMA report, "Young Lives Behind Bars: The health and human rights of children and young people detained in the criminal justice system", said many children in detention reflect a failure by society to protect them.
Dr John Chisholm, chair of the BMA's Medical Ethics Committee, said: "Every child deserves the help and support necessary to preserve their physical, psychological, and emotional health and well-being.
"But for the thousands of children and young people in the UK who come into contact with the criminal justice system each year, this is not always the reality.
"Many of these young people come from chaotic home lives, often characterised by violence, abuse or neglect, and are not thriving socially, emotionally or physically.
"Long before they reach detention they are among the most vulnerable individuals in our society and have been continually let down by the individuals and agencies whose job it is to care for them and support them.
"Despite their high level of need, inadequacies in the systems that these children pass through sadly mean that they often fall between the cracks, and time spent in custody becomes an almost inevitable consequence.
In 2012/13, the average population in England and Wales of young people in custody - under 18s - was 1,544.
Three quarters of children in jail have lived with someone other than a parent and 40% had been homeless in the six months before entering custody, the BMA found.
Some 24% of boys and 49% of girls, aged between 15 and 18 and in custody, have been in care, the report said.
Around 60% of children in detention have "significant" speech, language and learning difficulties, the BMA said, while 25 to 30% are learning disabled and up to 50% have learning difficulties.
More than a third of children in custody were diagnosed with a mental health disorder, the report added.
The report also raises concern over the low age of criminal responsibility in place in the UK and the suitability of imprisonment in dealing with youth offending.