The three women held captive for about a decade at a run-down house in the US were apparently bound with ropes and chains, police said.
Police later said in a press conference that Ariel Castro was charged with four counts of kidnapping and three counts of rape. No charges have yet been brought against his brothers Onil and Pedro.
Neighbours in the largely Puerto Rican neighbourhood in Cleveland, Ohio, said one of the brothers, 52-year-old Ariel, had taken part in the search for one of the missing women, performed music at a fundraiser for her and attended a candlelight vigil, where he comforted her mother.
The women's plight has riveted the US since 27-year-old Amanda Berry kicked through a screen door at the house on Monday, used a neighbour's telephone to call authorities and told a police dispatcher: "I'm free now." An officer showed up minutes later and Ms Berry ran out and threw her arms around the officer, a neighbour said.
Ms Berry arrived at her sister's home on Wednesday to a cheering crowd. Another of the captives, Gina DeJesus, about 23, returned to her family's home to chants of "Gina! Gina!" The third woman, Michelle Knight, 32, was reported as in good condition at a local hospital.
Neither Ms Berry nor Ms DeJesus spoke publicly, and their families pleaded for patience and time alone. "Even the ones that doubted, I want to thank them the most," Nancy Ruiz, Ms DeJesus's mother, said. "They're the ones that made me stronger, the ones that made me feel the most that my daughter was out there."
All three women had apparently been held captive in the house since their teens or early 20s, police said. Law enforcement officials left many questions unanswered, including how the women were taken captive, whether they were sexually abused and who fathered Ms Berry's six-year-old daughter.
Police spokesman Sammy Morris said ropes and chains were taken from the house.
Police chief Michael McGrath told NBC he was "absolutely" sure police did everything they could to find the women over the years. He disputed claims by neighbours that officers had been called to the house before for suspicious circumstances.
Mr McGrath said that the women were restrained and "released out in the back yard once in a while".