But Manuel Pellegrini's men paid the price for failing to convert a series of other chances as Juve hit back with a Mario Mandzukic finish and a stunner from Alvaro Morata.
It was yet another example of City failing to deliver on the biggest stage, and was actually a surprise given the form of the two sides coming into the game.
Juve, last season's runners-up and Serie A champions for four successive years, had failed to win any of their opening three games while City had won five in a row without conceding.
But without star striker Sergio Aguero - who made a surprising late appearance off the bench having initially been ruled out with a knee injury - City lacked a cutting edge.
Raheem Sterling spurned an early chance after Fernandinho robbed Stefano Sturaro. The former Liverpool man opened his body and tried to place his shot inside the far post but veteran Gianluigi Buffon saved with his feet.
It was some time before City threatened again as Juventus gained a grip with £70million-rated Paul Pogba - a reported target for City last summer - strong in possession and on-loan Chelsea winger Juan Cuadrado looking dangerous.
Pogba slid the ball into the net from a Cuadrado cross but Morata was flagged offside.
City had a good spell as Fernandinho won the ball from Pogba and fired over before Aleksandar Kolarov sent a free-kick across goal.
Wilfried Bony then twice fluffed his lines by slicing horribly wide after a smart run and then failing to turn in a low Kolarov cross. Fernandinho had another good chance when a clearance fell to him and sat up nicely but his powerful drive flew wide.
The second half began slowly but changed completely after City grabbed the lead.
Kompany celebrated as if he had claimed his third goal of the season as he rose to meet a David Silva corner but the ball went in off Chiellini. Chiellini felt he had been impeded by Kompany and protested to the referee, but to no avail.
The goal energised City and moments later Silva broke free and played in Sterling. Buffon saved well and then quickly regained his footing to produce an even better stop to deny the inrushing Silva. Bony fired in another shot from a tight angle but again Buffon saved.
City were made to rue those misses as Juve responded. Joe Hart needed to save from Sturaro and City had a big let-off when Pogba miscued a volley.
But the Italians broke through when Pogba swung in a cross for Mandzukic and the Croatian timed his run to perfection to guide the ball in via the far post.
City manager Manuel Pellegrini surprisngly opted to take off the influential Kompany at this point and hand a debut to Nicolas Otamendi. The City skipper had shown no obvious signs of injury and did not look pleased.
Otamendi was caught out as Morata fired Juve ahead nine minutes from time, but it was all about the finish.
Morata broke clear on the left and curled an unstoppable shot round Hart and inside the far post.
TWEET OF THE MATCH
Gary Lineker @GaryLineker: Chiellini fuming at City's opening goal. He's either upset at Kompany leaning on him or the fact he wasn't given credit for the OG.
Joe Hart: 5 (out of 10)
Bacary Sagna: 6
Vincent Kompany: 7
Eliaquim Mangala: 6
Aleksandar Kolarov: 5
Yaya Toure: 6
David Silva: 7
Raheem Sterling: 5
Samir Nasri: 4
Wilfried Bony: 4
Kevin De Bruyne: 5
Nicolas Otamendi: 5
Sergio Aguero: 6
Gianluigi Buffon: 8
Patrice Evra: 6
Giorgio Chiellini: 8
Leonardo Bonucci: 7
Stephan Lichsteiner: 6
Paul Pogba: 8
Stefano Sturaro: 7
Alvaro Morata: 7
Juan Cuadrado: 8
Mario Mandzukic: 6
Paulo Dybala: 5
Andrea Barzagli: 6
Gianluigi Buffon. The various junctures at which City lost this game will be picked over but if they had taken more of their chances they could have won comfortably. Buffon was principally responsible for denying them, producing several good saves including a fine double stop from Raheem Sterling and David Silva in the second half.
MOMENT OF THE MATCH
Alvaro Morata. Juve won the game with quite a brilliant goal from Morata. Questions may be asked of City's defending in the immediate preceding moments but Morata had no right to score from his position on the left. His shot was curled around Joe Hart to perfection.
VIEW FROM THE BENCH
City boss Manuel Pellegrini seemed to have got things right as his side, sent out in familiar formation, took control of the game. But they failed to see out the job and opposite number Massimiliano Allegri will be highly relieved after his side's frustrating start to the season.
MOAN OF THE MATCH
City's critics will no doubt point out the stadium was not sold out for another Champions League glamour night, but there are always mitigating factors. £30 is pricey for many fans, midweek nights are inconvenient, especially given Manchester's current traffic problems and city centre diversions. There was only a small take-up from Juventus fans and the match was on TV.
Raheem Sterling focus
Raheem Sterling forced through a move from Liverpool to Manchester City in the summer, partly to satisfy his ambitions of playing regular Champions League football.
We tracked his first appearance for City in the competition.
Handed a brilliant chance after just two minutes when Yaya Toure picked out his overlapping run. Sterling opened his body and took aim but his effort was too close to Gianluigi Buffon. Spurned a second presentable chance in the second half, again making life too easy for the goalkeeper.
Started the game on the left flank but switched back and forth on a couple of occasions as City sought an outlet for his pace. Even drifted into the pocket behind striker Wilfried Bony during the second half but never really found himself with space to attack.
Sterling's fleetness of foot meant he was well able to get back and help when required but the nature of the game meant City were more keen for him to link up with Bony than shadow the full-backs. Worked doggedly to win the ball in a two-on-one with Stephan Lichtsteiner and Juan Cuadrado.
Never the kind of player to work himself into a demonstrative frenzy, the winger kept on an even keel, even when his two cracks at goal proved unsuccessful. No dramas when he made way for Kevin de Bruyne.
COMMENT RULES: Comments that are judged to be defamatory, abusive or in bad taste are not acceptable and contributors who consistently fall below certain criteria will be permanently blacklisted. The moderator will not enter into debate with individual contributors and the moderator’s decision is final. It is Belfast Telegraph policy to close comments on court cases, tribunals and active legal investigations. We may also close comments on articles which are being targeted for abuse. Problems with commenting? firstname.lastname@example.org