Ronny Deila admits Celtic deserved to be punished by Kilmarnock who battled back twice to draw the Ladbrokes Premiership clash 2-2 at Rugby Park.
Hoops striker Leigh Griffiths opened the scoring with a drive before two minutes had elapsed and the visitors should have had the game sewn up before the interval.
Killie's Northern Ireland striker Josh Magennis equalised just before the break with the home side's first goal of the season, making up for an earlier blatant miss.
It looked like the points were heading back to Glasgow after Nir Bitton lashed the ball in from close range - until four minutes from time when Kilmarnock were awarded a spot-kick after Celtic defender Emilio Izaguirre brought down Greg Kiltie inside the box.
Kallum Higginbotham cheekily dinked the ball over Craig Gordon for Killie's first point of the season and Parkhead boss Deila, who refused to talk about Southampton's reported interest in defender Virgil van Dijk, said: "I am disappointed of course.
"We should have scored more goals but I am very disappointed that we were sloppy in defence.
"It was like that against Partick Thistle in the second half and we continued that and we got punished.
"They got four or five chances and that is more than the other teams have had in 10 games.
"So we deserved to get punished when we don't score goals and are so sloppy.
"Everything starts with good defence, that is why we have had so many good results - and we have to be back on track on Saturday (at home to Inverness)."
Gary Locke apologised to the Killie fans after the crushing 4-0 opening day defeat at home by Dundee.
The Rugby Park boss, who revealed winger Chris Johnston is out for up to nine months with a cruciate ligament injury sustained in training on Tuesday, was delighted there was no repeat.
He said: "I was glad that we turned up tonight.
"I brought a number of players to the club and the last time here we didn't turn up.
"I had to apologise to the fans but thankfully I didn't have to do that tonight.
"I felt we showed the qualities we have got and hopefully that will give players belief to go on and have a good season."
On Johnston's injury, the former Hearts boss said: "I am gutted for the lad. It is going to be a hard eight to nine months for him but we will be there for him."
Higginbotham, who joined in the summer after leaving Partick Thistle, said: "I was always going to dink my first penalty if I got the chance to take it, no matter who it was against or what time of the season.
"Luckily it came early in the season against the champions.
"I had a bit of doubt with Craig Gordon filling the net but as soon as I hit it I knew it was going in."
Higginbotham's late penalty was a cheeky as the come, beating Scotland keeper Gordon with a neat chipped effort.
Griffiths opened the scoring right at the start and the Hoops should have had the game done and dusted before the interval.
Instead, Deila's side went in level after Magennis equalised with the home side's first goal of the season, making up for an earlier shocking miss.
Bitton arrowed in a tremendous 30-yard shot in the 55th minute to restore the champions' lead.
However, with four minutes remaining the home side were awarded a spot-kick when Izaguirre brought down Kiltie and Higginbotham cheekily dinked the ball over Gordon for Killie's first point of the season, with Celtic dropping their first points of the campaign.
Celtic are in second place, two points behind Hearts after three games.
Although this surprise result won't stop the title ending up at Parkhead, it was certainly food for thought for Deila ahead of next week's Champions League clash at home to Malmo.
Kilmarnock: MacDonald, Smith, Hamill, Connolly, McKenzie, Robinson (Splaine 66), Higginbotham, Westlake, Ashcroft (O'Hara 42), Kiltie (McLean 92), Magennis. Subs Not Used: Davies, Barbour, Hodge, Hawkshaw.
CELTIC: Gordon, Janko, Ambrose, Van Dijk, Izaguirre, Bitton (McGregor 85), Johansen, Commons (Mackay-Steven 29), Rogic, Forrest (Mulgrew 60), Griffiths. Subs Not Used: Bailly, Fisher, Henderson, Stokes.
Kilmarnock goals: Magennis 44, Higginbotham 88 (pen)
Celtic goals: Griffiths 3, Bitton 55
Man of the match: Greg Kiltie
Match rating: 8/10
Referee: Crawford Allan