City fell behind early on to a Cani strike, but Carlos Marchena's own goal levelled things right on half-time before Sergio Aguero won it in the third minute of injury time.
Not only was it their first win in the group, but also City’s first victory in either the European Cup or Champions League.
Kompany said afterwards: “It's totally deserved, but unbelievable really. We're happy, maybe we're going to carry on doing these sort of things this season.
“Europe seems to be so different for us this season.
“I've said it before, I think we are in the toughest group...Villarreal have been in it (the competition) much more than us - so it makes it a beautiful night.”
Aguero’s strike finally ignited City’s stuttering Champions League campaign with is dramatic late winner.
For much of the second-half City were laboured but fit-again Aguero, who came off the bench on the hour, glossed over a poor 90 minutes by snatching his side's first win in Group A.
City recovered from the shock of conceding early to dominate first-half possession but, despite the constant probing of David Silva, lacked their usual Premier League swagger.
The manner of their equaliser was fortunate and the goal failed to provide the spark they needed until Aguero saved the day.
Manager Roberto Mancini said: “I said before the game it was important to win and I think we deserved to because we had a lot of chances to score, but Villarreal were very dangerous on the counter-attack because they play football very well.”
On Aguero's winner, the Italian continued: “It was an important goal because it is now open for first place (in the group).
“When you play in the Champions League for the first time you need to improve game after game. We needed to win and I think after this game we can do better.”
Mancini revealed the 40th-minute substitution of Adam Johnson was for tactical reasons.
The Italian explained: “I think Adam can do better than tonight, but I needed another player in the middle and needed to move Yaya (Toure) in front.”
Asked if the win was important ahead of Sunday's Manchester derby in the Premier League, Mancini added: “The derby is a different game.”
He was also reluctant to discuss the outcome of today's investigation into Carlos Tevez's alleged refusal to play as a substitute at Bayern Munich last month.
Mancini added: “I understand everything but I would prefer to not speak about this situation.”
The game had barely begun when when Silva played Nigel de Jong into trouble outside his own area and Jonathan de Guzman nipped in to steal possession.
Former Manchester United striker Giuseppe Rossi, a constant menace, lashed a shot at goal that Joe Hart could only parry and Cani reacted quickly to score.
Johnson shook his head as he left the field to be replaced by Barry and although the change itself made little difference to City's overall play, Barry was involved in the build-up as the hosts pulled level three minutes later.
Kolarov made another incisive run and his low cross took a slight deflection before a sliding Marchena diverted it into his own goal attempting to clear.
Just as time looked to be up at the end of the game, Pablo Zabaleta crossed and Aguero tapped in to spark jubilant scenes.
COMMENT RULES: Comments that are judged to be defamatory, abusive or in bad taste are not acceptable and contributors who consistently fall below certain criteria will be permanently blacklisted. The moderator will not enter into debate with individual contributors and the moderator’s decision is final. It is Belfast Telegraph policy to close comments on court cases, tribunals and active legal investigations. We may also close comments on articles which are being targeted for abuse. Problems with commenting? firstname.lastname@example.org