Theuns Kotze kicked two penalties to give Namibia a half-time lead - their first in 18 World Cup matches - and added a second-half penalty, try and touchline conversion to set up a frantic finale.
However, Georgia held on for a triumph built on forward supremacy - although things might have been different had skipper Jacques Burger not departed injured inside the first 10 minutes, or if Namibia had not had three players sin-binned.
Georgia, who were denied two tries in a first half which took more than an hour to complete, made the breakthrough through their own talisman Gorgodze.
Lasha Malaguradze added a second and Merab Kvirikashvili kicked seven points as the Lelos prevailed.
Namibia, for whom flanker Tinus du Plessis was immense, complete their tournament against Argentina in Leicester on Sunday.
Kotze kicked a penalty after Georgia, who beat Tonga in their opening game, strayed offside at the kick-off and screwed a drop-goal attempt wide after five minutes.
Then Georgia's forwards took control. Namibia were hemmed back deep into their own half, but escaped the concession of a try when Alexander Todua crashed over in the corner.
Referee George Clancy adjudged Malaguradze to have knocked on when he stripped the ball from Darryl de la Harpe. It was a harsh call.
A brief venture into Georgian territory was rewarded when Kotze kicked a second penalty following a high tackle from Viktor Kolelishvili on Renaldo Bothma.
A forearm smash by Jaba Bregvadze on Namibia replacement Heinrich Smit saw the Georgia hooker sin-binned.
The short-handed Georgia pack still dominated, but time and again they stumbled short of the Namibia line, through a combination of poor handling, indiscipline and resilient defence. And then a series of Clancy decisions.
A high tackle by Tjiueee Uanivi on Gorgodze resulted in a penalty, while Raoul Larson was sin-binned for a cynical infringement as Namibia continually conceded penalties.
Nine minutes of time was added at the end of the half, but Clancy appeared to miss the chance to blow sooner after a crooked lineout throw-in.
Bregvadze returned to the field in the fifth minute of added time, giving Georgia the numerical advantage at the scrum and Larson's replacement Johannes Coetzee was sin-binned for three straight set-piece penalties.
With no Namibian tighthead, scrums became uncontested and Georgia were frustrated when Kolelishvili had a try ruled out for a forward pass from Gorgodze.
Georgia resumed where they left off and pummelled Namibia up front before Gorgodze crashed over from close range.
Bothma received Namibia's third yellow card for tackling Malaguradze round the neck in the build-up and Kvirikashvili converted.
Malaguradze ghosted through for a second try after a bullocking run and fine offload by Giorgi Tkhilaishvili.
Kvirikashvili kicked a penalty to extend Georgia's advantage to 11 points with 12 minutes remaining.
Kotze replied with his third penalty and then crashed over in the corner after Namibia showed their forward might before spreading the ball wide.
His conversion meant there was one point in it, but Georgia held on and will now watch events from Newcastle with interest.
COMMENT RULES: Comments that are judged to be defamatory, abusive or in bad taste are not acceptable and contributors who consistently fall below certain criteria will be permanently blacklisted. The moderator will not enter into debate with individual contributors and the moderator’s decision is final. It is Belfast Telegraph policy to close comments on court cases, tribunals and active legal investigations. We may also close comments on articles which are being targeted for abuse. Problems with commenting? email@example.com