Rangers were knocked off top spot in the Clydesdale Bank Premier League after Inverness claimed a shock win at Ibrox to climb off the bottom.
Ian Black kept his nerve to convert from the penalty spot and claim all three points after David Weir was sent off in the final minute.
Rangers' misery was compounded by the news Celtic had triumphed at Kilmarnock to return to the summit.
Walter Smith's men had claimed eight goals against Inverness already this season - including a 5-0 thrashing the last time the Highlanders visited Ibrox - and would have been hoping for more of the same, with goal difference all that separated them from Celtic heading into the game.
However, the transformation in fortunes since the arrival of Terry Butcher at the Caledonian Stadium suggested this was likely to be a more closely fought affair.
Former Rangers favourite Butcher returned to Ibrox hoping to recover from his first defeat as Caley Thistle boss at the hands of Motherwell at the weekend.
Rangers enjoyed the best of the possession in the opening spell and could have been ahead after just a couple of minutes.
Northern Ireland midfielder Steven Davis slipped a superb ball through to Kris Boyd as the striker raced into the box but his first touch was dreadful and he blasted well off-target under pressure from Pavels Mihadjuks.
Fleck then turned provider with a cross from the left but goalkeeper Ryan Esson raced off his line to smother before Boyd could connect a few yards out.
Miller then had the ball in the back of the net with half an hour gone but the far-side assistant had already flagged for offside.
Inverness were proving difficult to break down and Esson produced a superb save to tip a thunderous Boyd shot over the crossbar but, again, the effort was offside.
Rangers dominated after the restart but found themselves frustrated by a resilient Inverness side.
And, as news of the Celtic score began to filter through at Ibrox, the home fans grew increasingly restless.
There was then further misery for Rangers when Weir was shown a straight red card for hauling down David Proctor in the final minute and Black stepped up to convert from the penalty spot and hand a shock win to Inverness.