That dream run continued in Recife on Sunday as they rode their luck against Greece to win a tense clash at the Arena Pernambuco, triumphing 5-3 on penalties to set-up a quarter-final clash against the Netherlands.
The Central Americans came into this match in the unusual role as favourites and looked to be wilting under the pressure, with only an exquisite stop by goalkeeper Keylor Navas stopping Dimitris Salpingidis from giving the Pirate Ship, a first-half lead.
Los Ticos were struggling to replicate the form they had shown in the group stage but managed to open the scoring with their first shot on target, as Bryan Ruiz directed home in the 52nd minute.
Costa Rica should have had a penalty soon after and were reduced to 10 men after 66 minutes, with Oscar Duarte's sending-off eventually telling in stoppage-time as Sokratis Papastathopoulos struck to take the match into extra-time.
There was to be no separating the sides in those 30 minutes, though, and, after Greece coach Fernando Santos was sent to the stands, the match was decided on spot-kicks, with Navas saving Theofanis Gekas' penalty before Michael Umana scored the decider.
It was an exciting end to a scrappy match in which the only moment of note in the opening 20 minutes was a left-footed drive from the recalled Cristian Bolanos that flew just over.
Greece were enjoying the lion's share of chances but were looking susceptible defensively at times, with skipper Giorgos Karagounis avoiding a penalty more through luck than judgement when catching Joel Campbell on the edge of the box.
It looked like a penalty on first glance but referee Ben Williams was right adjudge the foul to have taken place just outside the area, with the resulting free-kick coming to nothing.
Veteran Karagounis tried his luck from distance when Greece returned to the attack - a chance Navas dealt with easily, before having to be at his best to keep the scores level in the 37th minute.
Meeting a fine left-wing Jose Holebas cross, Salpingidis looked certain to score from close range only for the Costa Rica goalkeeper to stop the goalbound strike with his right leg.
Duarte joined Andreas Samaris in the referee's notebook just before the half-time whistle went, with the locals letting their displeasure at the lack of quality known.
Los Ticos failed to conjure a single shot on target in the opening period but changed that just seven minutes after the restart, breaking the deadlock in the process.
Bolanos pulled the ball back from the left into the path of Ruiz, who was afforded too much time to coolly roll home a first-time left-footed strike, which trickled past Orestis Karnezis.
It was a fine placed effort and los Ticos should have had the chance to double their advantage moments later when Vasilis Torosidis turned away a cross with his hand.
Substitute Oscar Granado was booked for his remonstrations as tensions rose, with Costa Rica reduced to 10 men in the 66th minute when Duarte picked up a needless second booking for tripping Holebas.
Samaras flashed a header wide almost immediately before Navas flapped at his looped ball as Greece upped the ante.
Substitute Gekas turned wide, Kostas Manolas headed over and Karagounis wasted a free-kick as the game looked to have got away from them, only for Papastathopoulos to net a stoppage-time leveller.
With the ball fired into the box, Gekas saw a snapshot parried by Navas and the ball fell kindly for the centre-back to fire home.
It was the first time the Costa Rica goalkeeper had been beaten from open play in Brazil and he had to be at his best to stop Greece doing so again moments later, tipping over a powerful header from substitute Konstantinos Mitroglou.
The Fulham striker failed to get a telling touch to beat Navas again as extra-time began, before an unsighted Gekas directed wide and Konstantinos Katsouranis had a close-range effort blocked.
Randall Brenes came close for Costa Rica at the start of the second period of extra time in which Lazaros Christodoulopoulos stung the palms of Navas, who then denied Mitroglou at the death as the match went to penalties.
Greece coach Santos was sent to the stands before the spot-kicks began, with every one finding the net until Navas denied Gekas with Greece's fourth of the night.
It gave Umana the opportunity to secure victory, which he duly did by firing the ball into the top right-hand corner.
COMMENT RULES: Comments that are judged to be defamatory, abusive or in bad taste are not acceptable and contributors who consistently fall below certain criteria will be permanently blacklisted. The moderator will not enter into debate with individual contributors and the moderator’s decision is final. It is Belfast Telegraph policy to close comments on court cases, tribunals and active legal investigations. We may also close comments on articles which are being targeted for abuse. Problems with commenting? firstname.lastname@example.org