With five minutes left, Kai Havertz made it 3-2 from close range for Germany.
Substitute Niclas Fullkrog then scored 4-2 after VAR review.
Then ten minutes of added time awaited, a long agony for Germany who depended on a Spanish goal against Japan (it was 2-1 to Japan).
At the beginning of the second half, Costa Rica shocked by scoring 1-1.
Kendall Waston pounced and headed a ball that Manuel Neuer saved but dropped the rebound on.
Yeltsin Tejeda beat it.
The shock was complete after Juan Pablo Vargas gave Costa Rica a 2-1 lead after heading the ball in from close range.
It looked like it might have been offside a couple of times before but the goal was cleared after VAR review.
Just three minutes later, Kai Havertz equalized to make it 2-2 and there was a fight over the ball that was left in Costa Rica's goal.
Substitute Niclas Fullkrog came close to making it 3-2 for Germany but his close-range shot was saved with a header by Paris SG goalkeeper Keylor Navas.
After the opening loss against Japan, 1-2, the German national team was sawn off, which equalized against Spain, 1-1, with seven minutes left in regular time.
Thanks to Costa Rica's victory against Japan, they got a lifeline back to the playoffs.
With Spain beating Japan, Germany could concentrate on doing their part - victory against Costa Rica, who opened the World Cup with a 7-0 loss to Spain, was enough.
Already in the tenth minute, Germany took the lead when Serge Gnabry headed in a ball struck from the left wing by David Raum.
Just before the break, however, Costa Rica stood up.
After a huge mistake by Antonio Rüdiger, Keysher Fuller was freed but Manuel Neuer was able to raise a fist and save the ball over the crossbar for a corner.
A goal in that situation would have taken Costa Rica through to the round of 16 – at Germany's expense.