Lautaro Martinez volleyed Inter into the lead from Nicolo Barella’s cross.
Ruslan Malinovskyi levelled with a fine long-range strike, had a hand in Rafael Toloi putting Atalanta ahead and hit the post with a free-kick.
Edin Dzeko scored a rebound from close range to making it 2-2 for Inter.
His fifth goal in six league games for the club.
The drama was not over yet though.
Inter were awarded a late penalty for handball which Federico Dimarco hit the bar with.
Atalanta scored what they thought was a late winner as Samir Handanovic spilled Roberto Piccoli’s shot.
But, a long video assistant referee review found the ball had gone out of play in the build-up.
Inter had won every home game in the league since a draw with Parma on 31 October 2020.
Their previous defeat was the game before that, a 2-1 loss to AC Milan on 17 October.
Simone Inzaghi’s Inter are third, two points behind leaders AC Milan, who won 2-1 at Spezia.