Joachim Low’s side, who ended the match with 10 men following Jerome Boateng’s dismissal, were in grave danger of crashing out of the World Cup when they trailed at half time but second-half goals by Marco Reus and Kroos improved their chances of reaching the knockout stages.
Germany started the match very well and Julian Draxler had a fourth-minute shot blocked.
However, Sweden had strong appeals for a penalty waved away by the referee in the 16th minute when Boateng appeared to bring Marcus Berg down just when the Al Ain striker was bearing down on goal.
Sweden did take the lead in the 32nd minute though with a well-taken goal by Ola Toivonen who took the ball down on his chest before chipping it over Manuel Neuer.
Germany were rattled but they almost equalised when Ilkay Gundogan’s deflected shot was kept out by Robin Olsen.
In first-half stoppage time, after a rapid Swedish counter-attack, Viktor Claesson opted to cut inside rather than shoot first time and a presentable opening for a second goal was lost.
Neuer then made a scrambling save to deny Berg after the striker had got his head to Seb Larsson’s excellent free-kick.
Germany showed more desire after the break and they equalised three minutes into the second period with a scrappy goal when Timo Werner’s cross was turned home by Reus.
As Germany pressed for a winner Thomas Muller headed wide and Kroos had a shot blocked.
Sweden could not get out of their own half as Germany piled forward at every opportunity but Mario Gomez contrived to lash the ball over the bar from five yards out.
As Sweden tired Andreas Granqvist almost turned the ball into his own net but Olsen kept it out before an unmarked Werner lashed a shot over the bar.
Germany were reduced to 10 men with eight minutes remaining when Boateng was handed a second yellow card but they should have taken the lead in the 88th minute when Olsen made an outstanding stop to keep Gomez’s close-range header out.
In the second minute of stoppage time Julian Brandt’s shot hit the post but there was still time for Kroos to curl home the winner from a well-worked free-kick.
Download our app and read this and other great stories on the move. Available for Android and iOS.