City suffered a surprise semi-final defeat against Arsenal on Saturday as Pierre-Emerick Aubameyang’s double sealed a 2-0 win at Wembley.
With two Premier League games remaining and City assured of second place behind champions Liverpool, Pep Guardiola’s team will turn their attention to Europe.
A 2-1 first leg win against Real Madrid gives City a chance of making the quarter-finals, with the second leg of their last-16 tie taking place at the Etihad Stadium on August 7.
But City were sluggish against Arsenal — days after a lacklustre win against Bournemouth — and German midfielder Gundogan knows improvements have to be made if they are to win the Champions League for the first time.
“We have two more league games to play and then we have a big game against Madrid,” Gundogan told City’s website.
“We should definitely improve because the way we played (Saturday) won’t be enough to go into the next round of the Champions League.
“We have to improve quite quickly because we don’t have much time. We need to try to do our best.”
Gundogan also admitted the way Arsenal outfought City in the first half suggested Mikel Arteta’s side played with more desire than the holders.
“I think the energy we had starting the second half was exactly what we needed at the beginning of the game,” he added.
“Sometimes one very bad half is enough to lose the game and it looked like Arsenal wanted it more to go to the final which was the most frustrating part.”