A second defeat in the Scottish champions’ first six league games, after also failing to qualify for the Champions League, leaves Celtic six points behind early leaders Hearts.
Brendan Rodgers’s men could also fall to sixth in the table should Rangers beat St. Johnstone later at Ibrox to move ahead of their Glasgow rivals.
Rodgers made six changes from the side that laboured to a 1-0 Europa League win over Rosenborg in midweek.
Leigh Griffiths was recalled to the starting line-up after coming off the bench to grab the winner against the Norwegian champions and his diving header put the visitors in front 11 minutes before half-time.
Celtic posted a record turnover of £100 million ($130 million) this week, but their failure to strengthen in the transfer window despite selling star striker Moussa Dembele to Lyon is already being exposed in their bid for an eighth straight Scottish title.
They have now scored just six times in their opening six games and rarely looked like adding to their lead before Chris Burke fired in off the post from 20 yards to bring Kilmarnock level.
Celtic continued to dominated possession without seriously threatening, and were caught again by a sucker punch in the 93rd minute when Findlay stooped to head Burke’s corner into the far corner, sparking wild scenes of celebration from Killie boss Steve Clarke.
In four meetings with Rodgers since Clarke took charge just under a year ago, Kilmarnock have now beaten and drawn with Celtic twice.
For more news your way, download The Citizen’s app for iOS and Android.
Download our app and read this and other great stories on the move. Available for Android and iOS.