With just one point in their last four league outings, Wits were desperate to stay in the hunt for the league title, but found themselves a goal down after just three minutes when Tebogo Potsane fired home for the hosts.
Gavin Hunt’s charges tried their utmost to find the equaliser and had the perfect opportunity in the 36th minute, after Mbhazima Rikhotso brought down Wits’ Malawian centre forward Gabadhino Mhango in the penalty area.
However, Celtic goalkeeper Patrick Tignyemb did well to save Keegan Ritchie’s spot kick.
Hunt’s nerves were eased slightly in the 58th minute when Mhango was able to work the ball home past Tignyemb to level matters.
The visitors could well have snatched the lead a minute later, when winger Deon Hotto’s effort found the Celtic crossbar, with his first touch of the game having come on as a second half substitute.
Wits kept pressing for the advantage and went 2-1 ahead in the 69th minute with Mhango being the provider for Gift Motupa to head his team in the lead.
Celtic striker Tshegofatso Mabasa seemed to have earned his team a point when he headed the men in green and white back on level terms in the 87th minute.
It was the 2016/2017 league champions who had the final say when their Zimbabwean forward Dzvukamanja scored a minute later, before netting his second goal of the night deep into referee’s optional time.
Wits, now on 41 points from 24 matches, will change their focus to Nedbank Cup action, ahead of their quarterfinal encounter at home to Chippa United on March 30.
Celtic, who remain on 31 points from 24 league outings, will be at home to Golden Arrows in their last-eight cup match on the same date.