With the Swans looking for a third boss in less than 12 months, Giggs had been talked about in relation with the role along with Ronald Koeman, Tony Pulis and Louis van Gaal.
Giggs was asked if he was interested in managing a club he was linked with before Bob Bradley was appointed in October 2016.
“No. I’ve spoken to them before, last time before they appointed Bob Bradley,” Giggs told Sky Sports.
Giggs said the sales of Gylfi Sigurdsson and Fernando Llorente in the summer had weakened the Swansea squad.
“I don’t know, I’m not in the club, but obviously over the years (it has been) a really well-run club, (who) play the right way and are an established Premier League team,” said Giggs.
“But it’s tough now, you see the teams down there. It gets tougher each year. They lost good players, didn’t they? They lost Sigurdsson, they lost Llorente, and probably never replaced them. That’s always a problem.”
Giggs, 44, served as caretaker manager at Manchester United in 2014 following the sacking of David Moyes and was assistant manager under Van Gaal.
But despite being linked with a number of jobs, he has not managed to find a role since leaving United in 2016.
Clement saved Swansea from relegation after being appointed in January, but his departure was confirmed less than 48 hours after the 3-1 loss at Everton on Monday — Swansea’s ninth defeat in 11 games.
Swansea on Thursday announced long-serving midfielder Leon Britton, who has been part of the coaching staff, has been put in caretaker charge and will oversee Saturday’s match against Crystal Palace.
Speaking at a press conference, Britton said he had no interest in taking on the role full-time.
“It is not to say I wouldn’t take it in the future, but I just think at this moment in time it is not the right move for me,” said the 35-year-old.
“I’ve been asked to help the club out, which is something I will always do. But in terms of the immediate future, I am not looking to be the full-time manager.”
Download our app and read this and other great stories on the move. Available for Android and iOS.