Philander played his last Test in January and was set to feature for Somerset after signing a Kolpak deal with the club, located in southwest England.
Under such deals cricketers are not classed as overseas players.
However, cricket in England is currently postponed until at least July 1 because of the health crisis, with no guarantee that matches will be played at all this season.
Philander, who took 224 wickets in 64 Tests for South Africa, has therefore consented to the termination of his deal.
Somerset director of cricket Andy Hurry left the door open for the 34-year-old, who played for the county in 2012, to return to Taunton in the future.
“These are clearly very uncertain and challenging times for all counties with no cricket until at least 1 July and the likelihood of a condensed and truncated season,” Hurry said on Monday.
“It was such an exciting prospect to visualise Vernon returning to the Cooper Associates County Ground, but given the current uncertainty the cancellation of his contract was a prudent and sensible decision.”
Download our app and read this and other great stories on the move. Available for Android and iOS.