The Welshman, 55, took over at St Mary’s in March and guided them to Premier League survival and an FA Cup semi-final but was unable to inspire a better campaign this season.
“We would like to express our thanks to Mark and his staff for all of their efforts during their time at St Mary’s,” the club said in a statement.
“The search for a new manager to take the club forward is already under way.
“First team assistant coach Kelvin Davis will lead the team for Wednesday’s Premier League game against Tottenham Hotspur at Wembley.”
Assistant first team manager Mark Bowen and assistant first team coach Eddie Niedzwiecki have also left the club.
Southampton, who were 2-0 up against Jose Mourinho’s United at home on Saturday at one stage, are 18th in the Premier League, just one point ahead of bottom club Fulham.
They have won just one of their 14 Premier League matches this season under Hughes, suffering seven defeats.
The former Manchester United, Barcelona and Chelsea player signed for Southampton as a player towards the end of an illustrious career.
During his coaching career he has managed Wales as well as Blackburn, Manchester City and Stoke.
For more news your way, download The Citizen’s app for iOS and Android.
Download our app and read this and other great stories on the move. Available for Android and iOS.