Forward Dlamini headed the hosts in front just on the half hour, before replacement Gamildien’s composed finish straight after half time.
Richard Henyekane pulled a goal back with 10 minutes to play, but it proved a mere consolation as the visitors dropped to second bottom on the table.
Stars, who began the game well, were the first to test either of the goalkeepers with AmaZulu custodian Philani Zikalala forced to punch away unsighted a whipped-in Henyekane free-kick from wide on the left.
The Free Staters continued to look a threat on the break and when Usuthu carelessly conceded possession in midfield, Mthokozisi Yende played in Henyekane, whose effort from the right side of the box was fisted away by Zikalala.
At the other end, efforts from Dlaminis Ayanda and Goodman were blocked by defenders in the opening quarter, while Nkosinathi Mthiyane saw a wild swish fly over.
However, the hosts went in front after 29 minutes when Sifiso Hlanti swung over an inviting ball from the left and Ayanda Dlamini arrived late to head in.
Gamildien, on the field for less than a minute after replacing the injured Goodman Dlamini, almost scored with his first touch at the back post but was denied by the flying Tshepo Motsoeneng.
Thamsanqa Teyise and Bongani Ndulula had good chances to double AmaZulu’s tally before the break, only to shoot wide.
Usuthu, though, did double the lead straight after the break when a cool Gamildien finish put them 2-0 up – the attacking midfielder collecting a 46th-minute pass from Ayanda Dlamini on the right side of the box and crashing into the top corner first time.
The goalscorer was denied a second straight after when his clever flick at the near post was somehow turned wide by Motsoeneng.
After a long-range Luyolo Nomandela strike was straight at Zikalala, it should have been 3-0 after 70 minutes when Bongani
Ndulula found himself one-on-one with the Stars goalkeeper, but instead of chipping over, he fired straight at the Motsoeneng.
Camaldine Abraw then handed Henyekane a great chance to pull a goal back, but the striker wastefully headed over.
But he made no mistake when he played in with a low Abraw ball in the 80th minute, slotting calmly beyond Zikalala to give the visitors hope.
However, they failed to produce any late rally and slipped to a defeat that compounded their relegation woes.