Bolt received the baton with Jamaica in third, but halfway down his leg the towering sprinter pulled up clutching his left thigh, eventually doing a forward roll to the ground, to gasps from the 60,000 sell out London Stadium.
The British quartet of Chijindu Ujah, Adam Gemili, Daniel Talbot and Nethaneel Mitchell-Blake claimed gold in 37.47 seconds, with the Justin Gatlin-led US four taking silver at 0.05sec and Japan a surprise bronze (38.04).
Bolt missed out on his bid to retain his 100m title earlier in the week, losing out to Gatlin and silver medallist Christian Coleman, who ran relay anchor for the Americans on Saturday.
But hopes were high for Bolt’s final competitive race, with Jamaica also boasting newly-crowned 110m hurdles champion Omar McLeod, Julian Forte and Yohan Blake in their line-up.
Jamaica were afforded a rousing welcome from the crowd, Bolt applauding the stands, with pictures of him constantly shown on the stadium’s big screens.
Gatlin and the US team also including another convicted doping cheat, Mike Rodgers, and Jaylen Bacon were booed when introduced although the jeering was less than for the individual 100m event.
A close first three legs saw Britain, the United States and Jamaica level-pegged for the final leg.
But there was to be drama as a visibly swearing Bolt pulled up, allowing the Japanese quartet to nip in for third.
The result means Bolt, 100 and 200m world record holder, finishes his career with 14 world medals to go with eight Olympic golds.
Download our app and read this and other great stories on the move. Available for Android and iOS.