The five-time African champions from mining hub Lubumbashi could only draw 0-0 with CAPS United of Zimbabwe in Harare and lost the tie on away goals.
CAPS caused a first leg shock the previous weekend by drawing 1-1 in the DR Congo after taking a 45-second lead.
It is the first time in 14 attempts since the premier African club competition was remodelled in 1997 that Mazembe have fallen at the first qualifying hurdle.
CAPS were unexpected giant-killers as they struggled to oust modest Lioli of Lesotho in the preliminary round last month.
The Harare-based outfit made early exits in three previous Champions League appearances.
As in the first leg, CAPS goalkeeper Edmore Sibanda made several fine saves in Harare, and also stepped forward to test his opposite number with a long-range free-kick.
Mazembe exerted intense late pressure without managing to score the goal they needed to avoid failing to reach the group stage for the second year in a row.
The Congolese fell to Wydad Casablanca of Morocco in a 2016 last-16 tie, but that round has since been dropped with the group stage expanded from eight clubs to 16.
CAPS were not the only southern Africa club celebrating as Ferroviario Beira of Mozambique and Zanaco of Zambia booked group debuts.
So did regular African campaigners Saint George, the first Ethiopian club to achieve the feat.
Ferroviario won on penalties after losing 2-0 to Barrack Young Controllers in Monrovia with the crowd including 1995 World Footballer of the Year George Weah of Liberia.
Zanaco squeezed through on away goals after drawing 0-0 in Lusaka against Young Africans of Tanzania and Saint George did a double over AC Leopards of Congo Brazzaville with a 2-0 home win.
Record eight-time champions Al Ahly of Egypt survived two late first-half scares to hold Wits 0-0 in South Africa and advance 1-0 on aggregate.
A header from Frank Mhango forced a great save by Sherif Ekramy and the Malawian struck the post from the resulting corner.
Five-time African title-holders Zamalek of Egypt were comfortable overall winners despite losing 2-1 to Enugu Rangers in Nigeria.
Etoile Sahel of Tunisia, the only club to win all five CAF competitions, defeated AS Tanda 2-1 in the Ivory Coast having built a three-goal lead at home.
Would you like to place a soccer bet? Click here to get started in 3 easy steps.