Frank Lampard’s men drew 3-3 for the second time this season against Southampton on Saturday after twice giving up a lead.
“When you are weak defensively sometimes we feel like we have to score three or four goals every game, which sometimes you cannot do, it’s a reality,” Azpilicueta told reporters ahead of Tuesday’s Champions League opener against Sevilla.
Chelsea’s £220 million (243 million-euro) splurge in the transfer market has so far yet to see results as they sit seventh in the Premier League having won just two of their opening five games.
Big money signings Timo Werner and Kai Havertz did score their first league goals against Southampton, but Azpilicueta highlighted the need for Chelsea’s array of attacking talent to also defend from the front.
“I see football as a collective,” he added. “When we don’t score, as a defender I think how can we put our attacking players in better situation.
“As a defender you feel bad when you concede so many golas. Collectively we have to improve so we all defend together, it doesn’t matter our position.
“Through a season you are not going to be winning 3-0 or 4-0 every week, you have to work to get results and at the moment we have to improve that.”
A big part of Chelsea’s defensive problems has been the form of goalkeeper Kepa Arrizabalaga.
The Spaniard was again at fault for Southampton’s second goal at Stamford Bridge at the weekend, but with new signing Edouard Mendy sidelined by injury, Lampard must choose between Kepa and 39-year-old Willy Caballero for the visit of Europa League winners Sevilla.