In chatbots we trust, but should we?

Almost gone are days of long queues and lengthy telephone calls to resolve queries at your bank, municipality and your telephone company.

Now we have artificial intelligence (AI)chatbots that have seamlessly integrated into our daily lives, aiding us with banking tasks, resolving inquiries, and entertaining us with trivia games.

But the burning question remains: should we truly trust them and the technology?

Rise of the bots

The rise of advanced language models like ChatGPT has ushered in a new era of human-like interactions, where chatbots can engage in natural conversations, solve complex problems, and even exhibit creative thinking.

This remarkable progress has opened up a world of possibilities, but it also raises concerns about the reliability and accountability of these systems, warns Anna Collard, senior vice president of content strategy and evangelist at KnowBe4 Africa.

“While most users have reported positive experiences with chatbots, instances of factual inaccuracies, hallucinations, and potential privacy risks have surfaced,” says Collard. A recent study by Consumers International found that although 64% of participants would use chatbots again, a significant portion encountered reliability issues.

ALSO READ: Average cost of a data breach in SA is R53.1m – Report

Dangers of using chatbots

Collard said the cautionary tale of sports journalist Karien Jonckheere serves as a poignant example.

Seeking inspiration for a cricket promo, Karien turned to ChatGPT, only to be met with a mix of accurate and fabricated match details that left her astounded.

 “It listed six matches, of which three were correct,” she remembered. “They completely fabricated the other three matches. It had given dates, venues, names of players, who scored the winning runs—all very specific, but completely made up.”

Collard warns that this is one of the obvious dangers of using chatbots.

“While they excel at answering straightforward questions, they can sometimes generate responses that are factually incorrect or nonsensical.”

Collard said privacy and security are also crucial factors to consider. As chatbots become privy to our personal information and preferences, concerns arise regarding the protection and potential misuse of this data.

“The stakes are high, as chatbots are no longer mere novelties but integral components of our digital ecosystem. From banking transactions to critical decision-making processes, their influence continues to grow.”

ALSO READ: WATCH #DellTechForum: ‘AI has potential to transform industries’ – Dell

Benefits of using chatbots

Collard said the advantages of chatbots for individuals and businesses are undeniable.

“Chatbots are available around the clock, providing immediate responses to queries, which is especially useful for different time zones and those needing after-hours help.”

“Chatbots excel in managing repetitive tasks tirelessly, such as addressing common questions or assisting users through standard procedures. They can juggle many interactions concurrently, a feat unattainable for a team of human agents,” said Collard.

Risks: ‘They don’t understand what they’re saying’

Collard said challenges arise when chatbots encounter complex inquiries beyond their scope.

“It’s important to remember that they don’t understand what they’re saying. They collate information from all over the web and stitch it together to formulate a response.”’

As Jonckheere soon realised, chatbots are—for now at least—ill-equipped at intricate problem-solving and can invent details when they are unsure of facts.

Collard said chatbots also lack human intuition, leading to difficulties in grasping subtleties, sarcasm, and context.

“That is why chatbots could be programmed to transfer complex queries to human operators where relevant. A robust chatbot system should offer the option to escalate the conversation to a human customer service representative for complex or sensitive queries that automated responses may not effectively address.”

ALSO READ: AI is where ‘curiosity and discovery live’, says Google

Privacy and security

Collard stressed that it’s important to exercise caution when sharing sensitive details with a chatbot, adding that companies using these novel technologies should also have strict privacy compliance regulations in place.

“Chatbots typically collect user data to personalise interactions and improve services. And you may not want all your personal data being reused by the algorithm for other queries. Remember whatever we upload to public models such as ChatGPT, will be fed into their model unless you specifically tick a setting that says otherwise.

“For companies making use of chatbots, it’s crucial that it is managed like any other system, meaning restrict its access to information that it absolutely needs access to, and ensuring that personal information is stored securely and managed according to strict privacy regulations, such as POPIA,” asserts Collard.

Personal info

Collard said that chatbots that handle sensitive transactions, such as banking queries, should authenticate users before any personal information is accessed or shared.

Collard said from a security perspective, it is important to test chatbots for prompt injection attacks before launching them to consumers.

“Similar to other software, regular updates are essential for chatbots to address vulnerabilities that malicious actors could exploit. A purpose-built chatbot should also incorporate security measures to thwart automated threats such as spam bots.”

So, can we trust chatbots?

Collard said while she likes using chatbots, she will always double-check the original sources when she uses chatbots for research or for accurate data,

“It is great that chatbots are available 24/7 and can help with a task or query at lightning speed. However, collaboration between humans and machines is key.

“The crucial aspect is integrating chatbots in a manner that complements the strengths of human agents, enabling a seamless transition to live assistance when needed. Businesses can leverage the scalability and data-processing capabilities of chatbots, while users enjoy the convenience of instant and accessible service,” Collard said.

ALSO READ: AI and the future of work – the opportunity for Africa