Hidden dangers of increasingly advanced AI assistants

Artificial Intelligence (AI) assistants are becoming more prominent and continue to increase as people require efficient and speedy service.

From querying about a product online to seeking help with your mobile contract, more and more companies are using digital assistants (DA) to cope with the influx of customers wanting attention.

Risks

However, as AI assistants in South Africa proliferate, so do the risks.

Trend Micro Solutions Architect for the Middle East and Africa Zaheer Ebrahim said the evolution of AI assistants over the past few years is nothing short of remarkable.

“Initially designed to perform simple tasks such as setting reminders and playing music, these AI-driven software systems have transformed into sophisticated conversational agents, powered by advanced large language models like ChatGPT and Google Gemini.”

ALSO READ: South Africa on track to regulate artificial intelligence

Extraordinary ability

DAs exhibit an extraordinary ability to understand and respond to natural language queries with impressive accuracy and context sensitivity.

However, Ebrahim warns that as advanced helpers become more sophisticated, the security risks become more complex.

“It is imperative for both developers and users to be proactive and vigilant in addressing emerging threats, ensuring that the benefits of AI are not overshadowed by potential vulnerabilities.

New threats

As DA’s become increasingly integrated into daily life and interconnected with various devices and services, they become attractive targets for malicious actors.

Ebrahim said continuous interaction with DA’s generates enormous amounts of personal data, including names, addresses, e-mail addresses, phone numbers and even sensitive health information.

“This data is essential for providing personalised and proactive assistance. However, it also raises significant privacy concerns.

“Unauthorised access or misuse of this data can lead to severe consequences, making robust data protection measures crucial.

“Encryption of sensitive data, both at rest and in transit, is a fundamental security measure that needs to be prioritised,” Ebrahim warned.  

ALSO READ: Opera adds Aria AI assistant to Opera Mini on Android

Custom skills

Ebrahim added that the advanced helpers also bring malicious “custom skills” that seem legitimate but actually contain harmful functions.

“In this context, ‘skills’ refer to specific features or abilities that can be added to AI assistants to enhance their functionality, similar to apps on smartphones. These rogue skills can manipulate the assistant’s responses to provide false information.”

Social engineering

He said social engineering can also be executed through DAs, where attackers manipulate the output generated by these assistants to deceive users.

“For instance, a DA might be instructed to relay a seemingly legitimate message from a trusted source, leading users to take harmful actions.

“This threat is particularly concerning as DAs take on more sophisticated tasks, such as managing user finances.”

Recommendations

Ebrahim said that to address these digital assistant-based threats, it is essential for both developers and users to be able to recognise the types of personally identifiable information collected by DAS and collaborate to promote a culture of security awareness, responsible behaviour and best practices.

“A holistic approach incorporating multiple layers of defence is key. Robust authentication mechanisms can ensure user identity verification and safeguard against unauthorised access.

“Encrypting sensitive data at various levels – at rest, in transit and within the DA’s internal memory -protects user information from malicious actors.”

Ebrahim advised that employing advanced threat detection techniques, utilising machine learning algorithms and behavioural analysis can proactively defend against evolving cyber threats.

ALSO READ: The rise of AI tools and their impact on smartphone use