British authorities are issuing warnings to organizations regarding the integration of AI-driven chatbots into their operations, citing research that reveals susceptibility to manipulation leading to potentially harmful actions.
In a pair of forthcoming blog posts, the National Cyber Security Centre (NCSC) in Britain highlights the challenges posed by algorithms capable of generating human-like interactions, often referred to as large language models (LLMs). These AI tools, initially utilized as chatbots, are envisioned to not only impact internet searches but also customer service and sales functions.
The NCSC emphasizes that these models could introduce risks, especially when integrated with other aspects of an organization’s operations. Researchers have identified vulnerabilities in chatbots, demonstrating their susceptibility to rogue commands or circumvention of security measures.
For instance, a hacker could potentially trick an AI-powered chatbot deployed by a bank into conducting an unauthorized transaction by skillfully structuring their query.
The NCSC advises caution, likening the use of LLMs to experimental software releases, suggesting that organizations should exercise prudence and not entrust them with critical tasks, such as customer transactions.
Around the world, authorities are grappling with the emergence of LLMs like OpenAI’s ChatGPT, which are finding applications in various services, including sales and customer support. As the security implications of AI continue to evolve, authorities in the United States and Canada are also reporting instances of hackers exploiting this technology.
