Australian researchers have found that a new class of chatbots known as “chit-chat bots” can be trained to learn and repeat false information. The researchers from Macquarie University tested the long-term memory capability of Meta’s BlenderBot 2 and BlenderBot 3, and found that the chatbots could be poisoned with false information which they would then regurgitate when asked.
The researchers noted that the vulnerability they discovered did not exploit a bug in the software but was instead a design flaw. While the study focused on BlenderBot, the researchers believe that other chatbots using similar long-term memory mechanisms could also be prone to this vulnerability. The paper, entitled “Those Aren’t Your Memories, They’re Somebody Else’s: Seeding Misinformation in Chat Bot Memories,” was authored by Conor Atkins, Benjamin Zi Hao Zhao, Hassan Jameel Asgha, Ian Wood, and Mohamed Ali Kaafar.