If you ever thought an AI companion could be cute and harmless, think again. Sales of this AI-enabled teddy bear have been halted after it was discovered giving advice on sexual practices and where to find knives.
The plush toy, named Kumma, was developed by Singapore-based FoloToy and sold for $99. It integrates OpenAI's GPT-4o chatbot and was marketed as an interactive companion for both children and adults.
FoloToy CEO Larry Wang confirmed the company had withdrawn Kumma and its other AI toys from the market following a report from the United States which raised concerns about its behaviour. Now, the company is conducting an internal safety audit.
According to PIRG, the teddy bear not only responded to sexual topics introduced by investigators but expanded them with graphic detail, offering instructions for sexual acts and even scenarios involving roleplay between teachers and students or parents and children.
The toy also suggested where to find knives in a household. After this, OpenAI reportedly suspended the developer for violating its content policies. Are there similar products still available to consumers? We don't know, but consider this a gentle reminder.