SOPA Images
- Amazon Web Services unveiled Q, its business-focused generative AI chatbot, last week.
- Amazon employees said the chatbot is leaking confidential data, according to Platformer.
- Q was designed to be a more reliable alternative to consumer-focused AI chatbots.
Amazon's Q, the AI chatbot for workers its cloud division unveiled on Tuesday, appears to have a few issues.
Q is blurting out confidential information — including the location of AWS data centers, internal discount programs, and even unreleased features — through its internal message channels, like Slack, and ticketing systems, according to leaked internal communications shared with Platformer, a tech newsletter.
The bot is also "experiencing severe hallucinations," a phenomenon in which AI confidently spits out inaccuracies like they're facts.
In Q's case, it's also been doling out bad legal advice that could "potentially induce cardiac incidents in Legal," as one employee put it in a company Slack channel, according to Platformer. The bot has also been returning harmful or inappropriate responses that could put customer accounts at risk.
Q's hallucinations are so "broad and egregious," as a manager at AWS described them, that the manager even cautioned employees from discussing them in public Slack channels, according to Platformer.
Amazon told Business Insider it had not identified any security issues related to Q.
"We appreciate all of the feedback we've already received and will continue to tune Q as it transitions from being a product in preview to being generally available," the company said in the statement.
It's not uncommon for generative AI chatbots to falter.
It wasn't long after Microsoft released its consumer-focused generative AI assistant, Sydney, that it went viral with its own hallucinations. But Q's transgressions are all the more ironic given that the bot was designed to be a safer and more secure option that businesses could rely on.
Q was built to help workers generate emails, summarize reports, troubleshoot, research, and code. It was also designed to provide them with helpful answers, but only related to content "each user is permitted to access," Amazon said in a blog post about Q.