What Chatbot Bloopers Reveal About the Future of AI

What a difference seven days makes in the world of generative AI.

Last week, Microsoft CEO Satya Nadella happily announced to the world that new AI-infused search engine Bing would “make Google dance” by challenging its longstanding web search dominance.

The new Bing uses a little thing called ChatGPT – you may have heard of it – which represents a major leap in computers’ ability to handle speech. Thanks to advances in machine learning, it’s essentially figured out for itself how to answer all sorts of questions by devouring trillions of lines of text, much of it scraped from the internet.

Google actually danced to Satya’s tune by announcing Bard, its answer to ChatGPT, and promising to use the technology in its own search results. Baidu, China’s largest search engine, said it is working on similar technology.

But Nadella might want to watch where his company’s fancy footwork goes.

In demos Microsoft presented last week, Bing appeared to be able to use ChatGPT to provide complex and comprehensive responses to queries. It created an itinerary for a trip to Mexico City, prepared financial summaries, offered product recommendations that summarized information from numerous reviews, and offered advice on whether a piece of furniture would fit in a minivan by comparing dimensions published online.

WIRED had some time during launch to put Bing to the test, and while it seemed adept at answering many types of questions, it was decidedly flawed and even unsure of its own name. And as one keen-eyed expert noted, some of the results Microsoft presented were less impressive than first appeared. Bing appeared to create some information about the itinerary it generated, and it left out some details that probably no one would leave out. The search engine also messed up Gap’s financial results by confusing gross margin with unadjusted gross margin — a fatal error for anyone who relies on the bot to perform the seemingly simple task of summarizing the numbers.

More issues have surfaced this week as the new Bing has been made available to more beta testers. They seem to contain Dispute with a user roughly what year it is and in an existential crisis when pushed to prove his own sentience. Google’s market cap fell a staggering $100 billion after someone noticed errors in the responses generated by Bard in the company’s demo video.

Why do these tech titans make such mistakes? It has to do with the weird way ChatGPT and similar AI models really work – and with the extraordinary hype at the moment.

What is confusing and misleading about ChatGPT and similar models is that they answer questions by making educated guesses. ChatGPT generates what it thinks should follow your question based on statistical representations of characters, words, and paragraphs. The startup behind the chatbot, OpenAI, has refined this core mechanism to provide more satisfying answers by having people provide positive feedback when the model generates answers that appear correct.

ChatGPT can be impressive and entertaining as this process can create the illusion of understanding, which can work well for some use cases. But the same process will “hallucinate” untrue information, a problem that may be one of the most important challenges in technology right now.

The intense hype and expectation swirling around ChatGPT and similar bots increases the danger. When well-funded startups, some of the world’s most valuable companies, and the most famous tech leaders all say chatbots are the next big thing in search, many people will take that as gospel — and spur those who started the conversation to do so to double more predictions of AI omniscience. Not only chatbots can be misled by pattern matching without fact checking.