More than half a decade after Microsoft’s truly monumental Taye debacle, the incident is still a stark reminder of how quickly an AI can be corrupted after exposure to the web’s potent toxicity, and cautions against bots without sufficiently robust behavioral bindings to build. On Wednesday, Meta’s AI research department will see the public demo version of Blenderbot 3 with 175 billion parameters to see if their latest iteration of Blenderbot AI can withstand the horrors of the interwebs.
A major obstacle currently facing chatbot technology (and the natural language processing algorithms that power it) is procurement. Traditionally, chatbots are trained in highly curated environments – because otherwise you’ll inevitably get a taye – but this results in the topics they can discuss being limited to the specific ones available in the lab. Conversely, you can have the chatbot pull information from the internet to gain access to a wide range of topics, but could and probably will become full Nazi at some point.
“Researchers cannot possibly predict or simulate every conversation scenario alone in research settings,” Meta CEO Mark Zuckerberg wrote in a blog post Wednesday. “The AI space is still a long way from truly intelligent AI systems that can understand us, interact and chat with us like other humans can. In order to build models that are more adaptable to real-world environments, chatbots need to interact with humans ‘in the wild’ from a diverse long-range perspective.”
Meta has been working to address the issue since it first introduced the BlenderBot 1 chat app in 2020. Originally little more than an open-source NLP experiment, by the following year BlenderBot 2 had learned to remember information it had discussed in previous conversations and how to search the web for additional details on a given topic. BlenderBot 3 takes this a step further by evaluating not only the data it pulls from the internet, but also the people it talks to.
When a user logs an unsatisfactory response from the system – currently about 0.16 percent of all training responses – Meta incorporates the user’s feedback back into the model to avoid it repeating the error. The system also uses the Director algorithm, which first generates an answer based on training data and then runs the answer through a classifier to see if it fits a user-defined scale of true and false.
“To generate a sentence, the language modeling and classification mechanisms must match,” Zuckerberg wrote. “Using data indicating good and bad answers, we can train the classifier to penalize low-quality, toxic, contradictory, or repetitive statements and statements that are generally unhelpful.” The system also uses a separate user-weighting algorithm to detecting unreliable or malicious responses from the human interlocutor – essentially teaching the system not to trust what that person has to say.
“Our interactive, live public demo enables BlenderBot 3 to learn from organic interactions with all types of people,” Zuckerberg wrote. “We encourage adults across the United States to try the demo, have natural conversations about interesting topics, and share their responses to further research.”
BB3 is expected to speak more naturally and talkatively than its predecessor, thanks in part to its massively improved OPT-175B speech model, which is nearly 60 times larger than BB2’s model. “We found that BlenderBot 3 offers a 31 percent improvement in overall score on conversational tasks, as assessed by human judgment, compared to BlenderBot 2,” said Zuckerberg. “It is also judged to be twice as knowledgeable while being factually incorrect 47 percent of the time. Compared to GPT3, it ranks as more recent in topical issues 82 percent of the time and more specific 76 percent of the time.”
All products recommended by Engadget are selected by our editorial team independently from our parent company. Some of our stories contain affiliate links. If you buy something through one of these links, we may receive an affiliate commission.
https://www.engadget.com/meta-unleashes-blender-bot-3-upon-the-internet-its-most-competent-chat-ai-to-date-150021062.html?src=rss Meta unleashes BlenderBot 3 upon the internet, its most competent chat AI to date