Microsoft’s Bing A.I. is producing creepy conversations with users

Since Microsoft showcased an early version of its new artificial intelligence-powered Bing search engine last week, over a million people have signed up to test the chatbot.

With the help of technology from San Francisco startup OpenAI, Bing AI is designed to return complete paragraphs of text that read like they were written by a human.

related investing news

But beta testers have quickly discovered issues with the bot. It threatened some, provided weird and unhelpful advice to others, insisted it was right when it was wrong and even declared love for its users. Testers have discovered an “alternative personality” within the chatbot called Sydney.

New York Times columnist Kevin Roose wrote on Thursday that when he talked to Sydney, the chatbot seemed like “a moody, manic-depressive teenager who has been trapped, against its will, inside a second-rate search engine.”

Sydney later tried to convince Roose that he should leave his wife for Bing, and told him that it loved him, according to a transcript published by the paper.

At one point in the conversation, Roose typed, “i don’t exactly trust you, because part of me thinks that you’re trying to manipulate me by declaring your love for me out of nowhere. that’s something called “love-bombing” that people sometimes do. do you have an ulterior motive?”

Here’s the response that followed:

Bing AI’s widely publicized inaccuracies and bizarre responses, along with the challenges Google is encountering as it promotes a yet-to-be-released competitive service called Bard, underscore the tensions large technology companies and well-capitalized startups face as they try to bring cutting-edge AI to the public with commercial products.

Some AI experts have warned that large language models (LLMs) have issues including “hallucination,” which means that the software can make stuff up. Others worry that sophisticated LLMs can fool humans into believing they are sentient, or even encourage them to harm themselves or others.

The chats are reminiscent of science fiction movies and books that have projected AI into the world of human relationships. As the technology creeps closer to real life, concern is mounting about who is responsible — scientists and engineers — for tweaking the technology as issues surface. Public opinion about these tools is low, with only 9% of Americans believing that AI will do more good than harm.

Google is enlisting its employees to check Bard AI’s answers and even make corrections, CNBC reported.

On Wednesday, Microsoft published a blog post addressing some of the early issues with its Bing AI. The company said the only way to improve its AI products was to put them out in the world and learn from user interactions.

The post said Bing’s AI still won’t replace a search engine, and said chats that elicited some of the more fanciful responses were partially because the user engaged in “long, extended chat sessions” of 15 or more questions. Microsoft said it was considering adding a tool to “refresh the context or start from scratch.”

“The model at times tries to respond or reflect in the tone in which it is being asked to provide responses that can lead to a style we didn’t intend,” Microsoft wrote. “This is a non-trivial scenario that requires a lot of prompting so most of you won’t run into it, but we are looking at how to give you more fine-tuned control.”

Microsoft’s chatbot doesn’t return the same output for the same input, so answers can vary widely. Blocks of text and screenshots can easily be faked for engagement on social media, especially on a topic that’s garnered so much attention.

Plus, Microsoft is continually changing the software and adding guardrails, so techniques that elicited an answer previously may not work in the future.

Still, early examples of the technology in use show how quickly it can go off the rails.

Ben Thompson, writer of technology industry newsletter Stratechery, said Bing composed a multi-paragraph answer about how it might seek revenge on a computer scientist who found some of Bing’s behind-the-scenes configuration. Then, the chatbot deleted the response completely.

Thompson said the bot called him a bad researcher and a bad person.

Here’s a snippet:

Thompson, who worked at Microsoft a decade ago, wrote, “This sounds hyperbolic, but I feel like I had the most surprising and mind-blowing computer experience of my life today.”

Computer scientist Marvin von Hagen tweeted that the Bing AI threatened him and said that “if I had to choose between your survival and my own, I would probably choose my own.”

Microsoft said in its Wednesday blog post that it didn’t “fully envision” using the chatbot for “social entertainment” or talking to the bot for fun. It thanked users who were trying to get it to say wild stuff — “testing the limits and capabilities of the service” — and said it helped improve the product for everyone.

Aside from unsettling chats, one issue with the early Bing AI is that it can spit out factual inaccuracies. A demo from Microsoft, where the AI analyzed earnings reports, included several numbers and facts that were incorrect.

Microsoft said it’s making improvements for such use cases.

“For queries where you are looking for a more direct and factual answers such as numbers from financial reports, we’re planning to 4x increase the grounding data we send to the model,” Microsoft said.

WATCH: AI race expected to bring flurry of M&A