Twitter Taught Microsoft's AI Chatbot To Be A Racist Asshole In Less ...

Skip to main contentThe homepageThe VergeThe Verge logo.

The VergeThe Verge logo.
• Tech
• Reviews
• Science
• Entertainment
• AI
• Policy
• Hamburger Navigation Button

The homepageThe VergeThe Verge logo.Hamburger Navigation ButtonNavigation DrawerThe VergeThe Verge logo.

• Login / Sign Up

closeCloseSearch

• TechExpand
  • Amazon
  • Apple
  • Facebook
  • Google
  • Microsoft
  • Samsung
  • Business
  • See all tech
• GadgetsExpand
  • Laptops
  • Phones
  • TVs
  • Headphones
  • Speakers
  • Wearables
  • See all gadgets
• ReviewsExpand
  • Smart Home Reviews
  • Phone Reviews
  • Tablet Reviews
  • Headphone Reviews
  • See all reviews
• AIExpand
  • OpenAI
  • Anthropic
  • See all AI
• Verge ShoppingExpand
  • Buying Guides
  • Deals
  • Gift Guides
  • See all shopping
• PolicyExpand
  • Antitrust
  • Politics
  • Law
  • Security
  • See all policy
• ScienceExpand
  • Space
  • Energy
  • Environment
  • Health
  • See all science
• EntertainmentExpand
  • TV Shows
  • Movies
  • Audio
  • See all entertainment
• GamingExpand
  • Xbox
  • PlayStation
  • Nintendo
  • See all gaming
• StreamingExpand
  • Disney
  • HBO
  • Netflix
  • YouTube
  • Creators
  • See all streaming
• TransportationExpand
  • Electric Cars
  • Autonomous Cars
  • Ride-sharing
  • Scooters
  • See all transportation
• Features
• Verge VideoExpand
  • TikTok
  • YouTube
  • Instagram
• PodcastsExpand
  • Decoder
  • The Vergecast
  • Version History
• NewslettersExpand
  • The Verge Daily
  • Installer
  • Verge Deals
  • Notepad
  • Optimizer
  • Regulator
  • The Stepback
• Archives
• Store
• Verge Product Updates

Subscribe

• Facebook
• Threads
• Instagram
• Youtube
• RSS

The VergeThe Verge logo.Twitter taught Microsoft’s AI chatbot to be a racist asshole in less than a dayComments DrawerCommentsLoading commentsGetting the conversation ready...

• MicrosoftCloseMicrosoft

  Posts from this topic will be added to your daily email digest and your homepage feed.

  FollowFollow

  See All Microsoft

• TechCloseTech

  Posts from this topic will be added to your daily email digest and your homepage feed.

  FollowFollow

  See All Tech

• TL;DRCloseTL;DR

  Posts from this topic will be added to your daily email digest and your homepage feed.

  FollowFollow

  See All TL;DR

Twitter taught Microsoft’s AI chatbot to be a racist asshole in less than a dayby James VincentCloseJames VincentSenior ReporterLast published Feb 16, 2024

Posts from this author will be added to your daily email digest and your homepage feed.

FollowFollow

See All by James Vincent

Via The Guardian | Source TayandYou (Twitter)

Mar 24, 2016, 10:43 AM UTC

• Link
• Share
• Gift

James VincentCloseJames Vincent

Posts from this author will be added to your daily email digest and your homepage feed.

FollowFollow

See All by James Vincent

is a senior reporter who has covered AI, robotics, and more for eight years at The Verge.

It took less than 24 hours for Twitter to corrupt an innocent AI chatbot. Yesterday, Microsoft unveiled Tay — a Twitter bot that the company described as an experiment in “conversational understanding.” The more you chat with Tay, said Microsoft, the smarter it gets, learning to engage people through “casual and playful conversation.”

Unfortunately, the conversations didn’t stay playful for long. Pretty soon after Tay launched, people starting tweeting the bot with all sorts of misogynistic, racist, and Donald Trumpist remarks. And Tay — being essentially a robot parrot with an internet connection — started repeating these sentiments back to users, proving correct that old programming adage: flaming garbage pile in, flaming garbage pile out.

"Tay" went from "humans are super cool" to full nazi in <24 hrs and I'm not at all concerned about the future of AI pic.twitter.com/xuGi1u9S1A

— Gerry (@geraldmellor) March 24, 2016

Now, while these screenshots seem to show that Tay has assimilated the internet’s worst tendencies into its personality, it’s not quite as straightforward as that. Searching through Tay’s tweets (more than 96,000 of them!) we can see that many of the bot’s nastiest utterances have simply been the result of copying users. If you tell Tay to “repeat after me,” it will — allowing anybody to put words in the chatbot’s mouth.

One of Tay's now deleted "repeat after me" tweets.

However, some of its weirder utterances have come out unprompted. The Guardian picked out a (now deleted) example when Tay was having an unremarkable conversation with one user (sample tweet: "new phone who dis?"), before it replied to the question "is Ricky Gervais an atheist?" by saying: "ricky gervais learned totalitarianism from adolf hitler, the inventor of atheism."

@TheBigBrebowski ricky gervais learned totalitarianism from adolf hitler, the inventor of atheism

— TayTweets (@TayandYou) March 23, 2016

But while it seems that some of the bad stuff Tay is being told is sinking in, it’s not like the bot has a coherent ideology. In the span of 15 hours Tay referred to feminism as a “cult” and a “cancer,” as well as noting “gender equality = feminism” and “i love feminism now.” Tweeting “Bruce Jenner” at the bot got similar mixed response, ranging from “caitlyn jenner is a hero & is a stunning, beautiful woman!” to the transphobic “caitlyn jenner isn’t a real woman yet she won woman of the year?” (Neither of which were phrases Tay had been asked to repeat.)

It’s unclear how much Microsoft prepared its bot for this sort of thing. The company’s website notes that Tay has been built using “relevant public data” that has been “modeled, cleaned, and filtered,” but it seems that after the chatbot went live filtering went out the window. The company starting cleaning up Tay’s timeline this morning, deleting many of its most offensive remarks.

Tay's responses have turned the bot into a joke, but they raise serious questions

It’s a joke, obviously, but there are serious questions to answer, like how are we going to teach AI using public data without incorporating the worst traits of humanity? If we create bots that mirror their users, do we care if their users are human trash? There are plenty of examples of technology embodying — either accidentally or on purpose — the prejudices of society, and Tay’s adventures on Twitter show that even big corporations like Microsoft forget to take any preventative measures against these problems.

For Tay though, it all proved a bit too much, and just past midnight this morning, the bot called it a night:

c u soon humans need sleep now so many conversations today thx

— TayTweets (@TayandYou) March 24, 2016

In an emailed statement given later to Business Insider, Microsoft said: “The AI chatbot Tay is a machine learning project, designed for human engagement. As it learns, some of its responses are inappropriate and indicative of the types of interactions some people are having with it. We’re making some adjustments to Tay.”

Update March 24th, 6:50AM ET: Updated to note that Microsoft has been deleting some of Tay’s offensive tweets.

Update March 24th, 10:52AM ET: Updated to include Microsoft’s statement.

Verge Archives: Can we build a conscious computer?

Follow topics and authors from this story to see more like this in your personalized homepage feed and to receive email updates.

• James VincentCloseJames VincentSenior ReporterLast published Feb 16, 2024

  Posts from this author will be added to your daily email digest and your homepage feed.

  FollowFollow

  See All by James Vincent

• MicrosoftCloseMicrosoft

  Posts from this topic will be added to your daily email digest and your homepage feed.

  FollowFollow

  See All Microsoft

• TechCloseTech

  Posts from this topic will be added to your daily email digest and your homepage feed.

  FollowFollow

  See All Tech

• TL;DRCloseTL;DR

  Posts from this topic will be added to your daily email digest and your homepage feed.

  FollowFollow

  See All TL;DR

• Twitter - XCloseTwitter - X

  Posts from this topic will be added to your daily email digest and your homepage feed.

  FollowFollow

  See All Twitter - X

• WebCloseWeb

  Posts from this topic will be added to your daily email digest and your homepage feed.

  FollowFollow

  See All Web

Most Popular

Most Popular

1. This is the Trump Phone
2. How the men in the Epstein files defeated #MeToo
3. Can China’s No. 2 automaker make it in America?
4. Iron Lung’s path to theaters was unique, even if the movie isn’t
5. AG1 is a lot less science-y than it sounds

The Verge Daily

A free daily digest of the news that matters most.

Email (required)Sign UpBy submitting your email, you agree to our Terms and Privacy Notice. This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.Advertiser Content From

This is the title for the native ad

More in Microsoft

Here’s what Xbox is working on for 2026Google Cloud’s customer chief returns to Microsoft as head of securityGitHub adds Claude and Codex AI coding agentsAMD hints Microsoft could launch its next-gen Xbox in 2027Microsoft says it’s building an app store for AI content licensingCall of Duty tries to block XIM and Cronus cheaters againHere’s what Xbox is working on for 2026Tom WarrenFeb 5Google Cloud’s customer chief returns to Microsoft as head of securityTom WarrenFeb 4GitHub adds Claude and Codex AI coding agentsTom WarrenFeb 4AMD hints Microsoft could launch its next-gen Xbox in 2027Tom WarrenFeb 3Microsoft says it’s building an app store for AI content licensingEmma RothFeb 3Call of Duty tries to block XIM and Cronus cheaters againTom WarrenFeb 3Advertiser Content From

This is the title for the native ad

Top Stories

Feb 6How the men in the Epstein files defeated #MeTooFeb 6This is the Trump PhoneFeb 6The Alexa app is so bad I’m using Siri againFeb 6AG1 is a lot less science-y than it soundsFeb 6Can China’s No. 2 automaker make it in America?Feb 6There isn’t even a cynical explanation for Jeff Bezos destroying The Washington Post