Instagram uses AI to ask "Are you sure?" before posting harmful comments

Artificial intelligence will step in to suggest bullies reconsider before saying something hurtful

Published July 13, 2019 12:59PM (EDT)

LONDON, ENGLAND - AUGUST 03: The Instagram app logo is displayed on an iPhone on August 3, 2016 in London, England.  (Photo by Carl Court/Getty Images) (Photo by Carl Court/Getty Images)
LONDON, ENGLAND - AUGUST 03: The Instagram app logo is displayed on an iPhone on August 3, 2016 in London, England. (Photo by Carl Court/Getty Images) (Photo by Carl Court/Getty Images)

This article originally appeared on GearBrain.
Gear Brain

Bullying is a problem on all forms on social media, but can be particularly harmful in the comments section of Instagram posts.

To help prevent this, Instagram has introduced a new artificial intelligence-powered feature which reads the comment you are about to post if it detects something negative, then asks if you are sure you want to make the post.

An example given by the Facebook-owned image sharing platform shows a message saying "You are so ugly and stupid." As the comment is being posted, the app steps in and asks: "As you sure you want to post this?"

Instagram then gives the commenter the option to undo their message, preventing it from appearing on the image, and stopping the recipient from being notified by it. The social network says it is "asking people to rethink comments that seem similar to others that have been reported."

There is also the option for users posting comments to suggest Instagram has made a mistake. We imagine a real challenge here will be for the artificial intelligence to understand context, sarcasm, and when the poster is joking with the offensive words they use.

instagram-screengrab
AI suggests users rethink potentially harmful posts. (Instagram)

A second system announced by Instagram this week is called Restrict. This is a way for users to sever ties with a bully, but without the bully knowing. A new option comes up if you're trying to delete a comment, allowing you to restrict the comment so that only the original person who wrote the statement can view it — but no one else.

Instagram users can make a restricted person's comments visible to all by approving them, but if you choose to they can be ignored and not read at all, by you or anyone else. All future comments from that person are also restricted on that Instagram page going forward, however the bully will not know. This process is also known as shadow-banning, where the banned person does not know they have had their abilities restricted.

Once restricted, the person will also not be able to see when you are online on Instagram, and when you have read their direct messages. Instagram users will start to see the feature testing soon.

"We've heard from young people in our community that they're reluctant to block, unfollow, or report their bully because it could escalate the situation, especially if they interact with their bully in real life," said Instagram. "It's our responsibility to create a safe environment...This has been an important priority for us for some time, and we are continuing to invest in better understanding and tackling this problem."


By Alistair Charlton

MORE FROM Alistair Charlton


Related Topics ------------------------------------------

All Salon Artificial Intelligence Bullying The Economy & Innovation Gearbrain Instagram Social Media