Researchers Create Artificial Intelligence To Flag Cyberbullying

The Citizen's Guide to the Future
July 11 2012 6:18 PM

Researchers Create Artificial Intelligence To Flag Cyberbullying

Russian teenage girls work on laptop computers while wating for a public bus.
Russian teenage girls work on laptop computers while wating for a public bus.

Photo by YURI KADOBNOV/AFP/Getty Images

Technology is often accused of facilitating bullying among kids. But now some researchers have created an AI system that can recognize abusive language in user-posted text—and perhaps nip it in the bud.

The Software Agents Group of the MIT Media Lab has developed an algorithm that identifies certain groups of words within a post. It then assigns them to 30 themes pertaining to sensitive topics and possible harassment, according to the New Scientist.


The AI flags clusters of words in user-submitted text, displaying how likely the post is related to a certain theme. For example, the terms “cheat,” “trust,” and “upset” might cause the system to identify a comment as referring to a breakup.

“It could tell you how much of that story is talking about weight and appearance issues or the duration of a relationship,” said Karthik Dinakar, a member of the Software Agents Group and MIT Media Lab research assistant, told me.

What happens from there is dependent upon the service using the AI. The system could notify a moderator of the offending post, or go as far as to warn the user about the consequences of cyberbullying.

Dinakar explains that most content flagging systems rely on matched keywords, which tends to be far less accurate than thematic recognition. Additionally, humans monitoring comments can be overwhelmed by the massive number of flagged posts. Facebook, for instance, relies on user reporting when handling abusive content.

The Software Agents Group partnered with MTV’s A Thin Line, a website dedicated to combating bullying by encouraging teenagers to share their experiences. The algorithm analyzed 5,500 of these posts during its initial development.

Birago Jones, also a member of the Software Agents Group, explains the overall goal of this project is to provide social networks with better tools to combat youth-oriented cyberbullying. However, he clarifies that their efforts are “more inspiration that strict problem solving.” He believes it will lead to better discussion between victims, bullies, parents and social network moderators.

This AI is not without its limitations, however. The system requires initial human approval to determine if a theme or word cluster is actually offensive before it can flag subsequent posts. This means that there might be a period of time before it can recognize new abusive terminology. And teenagers are certainly nventive when it comes to creating new ways to hurt one another’s feelings.

Future Tense is a partnership of SlateNew America, and Arizona State University.



Blacks Don’t Have a Corporal Punishment Problem

Americans do. But when blacks exhibit the same behaviors as others, it becomes part of a greater black pathology. 

I Bought the Huge iPhone. I’m Already Thinking of Returning It.

Scotland Is Just the Beginning. Expect More Political Earthquakes in Europe.

Lifetime Didn’t Think the Steubenville Rape Case Was Dramatic Enough

So they added a little self-immolation.

Two Damn Good, Very Different Movies About Soldiers Returning From War

Medical Examiner

The Most Terrifying Thing About Ebola 

The disease threatens humanity by preying on humanity.

Students Aren’t Going to College Football Games as Much Anymore, and Schools Are Getting Worried

The Good Wife Is Cynical, Thrilling, and Grown-Up. It’s Also TV’s Best Drama.

  News & Politics
Sept. 19 2014 9:15 PM Chris Christie, Better Than Ever
Business Insider
Sept. 20 2014 6:30 AM The Man Making Bill Gates Richer
Inside Higher Ed
Sept. 19 2014 1:34 PM Empty Seats, Fewer Donors? College football isn’t attracting the audience it used to.
  Double X
The XX Factor
Sept. 19 2014 4:58 PM Steubenville Gets the Lifetime Treatment (And a Cheerleader Erupts Into Flames)
  Slate Plus
Slate Picks
Sept. 19 2014 12:00 PM What Happened at Slate This Week? The Slatest editor tells us to read well-informed skepticism, media criticism, and more.
Brow Beat
Sept. 19 2014 4:48 PM You Should Be Listening to Sbtrkt
Future Tense
Sept. 19 2014 6:31 PM The One Big Problem With the Enormous New iPhone
  Health & Science
Medical Examiner
Sept. 19 2014 5:09 PM Did America Get Fat by Drinking Diet Soda?   A high-profile study points the finger at artificial sweeteners.
Sports Nut
Sept. 18 2014 11:42 AM Grandmaster Clash One of the most amazing feats in chess history just happened, and no one noticed.