Emoticons and emojis as evidence in court.

Courts Are Evaluating Emoticons as Evidence. Too Bad Nobody Knows What They Mean.

Courts Are Evaluating Emoticons as Evidence. Too Bad Nobody Knows What They Mean.

Notes on the culture of the Internet.
Oct. 26 2015 4:34 PM

Exhibit A: ;-)

Courts are evaluating emoticons as evidence—but nobody really knows what they mean.

emoji ruling.

Photo illustration by Juliana Jiménez. Photo by Deborah Cheramie/Thinkstock.

Last spring, a U.S. District Court judge in Michigan was asked to rule on the meaning of “:-P.” The case concerned a University of Michigan law student who was investigated by local cops after a female classmate reported that he had been stalking and harassing her online. When prosecutors declined to press charges, the student sued the woman, his school, and the cops, claiming that the investigation had been launched without cause and executed unlawfully. In his defense, the student argued that several text messages he’d sent to a friend—in which he called himself a “petty bastard” who wanted to make his classmate “feel crappy” and experience “deep dark pits of depression”—should never have been taken seriously by police because he’d thrown a couple of wry emoticons in, including one indicating a stuck-out tongue. A proper understanding of the symbol, the man argued, would have led a reasonable officer to understand that he was not “sadistically bloodthirsty for revenge” against the woman, but rather just “deeply unhappy” about the status of their friendship. The judge disagreed, writing that the emoticon “does not materially alter the meaning of the text message.”

Amanda Hess Amanda Hess

Amanda Hess is a David Carr fellow at the New York Times. Follow her on Twitter.

When people first started making faces with their keyboards, emoticons were perceived as a tool that would help clarify the tone of online texts. In 1982, Carnegie Mellon University computer science professor Scott Fahlman noticed a problem with the rudimentary online bulletin board his department used to share messages between students, faculty, and staff. Department members liked to make jokes on the forum, but the flat affect of the virtual environment sometimes made it hard to land a punch line. If “someone made a sarcastic remark, a few readers would fail to get the joke, and each of them would post a lengthy diatribe in response,” Fahlman later wrote of the board. “That would stir up more people with more responses, and soon the original thread of the discussion was buried.” On Sept. 19, 1982, Fahlman posted this message:

I propose … the following character sequence for joke markers:
Read it sideways. Actually, it is probably more economical to mark things that are NOT jokes, given current trends. For this, use

Fahlman is widely credited with inventing the keyboard-rendered facial expression. But as his emoticons gained traction on early computer systems, they took on emotional lives of their own. The frown emoticon that Fahlman pitched to mark a serious post “quickly evolved into a marker for displeasure, frustration, or anger,” Fahlman wrote. “Within a few months, we started seeing the lists with dozens of ‘smilies’: open-mouthed surprise, person wearing glasses, Abraham Lincoln, Santa Claus, the pope, and so on,” each of which conjured a slightly different vibe when deployed in a new context. Fahlman had devised the emoticons to help clarify the meaning of online texts. But he soon realized that the meanings of emoticons themselves were highly subjective. And that was before we started communicating our feelings through endless combinations of tiny, colorful images—pouting cats, silly ghosts, and grinning piles of poo.

What do emoticons mean? As keyboard-crafted emoticons—and their slicker younger sisters, emoji—have become increasingly central to online speech, the “rampant interplay of words and symbols compounds the complexity of our language,” linguist and emoji enthusiast Tyler Schnoebelen argued last month. Our old-school text keyboards are now enhanced by thousands of digital symbols with no fixed emotional resonance, clear dictionary definition, or established grammatical rules for interpreting them in the various contexts in which they appear. As Google software engineer Mark Davis told the New York Times recently, “I can tell you, using language, I need to go get a haircut, but only if I can get there by 3 p.m., and otherwise I have to pick up the kids. You try to express that in emojis and you get a series of symbols that people could interpret in a thousand different ways.” Last Wednesday, Apple expanded the average texter’s emotional range even further by adding 184 new emojis to its standard iPad and iPhone keyboards, including an eye-rolling face, an ever-so-slightly smiling face, a hand flipping the bird, and a man wearing a business suit and levitating—whatever that means.

The emotional flexibility of emojis is part of what makes the symbols so fun to use. What digital native would begrudge the eggplant its transformation from sad supermarket fruit to celebrated phallic symbol? But sometimes the meaning really does matter. In January, the judge presiding over the Silk Road trial ruled that all online communications entered as evidence in the case must include the original emoticons as they appeared throughout the text. In June, the U.S. Supreme Court overturned the conviction of Anthony Elonis, a Pennsylvania man who had been found guilty of threatening his ex-wife by way of virulent public Facebook postings. Part of Elonis’ defense relied on a stuck-out tongue emoticon he’d keyed in at the end of one of his posts: The “:-P” proved he didn’t seriously want to harm his ex-wife, he said. And last year, British attorney Laura Saife published a Handbook of Social Media and the Law that noted how British courts are beginning to interpret emoticons in the service of “determining the state of mind of the poster” or “considering if there has been an element of malice” in cases of defamation, stalking, and threats. Every case that includes a digital record presents a high-stakes venue for arguing over what emoticons really mean.

Courts have always had to interpret nonverbal cues, like shrugs and winks, that arise in face-to-face conversations. But digital symbols are something new. In recent years, academics and engineers have made some preliminary attempts to establish a more formal understanding of emoticons and emojis. Google’s Mark Davis is also a co-founder of the Unicode Consortium, a group of tech and language experts that works to standardize emojis (and other text characters) to ensure they work on all the major computer systems used around the world. The consortium first recognized emojis in 1993, when it approved 76 symbols for inclusion in the Unicode Standard. Today, its list is 1,281 emojis long. Apple, which complies with the standard, released its latest 184 emojis only after the consortium gave the symbols its blessing; next May, the consortium will debate yet another batch of proposed emojis, and decide whether the selfie, shrug, and avocado will be cleared for inclusion. Unicode also standardizes an English “name” for each emoji— “Face With Tears of Joy,” “Face Savouring Delicious Food”—which doubles as the seed of an official definition.


Meanwhile, a group of Slovenian researchers have been working on building an emotional scale for charting the likely sentiment attached to each little glyph. In a study published last month, the researchers compiled 1.6 million tweets written in 13 European languages, commissioned 83 humans to code the text of each tweet as positive, negative, or neutral in emotional valence, then analyzed the sentimental context within which hundreds of emojis are typically deployed. Their published “Emoji Sentiment Ranking” produces some obvious insights: The smiley face emoji is typically used in positive contexts on Twitter, while the crying cat emoji is appended to more negative-sounding tweets. But other conclusions were counterintuitive: The “disappointed but relieved face” was generally used in positive contexts, while the “expressionless face” is appended to negative statements. A few of the researchers’ findings are just mystifying: The bento box emoji is used in largely negative contexts, while the panda face is associated with less positive emotions than most other animals featured on the emoji keyboard. How judges and attorneys will interpret the bento box and panda face remains to be seen.

So far, efforts to build a unified emotional context for hundreds of emojis used by millions of people around the world have failed. The Emoji Sentiment Ranking relies on a rudimentary emotional scale, and it doesn’t acknowledge that emojis are often used to soften or counter the emotion present in a written text, or are just casually appointed to a tweet like a non sequitur. In one 2013 Twitter defamation case, a British judge wrote that an emoticon is tantamount to a digital “stage direction” that instructs readers on how to imagine the writer’s facial expression as she formulated the statement. But he cautioned that even within that fairly literal interpretation of the emoji, parties could spar over whether the face had been presented as a “sincere” communication of emotion or an “ironic” prank. As Schnoebelen told me: “The Joker’s smile is not reassuring.”

In his linguistics dissertation at Stanford, Schnoebelen (who is now the founder and chief analyst of the text analytics company Idibon) noted that the emoticons for a winky face “;-)” and that for a stuck-out tongue “:-P” are trickier to interpret than the standard smile or frown. Schnoebelen’s research has shown that the wink and the tongue are often used to denote teasing or flirting, and interpreting the subtext of those activities requires the reader to understand the power dynamics between texter and recipient. Depending on the context, the emoticons can read as either creepy or cute.

Often, these are the emoticons that are most heatedly contested in court. Last year, a Michigan appeals court ruled that an anonymous comment posted to an online message board that appeared to accuse a city worker of corruption was negated by the inclusion of a tongue-out emoji. “The use of the ‘:P' emoticon makes it patently clear that the commenter was making a joke,” the court ruled; “a ‘:P' emoticon is used to represent a face with its tongue sticking out to denote a joke or sarcasm. Thus, a reasonable reader could not view the statement as defamatory.” And this August, a Delaware judge interpreted a winking emoticon as a menacing signal. When a man secretly bought a plane ticket next to a colleague who clearly did not wish to see him again, “surprised” her on her flight to Paris, then boasted to friends in a text—“Was next to [the woman] on the plane to Paris and she switched seats;)”—the man claimed that the wink showed he had just been joking around. The judge disagreed, interpreting the wink as a sign that the man “was amused by yet another opportunity to harass” his target.

The most absurd emoticon-related argument to be parsed by the courts might be one proffered by a Texas man prosecuted for violently sexually assaulting an acquaintance in 2011. The defendant argued that his victim had preemptively consented to sex through several text messages they had exchanged prior to the incident, which he had interpreted as sexually suggestive. This text exchange had culminated in the victim texting the man “a ‘winkie face’ emoticon.” At trial, the defendant argued that “there were at least two consents for sex—three, if one counts the ‘winkie face.’ ” The court did not “count” the winkie face (or the other two supposed indications of prior consent-by-text), and sentenced the man to 10 years in prison. Emoticons are flexible, but they’re not that flexible. As Schnoeblen told me: “To my knowledge, no winkie face has ever meant ‘I consent to sex for all time.’ ”