Study examines how we understand emojis

A study was conducted to understand how people perceive emojis in their daily lives.

The study has been published in the 'Computers in Human Behavior Journal'.

Emojis are small image icons that originated in Japan: Japanese "e" means picture, "moji" means word.

"We set out to explore to what extent reading emoji is more similar to reading words or interpreting pictures," explained Tatjana Scheffler.

To this end, the researchers conducted an online study where participants read sentences in which some words were replaced by emoji, and measured the reading time for each word. Subsequently, they asked questions to test whether the participants understood the sentences.

"As we had suspected and as other studies suggest, our participants easily understood sentences with emoji instead of nouns," the researcher pointed out.

The comprehension questions were even answered correctly slightly more often - but not significantly so - following the sentences containing emoji than following the same sentences without any emoji.

Normal words, however, do not only lead to the meaning while they are being processed; they also have other properties that are activated during reading: for example their pronunciation. And words can be homophones: when two words have the same pronunciation but different meanings, for example, the knight and the night. The researchers wanted to find out if emoji can also trigger this phenomenon.

"Therefore, we had our test participants read sentences in which the emoji doesn't show the intended object but its homophone," Tatjana Scheffler explained.

"We showed that the sentences were almost always understood correctly, even when homophones were used," points out Tatjana Scheffler.

"This indicates that an emoji can be broken down into a complete 'lexicon entry' that contains information about its pronunciation. Based on this information, the participants then derive the other, homophonic meaning," Scheffler added.

Still, the team also showed that the reading times differed considerably. The average reading time for a written-out word is approximately 450 milliseconds, for a corresponding emoji approx. 800 milliseconds and for an emoji depicting a homophone more than 900 milliseconds.

This leads the researchers to conclude that the image must be interpreted first when reading emoji. Emoji are generally less recognisable and more unusual, and therefore not as easy to read as written words.

"This is also supported by the fact that those test participants who self-assessed as using emoji often, read matching emoji more quickly," Tatjana Scheffler said.

Since readers can also derive a pronunciation for emoji just like for words, homophone emoji are also readily understood. This takes longer, however, because the visual information has to be suppressed and the meaning of the homophonic word has to be recalled.

"In this case, the fact that someone is used to emoji is no longer helpful. Participants who use emoji more often read the homophone emoji just as slowly as the others," Tatjana Scheffler concluded.

Together with the team at Charite, she plans to conduct a similar study with people living with schizophrenia. Since some of them have difficulty identifying non-literal meanings, a comparison with a control group should provide additional insight into language processing, the linguistic structure of non-literal meaning and the linguistic effects of schizophrenia.