Science of Emotion and Sensibility (감성과학)

Korean Society for Emotion and Sensibility (한국감성과학회)
권호 표지
DOI QR코드

DOI QR Code

Conveying Emotions Through CMC: A Comparative Study of Memoji, Emoji, and Human Face

  • Received : 2023.08.24
  • Accepted : 2023.10.25
  • Published : 2023.12.31
Download PDF
⟨ Previous Next ⟩

Abstract

Emojis and avatars are widely used in online communications, but their emotional conveyance lacks research. This study aims to contribute to the field of emotional expression in computer-mediated communication (CMC) by exploring the effectiveness of emotion recognition, the intensity of perceived emotions, and the perceived preferences for emojis and avatars as emotional expression tools. The following were used as stimuli: 12 photographs from the Yonsei-Face database, 12 Memojis that reflected the photographs, and 6 iOS emojis. The results of this study indicate that emojis outperformed other forms of emotional expression in terms of conveying emotions, intensity, and preference. Indeed, the study findings confirm that emojis remain the dominant form of emotional signals in CMC. In contrast, the study revealed that Memojis were inadequate as an expressive emotional cue. Participants did not perceive Memojis to effectively convey emotions compared with other forms of expression, such as emojis or real human faces. This suggests room for improvement in the design and implementation of Memojis to enhance their effectiveness in accurately conveying intended emotions. Addressing the limitations of Memojis and exploring ways to optimize their emotional expressiveness necessitate further research and development in avatar design.

Keywords

Acknowledgement

This work was supported by the Ministry of Education of the Republic of Korea and the National Research Foundation of Korea (NRF-2022S1A5C2A0309359721).

References

  1. Aldunate, N., & Gonzalez-Ibanez, R. (2017). An integrated review of emoticons in computer-mediated communication. Frontiers in Psychology, 7, 2061.
  2. Antonijevic, S. (2008). From text to gesture online: A microethnographic analysis of nonverbal communication in the second life virtual environment. Information, Community and Society, 11(2), 221-238. https://doi.org/10.1080/13691180801937290
  3. Brechet, C. (2017). Children's recognition of emotional facial expressions through photographs and drawings. The Journal of Genetic Psychology, 178(2), 139-146. https://doi.org/10.1080/00221325.2017.1286630
  4. Cherbonnier, A., & Michinov, N. (2021). The recognition of emotions beyond facial expressions: Comparing emoticons specifically designed to convey basic emotions with other modes of expression. Computers in Human Behavior, 118, 106689.
  5. Chung, K. M., Kim, S., Jung, W. H., & Kim, Y. (2019). Development and validation of the Yonsei face database (YFace DB). Frontiers in Psychology, 10, 2626.
  6. Dalle Nogare, L., Cerri, A., & Proverbio, A. M. (2023). Emojis are comprehended better than facial expressions, by male participants. Behavioral Sciences, 13(3), 278.
  7. Ekman, P., & Friesen, W. V. (1971). Constants across cultures in the face and emotion. Journal of Personality and Social Psychology, 17(2), 124.
  8. Ekman, P., & Friesen, W. V. (2003). Unmasking the face: A guide to recognizing emotions from facial clues (Vol. 10). Ishk.
  9. Fischer, B., & Herbert, C. (2021). Emoji as affective symbols: affective judgments of emoji, emoticons, and human faces varying in emotional content. Frontiers in Psychology, 12, 645173.
  10. Franco, C. L., & Fugate, J. M. (2020). Emoji face renderings: Exploring the role emoji platform differences have on emotional interpretation. Journal of Nonverbal Behavior, 44(2), 301-328. https://doi.org/10.1007/s10919-019-00330-1
  11. Haxby, J. V., Hoffman, E. A., & Gobbini, M. I. (2002). Human neural systems for face recognition and social communication. Biological Psychiatry, 51(1), 59-67. https://doi.org/10.1016/S0006-3223(01)01330-0
  12. Herring, S. C., Dainas, A. R., Long, H. L., & Tang, Y. (2020). Animoji adoption and use: Gender associations with an emergent technology. In Proceedings of the 3rd International Workshop on Emoji Understanding and Applications in Social Media.
  13. Heyselaar, E., Hagoort, P., & Segaert, K. (2017). In dialogue with an avatar, language behavior is identical to dialogue with a human partner. Behavior Research Methods, 49, 46-60. https://doi.org/10.3758/s13428-015-0688-7
  14. Jia, H. M., Park, C. W., & Pol, G. (2015). Cuteness, nurturance, and implications for visual product design. In The Psychology of Design (pp. 168-179). Routledge.
  15. Kaye, L. K., Malone, S. A., & Wall, H. J. (2017). Emojis: Insights, affordances, and possibilities for psychological science. Trends in Cognitive Sciences, 21(2), 66-68. https://doi.org/10.1016/j.tics.2016.10.007
  16. Kiesler, S., Siegel, J., & McGuire, T. W. (1984). Social psychological aspects of computer-mediated communication. American Psychologist, 39(10), 1123.
  17. Koda, T., Ishida, T., Rehm, M., & Andre, E. (2009). Avatar culture: Cross-cultural evaluations of avatar facial expressions. AI & Society, 24, 237-250. https://doi.org/10.1007/s00146-009-0214-5
  18. Konrad, A., Herring, S. C., & Choi, D. (2020). Sticker and emoji use in Facebook Messenger: Implications for graphicon change. Journal of Computer-Mediated Communication, 25(3), 217-235. https://doi.org/10.1093/jcmc/zmaa003
  19. Miller Hillberg, H., Levonian, Z., Kluver, D., Terveen, L., & Hecht, B. (2018). What i see is what you don't get: The effects of (not) seeing emoji rendering differences across platforms. Proceedings of the ACM on Human-Computer Interaction, 2(CSCW), 1-24.
  20. Nowak, K. L., & Fox, J. (2018). Avatars and computer-mediated communication: A review of the definitions, uses, and effects of digital representations. Review of Communication Research, 6, 30-53.
  21. Oleszkiewicz, A., Frackowiak, T., Sorokowska, A., & Sorokowski, P. (2017a). Children can accurately recognize facial emotions from emoticons. Computers in Human Behavior, 76, 372-377. https://doi.org/10.1016/j.chb.2017.07.040
  22. Park, M., & Suk, H. J. (2022). The characteristics of facial emotions expressed in Memojis. Computers in Human Behavior Reports, 8, 100241.
  23. Park, M., & Suk, H. J. (2022, September). New mobile adaptation system for better avatar-mediated communication; facial expressions in Memoji. In Adjunct Publication of the 24th International Conference on Human-Computer Interaction with Mobile Devices and Services (pp. 1-5).
  24. Pessoa, L. (2009). How do emotion and motivation direct executive control?. Trends in Cognitive Sciences, 13(4), 160-166. https://doi.org/10.1016/j.tics.2009.01.006
  25. Rodrigues, D., Prada, M., Gaspar, R., Garrido, M. V., & Lopes, D. (2018). Lisbon Emoji and Emoticon Database (LEED): Norms for emoji and emoticons in seven evaluative dimensions. Behavior Research Methods, 50(1), 392-405.
  26. Stekelenburg, J. J., & de Gelder, B. (2004). The neural correlates of perceiving human bodies: An ERP study on the body-inversion effect. Neuroreport, 15(5), 777-780. https://doi.org/10.1097/00001756-200404090-00007
  27. Suda, M., & Oka, M. (2021, July). Evaluation of the effect of mimicry on facial expression in avatar-mediated communication. In Artificial Life Conference Proceedings 33 (Vol. 2021, No. 1, p. 48). One Rogers Street, Cambridge, MA 02142-1209, USA journals-info@ mit. edu: MIT Press.
  28. Tcherkassof, A., Bollon, T., Dubois, M., Pansu, P., & Adam, J. M. (2007). Facial expressions of emotions: A methodological contribution to the study of spontaneous and dynamic emotional faces. European Journal of Social Psychology, 37(6), 1325-1345.
  29. Tossell, C. C., Kortum, P., Shepard, C., Barg-Walkow, L. H., Rahmati, A., & Zhong, L. (2012). A longitudinal study of emoticon use in text messaging from smartphones. Computers in Human Behavior, 28(2), 659-663. https://doi.org/10.1016/j.chb.2011.11.012
  30. Van Der Land, S. F., Schouten, A. P., Feldberg, F., Huysman, M., & van den Hooff, B. (2015). Does avatar appearance matter? How team visual similarity and member-avatar similarity influence virtual team performance. Human Communication Research, 41(1), 128-153.
  31. Young, S. G., & Hugenberg, K. (2010). Mere social categorization modulates identification of facial expressions of emotion. Journal of Personality and Social Psychology, 99(6), 964.