Deepfake in Face Perception Research

981

Abstract

Presents the state-of-the-art Deepfake face replacement image collage method, an artificial intelligence (AI) product that can be used to create high-quality, realistic videos with a fake or replaced face, with no obvious signs of manipulation. Based on the DeepFaceLab (DFL) application, the process of creating video images of an “impossible face” is described step by step. The results of the experiments of studying the perception patterns of the moving “impossible face” and their differences in statics and dynamics are presented. The stimuli were two DFL-generated models of virtual sitters with impossible faces: a video image of a chimerical face, in which the right and left sides belong to different people, and a Tatchered face with the eyes and mouth areas rotated by 180°. It was shown that the phenomena of perception of the “impossible face”, registered earlier under static conditions (integrity of perception of the split image, distraction and inversion effect), are preserved and acquire a new content when dynamic models are exposed. In contrast to the collaged images, the original faces in statics and motion, regardless of egocentric orientation, are evaluated positively at the level of high values. Under all tested conditions the gender of the virtual sitter is determined adequately, the perceived age is overestimated. Estimates of the virtual sitter’s emotions from his video images are differentiated into basic (stable) and additional (changing) states, the ratio of which depends on the content of a particular episode. Deepfake image synthesis technology significantly expands the possibilities of psychological research of interpersonal perception. The use of digital technologies simplifies the creation of “impossible face” stimulus models necessary for in-depth study of representations of the human inner world, and creates a need for new experimental-psychological procedures corresponding to a higher level of ecological and social validity.

General Information

Keywords: IT-Deepfake, DeepFaceLab, interpersonal perception, video facial imaging, impossible face, virtual sitter, dynamic and static facial expressions, chimeric face, tethered face, perceptual integrity, dy¬namic distractor, inversion effect, facial attractiveness, basic emotional expressions

Journal rubric: Psychology of Digital Reality

DOI: https://doi.org/10.17759/exppsy.2021000001

Funding. The research was performed within the framework of the state order of the Ministry of Education of the Russian Federation №730000Ф.99.1.БВ09АА00006

For citation: Barabanschikov V.A., Marinova M.M. Deepfake in Face Perception Research. Eksperimental'naâ psihologiâ = Experimental Psychology (Russia), 2021. Vol. 14, no. 1, pp. 4–19. DOI: 10.17759/exppsy.2021000001. (In Russ., аbstr. in Engl.)

References

  1. Barabanshchikov V.A. Ekspressii litsa i ikh vospriyatie [Facial expressions and their perception]. Moscow: IP RAN Publ., 2012. (In Russ.).
  2. Barabanschikov V.A. (otv. red.) Kognitivnye mekhanizmy neverbal’noi kommunikatsii. M.: Kogito- Tsentr, 2017. (In Russ.).
  3. Barabanshchikov V.A., Korol’kova O.A. Vospriyatie ekspressii «zhivogo» litsa [Perception of “live” facial expressions]. Eksperimental’naya psikhologiya = Experimental Psychology, 2020, Vol. 8, no. 3. pp. 55— 73. DOI:10.17759/exppsy.2020130305 (In Russ.).
  4. Barabanshchikov V. A., Marinova M. M. Vosprinimaemye kachestva i otsenka ekspressii podvizhnogo khimericheskogo litsa [Perceived qualities and evaluation of expressions of the mobile chimeric face]. Litso cheloveka v kontekstakh prirody, tekhnologii i kul’tury [The Face of Man in Contexts of Nature, Technology, and Culture] K.I. Anan’eva, V.A. Barabanshchikov, A. A. Demidov (eds.). Moscow: Moskovskii institut psikhoanaliza, Kogito-Tsentr, 2020. (In Press). (In Russ.).
  5. Barabanshchikov V.A., Marinova M.M. Vospriyatie videoizobrazhenii khimericheskogo litsa [Perception of video images of a chimerical face]. Poznanie i perezhivanie = Cognition and Experience, 2020, Vol. 1, no. 1, pp. 112—134. doi:10.51217/cogexp_2020_01_01_07 (In Russ).
  6. Barabanshchikov V.A., Marinova M.M., Abramov A.D. The Virtual Personality of the Moving Thatcherized Person. Psikhologicheskaya nauka i obrazovanie = Psychological Science and Education. Vol. 23, no. 1, pp. 5—18. DOI:10.17759/pse.2021000001 (In Russ.).
  7. Barabanshchikov V.A., Marinova M.M., Abramov A.D. Virtual’naya lichnost’ tetcherizirovannogo litsa v statike i dinamike [Virtual Personality of the Thatcherized Person in Statics and Dynamics] // Eksperimental’naya psikhologiya v sotsial’nykh praktikakh [Experimental Psychology in Social Practices] V.A. Barabanshchikov, V.V. Selivanov (eds). Moscow: Universum, 2020. pp. 7—16. (In Russ.).
  8. Barabanshchikov V.A., Marinova M.M., Abramov A.D. Illyuziya Margaret Tetcher na podvizhnom litse [The illusion of Margaret Thatcher on a moving face]. Sbornik materialov ezhegodnoi konferentsii po kognitivnoi nauke, posvyashchennoi pamyati Dzh. S. Brunera, «Psikhologiya poznaniya: nizkourovnevye i vysokourovnevye protsessy» (g. Yaroslav’, 18—19 dekabrya 2020 g.) [Proceedings of the annual conference on cognitive science in memory of J. S. Bruner “Psychology of cognition: low-level and high-level processes”]. Yaroslavl: Yaroslavl State University. P.G. Demidov, 2021. (In press).
  9. Anna. DeepFaceLab: instrument dlya generatsii dipfeikov [Electronic resource] [DeepFaceLab: deepfake generation tool]. Neurohive. URL: https://neurohive.io/ru/frameworki/deepfacelab-instrument-dlya-generacii-dipfejkov/ (Accessed: 30.11.2020) (In Russ).
  10. Chawla R. Deepfakes: How a pervert shook the world. International Journal of Advance Research and Development, 2019. Vol. 4, no. 6, pp. 4—8.
  11. Demuthova S., Démuth A. Handedness and the Preference of the Visual Field in Face Perception // European Scientific Journal. 2018. http://doi.org/10.19044/esj.2018.c3p8
  12. Dole M., Méary D., Pascalis O. Modifications of Visual Field Asymmetries for Face Categorization in Early Deaf Adults: A Study With Chimeric Faces. Front. Psychol. 2017.Vol. 8, P. 30. http://doi. org/10.3389/fpsyg.2017.00030
  13. Maras M., Alexandrou A. Determining Authenticity of Video Evidence in the Age of Artificial Intelligence and in the Wake of Deepfake Videos. International Journal of Evidence and Proof. 2018, Vol. 23, pp. 255—262. DOI:10.1177/1365712718807226
  14. Perov I., Gao D., Chervoniy N., Liu K., Marangonda S., Ume C., Mr. Dpfks., Facenheim C. F., RP L., Jiang J., Zhang S., Wu P., Bo zhou, Zhang W. DeepFaceLab: A simple, flexible and extensible face swapping framework [Electronic resource]. 2020. ArXiv: Computer Vision and Pattern Recognition. URL: https:// arxiv.org/pdf/2005.05535.pdf (Accessed: 13.10.2020).
  15. Stephanie S.A.H. Blom, Henk Aarts & Gün R. Semin. Lateralization of facial emotion processing and facial mimicry. Laterality, 2020. Vol. 25, no. 3, pp. 259—274. DOI:10.1080/1357650X.2019.1657127
  16. Thompson P. Margaret Thatcher: A new illusion. Perception, 1980. V. 9, P. 483—484. DOI:10.1068/ p090483
  17. Weibert K., Müller V., Sänger J. Familiarity abolishes right-hemispheric bias in face perception. Journal of Vision, 2017. vol. 17. DOI: 10.1167/17.10.998.
  18. Westerlund M. The Emergence of Deepfake Technology: A Review // Technology Innovation Management Review. 2019.Vol. 9, no. 11, pp. 39—52. http://doi.org/10.22215/timreview/12
  19. Williams L.R., Grealy M.A., Kelly S.W., Henderson I., Butler S.H. Perceptual bias, more than age, impacts on eye movements during face processing. Acta Psychologica, 2016. Vol. 164, pp. 127—135. DOI:10.1016/j. actpsy.2015.12.012

Information About the Authors

Vladimir A. Barabanschikov, Doctor of Psychology, Professor, Director, Institute of Experimental Psychology, Moscow State University of Psychology and Education, Dean of the Faculty of Psychology, Moscow Institute of Psychoanalysis, Moscow, Russia, ORCID: https://orcid.org/0000-0002-5084-0513, e-mail: vladimir.barabanschikov@gmail.com

Mariya M. Marinova, Assistant Director of the Institute of Experimental Psychology, Academic Services Specialist, Moscow State University of Psychology and Education, Postgraduate Student, Moscow Institute of Psychoanalysis, Moscow, Russia, ORCID: https://orcid.org/0000-0002-8862-4007, e-mail: marinovamm@yandex.ru

Metrics

Views

Total: 1334
Previous month: 23
Current month: 34

Downloads

Total: 981
Previous month: 26
Current month: 39