Deepfake in Face Perception Research

Presents the state-of-the-art Deepfake face replacement image collage method, an artificial intelligence (AI) product that can be used to create high-quality, realistic videos with a fake or replaced face, with no obvious signs of manipulation. Based on the DeepFaceLab (DFL) application, the process of creating video images of an “impossible face” is described step by step. The results of the experiments of studying the perception patterns of the moving “impossible face” and their differences in statics and dynamics are presented. The stimuli were two DFL-generated models of virtual sitters with impossible faces: a video image of a chimerical face, in which the right and left sides belong to different people, and a Tatchered face with the eyes and mouth areas rotated by 180°. It was shown that the phenomena of perception of the “impossible face”, registered earlier under static conditions (integrity of perception of the split image, distraction and inversion effect), are preserved and acquire a new content when dynamic models are exposed. In contrast to the collaged images, the original faces in statics and motion, regardless of egocentric orientation, are evaluated positively at the level of high values. Under all tested conditions the gender of the virtual sitter is determined adequately, the perceived age is overestimated. Estimates of the virtual sitter’s emotions from his video images are differentiated into basic (stable) and additional (changing) states, the ratio of which depends on the content of a particular episode. Deepfake image synthesis technology significantly expands the possibilities of psychological research of interpersonal perception. The use of digital technologies simplifies the creation of “impossible face” stimulus models necessary for in-depth study of representations of the human inner world, and creates a need for new experimental-psychological procedures corresponding to a higher level of ecological and social validity.

1. Barabanshchikov V.A. Ekspressii litsa i ikh vospriyatie [Facial expressions and their perception]. Moscow: IP RAN Publ., 2012. (In Russ.).
2. Barabanschikov V.A. (otv. red.) Kognitivnye mekhanizmy neverbal’noi kommunikatsii. M.: Kogito- Tsentr, 2017. (In Russ.).
3. Barabanshchikov V.A., Korol’kova O.A. Vospriyatie ekspressii «zhivogo» litsa [Perception of “live” facial expressions]. Eksperimental’naya psikhologiya = Experimental Psychology, 2020, Vol. 8, no. 3. pp. 55— 73. DOI:10.17759/exppsy.2020130305 (In Russ.).
4. Barabanshchikov V. A., Marinova M. M. Vosprinimaemye kachestva i otsenka ekspressii podvizhnogo khimericheskogo litsa [Perceived qualities and evaluation of expressions of the mobile chimeric face]. Litso cheloveka v kontekstakh prirody, tekhnologii i kul’tury [The Face of Man in Contexts of Nature, Technology, and Culture] K.I. Anan’eva, V.A. Barabanshchikov, A. A. Demidov (eds.). Moscow: Moskovskii institut psikhoanaliza, Kogito-Tsentr, 2020. (In Press). (In Russ.).
5. Barabanshchikov V.A., Marinova M.M. Vospriyatie videoizobrazhenii khimericheskogo litsa [Perception of video images of a chimerical face]. Poznanie i perezhivanie = Cognition and Experience, 2020, Vol. 1, no. 1, pp. 112—134. doi:10.51217/cogexp_2020_01_01_07 (In Russ).
6. Barabanshchikov V.A., Marinova M.M., Abramov A.D. The Virtual Personality of the Moving Thatcherized Person. Psikhologicheskaya nauka i obrazovanie = Psychological Science and Education. Vol. 23, no. 1, pp. 5—18. DOI:10.17759/pse.2021000001 (In Russ.).
7. Barabanshchikov V.A., Marinova M.M., Abramov A.D. Virtual’naya lichnost’ tetcherizirovannogo litsa v statike i dinamike [Virtual Personality of the Thatcherized Person in Statics and Dynamics] // Eksperimental’naya psikhologiya v sotsial’nykh praktikakh [Experimental Psychology in Social Practices] V.A. Barabanshchikov, V.V. Selivanov (eds). Moscow: Universum, 2020. pp. 7—16. (In Russ.).
8. Barabanshchikov V.A., Marinova M.M., Abramov A.D. Illyuziya Margaret Tetcher na podvizhnom litse [The illusion of Margaret Thatcher on a moving face]. Sbornik materialov ezhegodnoi konferentsii po kognitivnoi nauke, posvyashchennoi pamyati Dzh. S. Brunera, «Psikhologiya poznaniya: nizkourovnevye i vysokourovnevye protsessy» (g. Yaroslav’, 18—19 dekabrya 2020 g.) [Proceedings of the annual conference on cognitive science in memory of J. S. Bruner “Psychology of cognition: low-level and high-level processes”]. Yaroslavl: Yaroslavl State University. P.G. Demidov, 2021. (In press).
9. Anna. DeepFaceLab: instrument dlya generatsii dipfeikov [Electronic resource] [DeepFaceLab: deepfake generation tool]. Neurohive. URL: https://neurohive.io/ru/frameworki/deepfacelab-instrument-dlya-generacii-dipfejkov/ (Accessed: 30.11.2020) (In Russ).
10. Chawla R. Deepfakes: How a pervert shook the world. International Journal of Advance Research and Development, 2019. Vol. 4, no. 6, pp. 4—8.
11. Demuthova S., Démuth A. Handedness and the Preference of the Visual Field in Face Perception // European Scientific Journal. 2018. http://doi.org/10.19044/esj.2018.c3p8
12. Dole M., Méary D., Pascalis O. Modifications of Visual Field Asymmetries for Face Categorization in Early Deaf Adults: A Study With Chimeric Faces. Front. Psychol. 2017.Vol. 8, P. 30. http://doi. org/10.3389/fpsyg.2017.00030
13. Maras M., Alexandrou A. Determining Authenticity of Video Evidence in the Age of Artificial Intelligence and in the Wake of Deepfake Videos. International Journal of Evidence and Proof. 2018, Vol. 23, pp. 255—262. DOI:10.1177/1365712718807226
14. Perov I., Gao D., Chervoniy N., Liu K., Marangonda S., Ume C., Mr. Dpfks., Facenheim C. F., RP L., Jiang J., Zhang S., Wu P., Bo zhou, Zhang W. DeepFaceLab: A simple, flexible and extensible face swapping framework [Electronic resource]. 2020. ArXiv: Computer Vision and Pattern Recognition. URL: https:// arxiv.org/pdf/2005.05535.pdf (Accessed: 13.10.2020).
15. Stephanie S.A.H. Blom, Henk Aarts & Gün R. Semin. Lateralization of facial emotion processing and facial mimicry. Laterality, 2020. Vol. 25, no. 3, pp. 259—274. DOI:10.1080/1357650X.2019.1657127
16. Thompson P. Margaret Thatcher: A new illusion. Perception, 1980. V. 9, P. 483—484. DOI:10.1068/ p090483
17. Weibert K., Müller V., Sänger J. Familiarity abolishes right-hemispheric bias in face perception. Journal of Vision, 2017. vol. 17. DOI: 10.1167/17.10.998.
18. Westerlund M. The Emergence of Deepfake Technology: A Review // Technology Innovation Management Review. 2019.Vol. 9, no. 11, pp. 39—52. http://doi.org/10.22215/timreview/12
19. Williams L.R., Grealy M.A., Kelly S.W., Henderson I., Butler S.H. Perceptual bias, more than age, impacts on eye movements during face processing. Acta Psychologica, 2016. Vol. 164, pp. 127—135. DOI:10.1016/j. actpsy.2015.12.012