Review of Studying Social Interactions With the Usage of Eye-Tracking Analysis

This article provides an overview of empirical studies focused on the possibilities of using the eye-tracking analysis to study social interactions. The whole text is divided into conceptual blocks, such as eye-tracking as a method of psychological research and its possibilities of studying social interactions. A brief review on the present topic revealed that one of the most relevant objectives in this research area is in line with the studies of the features of multimodal communication carried out simultaneously through several “channels” of information transmission (including visual feedback) with its correlation with the collaborative productivity and organizing of effective forms of joint acitivity. The observed data improve scientific understanding of the mechanisms by which different modes of interaction (joint attanetion, mutual understanding, reflection, communication) arise and are implemented by the main participants of joint activity. It was deliberately stressed that this line of research is needed to be supplemented by the studies on dynamics of oculomotor activity at different stages of joint actions and interactions.

1. Barabanshchikov V.A. (ed.), Aitreking v psikhologicheskoi nauke i praktike [Eye-tracking in psychological science and practice]. Moscow: “Kogito-Tsentr”, 2015. 410 p. (In Russ.).
2. Alekseeva A.S., Lomtatidze O.V., Bulatova E.V. Ispol'zovanie aitrekinga pri izuchenii vizual'nogo vospriyatiya sotsial'noi reklamy studentami vuza [The use of eye-tracking in the study of visual perception of social advertising by university students]. In Barabanshchikov V.A. (ed.), Aitreking v psikhologicheskoi nauke i praktike [Eye tracking in psychological science and practice]. Moscow: Kogito-Tsentr, 2015, pp. 175–181. (In Russ.).
3. Barabanshchikov V.A., Zhegallo A.V. Aitreking: Metody registratsii dvizhenii glaz v psikhologicheskikh issledovaniyakh i praktike [Methods for recording eye movements in psychological research and practice]. Moscow: Kogito-Tsentr, 2014. 128 p. (In Russ.).
4. Vygotskii L.S. Sobranie sochineniĭ: V 6 t. T. 5. Osnovy defektologii [Collected Works: in 6 vol. Vol. 5. Fundamentals of defectology]. Vlasovoi T.A. (ed.). Moscow: Pedagogika, 1983. 368 p. (In Russ.).
5. Zhegallo A.V. Spetsifika soderzhaniya zritel'nykh fiksatsiĭ pri opoznanii emotsional'nykh ekspressii po vyrazheniyu litsa [The specifics of the content of visual fixations in the recognition of emotional expressions by facial expressions]. In Barabanshchikov V.A. (ed.), Aitreking v psikhologicheskoi nauke i praktike [Eye tracking in psychological science and practice]. Moscow: Kogito-Tsentr, 2015, pp. 197–209. (In Russ.).
6. Konokotin A.V. Vklyuchenie detei s osobymi obrazovatel'nymi potrebnostyami i normativno razvivayushchikhsya detei v sovmestnoe reshenie uchebnykh zadach (na primere resheniya zadach na ponimanie mul'tiplikativnykh otnoshenii) [Inclusion of children with special educational needs and normatively developing children in the joint solution of educational problems (on the example of solving problems for understanding multiplicative relations)]. Kul'turno-istoricheskaya psikhologiya = Cultural-historical psychology, 2019. Vol. 15, no. 4, pp. 79–88. DOI:10.17759/chp.2019150408 (In Russ.).
7. Kuznetsov Yu.F. Deyatel'nost' kak psikhologo-pedagogicheskaya kategoriya [Activity as a psychological and pedagogical category]. Spetsial'noe obrazovanie [Special education], 2005, no. 5, pp. 37–44. (In Russ.).
8. Makashova A.V., Mudruk A.A., Smirnova Ya.K. Aitreker issledovanie defitsita sovmestnogo vnimaniya [Eyetracker joint attention deficit study]. Trudy molodykh uchenykh Altaiskogo gosudarstvennogo universiteta [Works of young scientists of the Altai State Institute], 2020, no. 17, pp. 174–177. (In Russ.).
9. Perre-Klermon A.-N. Rol' sotsial'nykh vzaimodeistvii v razvitii intellekta detei [The role of social interactions in the development of children's intelligence]. Moscow: Pedagogika, 1991. 248 p. (In Russ.).
10. Rubtsov V.V., Isaev E.I., Konokotin A.V. Uchebnaya deyatel'nost' kak zona blizhaishego razvitiya refleksivnykh i kommunikativnykh sposobnostei detei 6–10 let [Educational activity as a zone of proximal development of reflexive and communicative abilities of children aged 6–10]. Kul'turno-istoricheskaya psikhologiya = Cultural-historical psychology, 2022. Vol. 18, no. 1, pp. 28–40. DOI:10.17759/chp.2022180103 (In Russ.).
11. Rubtsov V.V. Sotsial'nye vzaimodeistviya i obuchenie: kul'turno-istoricheskii kontekst [Social Interactions and Learning: Cultural and Historical Context]. Сul'turno-istoricheskaya psikhologiya = Cultural-historical psychology, 2005. Vol. 1, no. 1, pp. 14–24. (In Russ.).
12. Ageev N.Ya. et al. Svyaz' tsifrovykh tekhnologii s razvitiem kognitivnykh i kommunikativnykh protsessov podrostkov i yunoshei: obzor empiricheskikh issledovanii [Elektronnyi resurs] [The connection of digital technologies with the development of cognitive and communicative processes in adolescents and young men: a review of empirical research]. Psikhologo-pedagogicheskie issledovaniya = Psychological and pedagogical research, 2023. Vol. 15, no. 1, pp. 37–55. DOI:10.17759/psyedu.2023150103 (In Russ.).
13. Farakhutdinov Sh.F., Panova A.V. Aitreking v marketingovykh i sotsiologicheskikh issledovaniyakh [Elektronnyi resurs] [Eye tracking in marketing and sociological research]. Sotsialogiya [Sociology], 2019, no. 5, pp. 189–198. Available at: https://cyberleninka.ru/article/n/aytreking-v-marketingovyh-i-sotsiologicheskih-issledovaniyah (Accessed 14.05.2023) (In Russ.).
14. Kharitonov A.N., Anan'eva K.I. Sovmestnaya identifikatsiya lits raznykh ras: soglasovanie poznavatel'nykh protsessov [Co-identification of faces of different races: coordination of cognitive processes]. In Barabanshchikov V.A., Nosulenko V.N., Samoilenko E.S. (eds.), Poznanie v deyatel'nosti i obshchenii: ot teorii i praktiki k eksperimentu [Cognition in activity and communication: from theory and practice to experiment]. Moscow: Institut psikhologii RAN, 2011, pp. 17– (In Russ.).
15. Shevel' T.M., Falikman M.V. «Podskazka vzglyadom» kak klyuch k mekhanizmam sovmestnogo vnimaniya: osnovnye rezul'taty issledovanii [Gaze Cueing as a Key to Joint Attention Mechanisms: Essential Research Findings]. Kul'turno-istoricheskaya psikhologiya = Cultural-historical psychology, 2022. Vol. 18, no. 1, pp. 6–16. DOI:10.17759/chp.2022180101 (In Russ.).
16. Yarbus A.L. Rol' dvizhenii glaz v protsesse zreniya [The role of eye movements in the process of vision]. Nyuberg N.D. (ed.). Moscow: Nauka [The science], 1965. 166 p. (In Russ.).
17. Amatuni A. et al. In-the-Moment Visual Information from the Infant's Egocentric View Determines the Success of Infant Word Learning: A Computational Study. Proceedings of the Annual Meeting of the Cognitive Science Society, 2021. Vol. 43, pp. 265–271. URL: https://escholarship.org/uc/item/9pj8b2cg (Accessed 14.05.2023)
18. Brennan S.E. et al. Coordinating cognition: The costs and benefits of shared gaze during collaborative search. Cognition, 2008. Vol. 106, no. 3, pp. 1465–1477.
19. Carletta J. et al. Eyetracking for two-person tasks with manipulation of a virtual world. Behavior Research Methods, 2010. Vol. 42, no. 1, pp. 254–265. DOI:10.1163/15685680432277825
20. Cowan D.G., Vanman E.J., Nielsen M. Motivated empathy: the mechanics of the empathic gaze. Cognition and Emotion, 2014. Vol. 28, no. 8, pp. 1522–1530.
21. de Belen R.A., Pincham H., Hodge A. et al. Eye-tracking correlates of response to joint attention in preschool children with autism spectrum disorder. BMC Psychiatry, 2023. Vol. 23. DOI:10.1186/s12888-023-04585-3
22. Eisenbarth H., Alpers G.W. Happy mouth and sad eyes: scanning emotional facial expressions. Emotion, 2011. Vol. 11, no. 4, pp. 860–865.
23. Hanna J.E., Brennan S. Speakers eye gaze disambiguates referring expressions early during face-to-face conversation. Journal of Memory and Language, 2007. Vol. 57, no. 4, pp. 596–615.
24. Herrera F. et al. Building long-term empathy: A large-scale comparison of traditional and virtual reality perspective-taking. PLoS ON, Vol. 13, no. 10. 37 p. DOI:10.1371/journal.pone.0204494
25. Hsiao J., Cottrell G. The influence of number of eye fixations on face recognition. Journal of vision, 2007. Vol. 7, no. 9. DOI:10/1167/7.9.494
26. Jermann P., Nüssli M.-A., Li W. Using dual eye-tracking to unveil coordination and expertise in collaborative Tetris. Proceedings of HCI "Human Computer Interaction" (Dundee, UK, September 6-10, 2010). Dundee: BCS Learning and Development Ltd, 2010, pp. 36–44. DOI:10.14236/ewic/HCI2010.7
27. Jing A. et al. Eye See What You See: Exploring How Bi-Directional Augmented Reality Gaze Visualisation Influences Co-Located Symmetric Collaboration. Virtual Real, 2021. Vol. 2. 17 p. DOI:10.3389/frvir.2021.697367
28. Just M.A., Carpenter P.A. A Theory of Reading: From Eye Fixations to Comprehension. Psychological Review, 1980. Vol. 87, no. 4, pp. 329–354.
29. Massé B., Ba S., Horaud R. Tracking Gaze and Visual Focus of Attention of People Involved in Social Interaction. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2018. Vol. 40, no. 11, pp. 2711–2724. DOI:10.1109/TPAMI.2017.2782819
30. Müller R. et al. Gaze transfer in remote cooperation: Is it always helpful to see what your partner is attending to? Quarterly Journal of Experimental Psychology, Vol. 66, no. 7, pp. 1302–1316. DOI:10.1080/17470218.2012.737813Pfeiffer U.J., Vogeley K., Schilbach L. From gaze cueing to dual eye-tracking: Novel approaches to investigate the neural correlates of gaze in social interaction. Neuroscience and Biobehavioral Reviews, 2013. Vol. 37, no. 10(2), pp. 2516–2528. DOI:10.1016/j.neubiorev.2013.07.017
31. Pfeiffer U.J., Vogeley K., Schilbach L. From gaze cueing to dual eye-tracking: Novel approaches to investigate the neural correlates of gaze in social interaction. Neuroscience and Biobehavioral Reviews, 2013. Vol. 37, no. 10(2), pp. 2516–2528. DOI:10.1016/j.neubiorev.2013.07.017
32. Rozhkova G.I., Ogninov V.V. Face recognition and eye movements: landing on the nose is not always necessary. ECVP Abstract Supplement, 2009. Vol. 38, no. 1, pp. 77. DOI:10.1177/03010066090380S101
33. Rubtsov V.V., Konokotin A.V. Formation of higher mental functions in children with special educational needs via social interaction. In. Darlyne G. Nemeth, Glozman J. (eds.). Evaluation and Treatment of Neuropsychologically Compromised Children. San Diego: Elsilver, 2020, pp. 179–195.
34. Schindler M., Lilienthal A.J. Domain-specific interpretation of eye tracking data: towards a refined use of the eye-mind hypothesis for the field of geometry. Educational Studies on Mathematics, 2019. Vol. 101, pp. 123–139. DOI:10.1007/s10649-019-9878-z
35. Schroer S.E., Chen Y. Looking is not enough: Multimodal attention supports the real-time learning of new words. Developmental Science, 2022. Vol. 26, no. 2. DOI:10.1111/desc.13290
36. Shvarts A. Joint Attention in Resolving the Ambiguity of Different Presentations: A Dual Eye-Tracking Study of the Teaching-Learning Process. In Presmeg N., Radford L., Roth W.-M., Kadunz G. (eds.). Signs of Signification. Cham: Springer International Publishing, 2018, pp. 73–102. DOI:10.1007/978-3-319-70287-2_5
37. Sumer O., Gerjets P., Trautwein U., Kasneci E. Attention Flow: End-to-End Joint Attention Estimation. Proceedings of "IEEE Winter Conference on Applications of Computer Vision" (Snowmass Village, USA, March 1-5, 2020). Snowmass Village: IEEE, 2020, pp. 3316–3325. DOI:10.1109/WACV45572.2020.9093515
38. Vaidya A.R., Jin C., Fellows L.K. Eye spy: The predictive value of fixation patterns in detecting subtle and extreme emotions from faces. Cognition, 2014. Vol. 133, no. 2, pp. 443–456. DOI:10.1016/j.cognition.2014.07.004
39. Velichkovsky B.M. Communicating attention: Gaze position transfer in cooperative problem solving. Pragmatics and Cognition, 1995. Vol. 3, no. 2, pp. 199–222.
40. Vernetti A. et al. Simulating interaction: Using gaze-contingent eye-tracking to measure the reward value of social signals in toddlers with and without autism. Developmental Cognitive Neuroscience, 2018. Vol. 29, pp. 21–29. DOI:10.1016/j.dcn.2017.08.004