Experimental Psychology (Russia)
2019. Vol. 12, no. 1, 98–118
doi:10.17759/exppsy.2019120108
ISSN: 2072-7593 / 2311-7036 (online)
Follow the hands of the interlocutor! (on strategies for the distribution of visual attention)
Abstract
General Information
Keywords: multichannel linguistics, natural interaction, visual attention, gesticulation
Journal rubric: Psycholinguistics
Article type: scientific article
DOI: https://doi.org/10.17759/exppsy.2019120108
Funding. This work was supported by the Russian Sience Foundation (project № 14-18-03819 “Language as is: Russian multimodal discourse”).
For citation: Fedorova O.V., Zherdev I.Y. Follow the hands of the interlocutor! (on strategies for the distribution of visual attention). Eksperimental'naâ psihologiâ = Experimental Psychology (Russia), 2019. Vol. 12, no. 1, pp. 98–118. DOI: 10.17759/exppsy.2019120108. (In Russ., аbstr. in Engl.)
References
- Barabanshchikov V.A. Vospriyatie vyrazhenii litsa. M.: Izd-vo IP RAN, 2009.
- Barabanshchikov V.A., Zhegallo A.V. Raspoznavanie ekspressii litsa v blizhnei periferii zritel’nogo polya. Eksperimental’naya psikhologiya, 2013. T. 6. № 2. P. 58—83.
- Kibrik A.A. Mul’timodal’naya lingvistika. Kognitivnye issledovaniya, 2010. vyp. IV. Moscow. P. 134—152.
- Kibrik A.A. Prosodicheskie portrety govoryashchikh. Kibrik A. A., Podlesskaya V. I. (red.). Rasskazy o snovideniyakh: korpusnoe issledovanie ustnogo russkogo diskursa. Moscow: YaSK, 2009. P. 478—487.
- Kibrik A.A. Russkiy mul’tikanal’nyy diskurs. Chast’ I. Postanovka problemy // Psikhologicheskiy zhurnal, 2018a. Vol. 39 (1), P. 70—80.
- Kibrik A.A. Russkiy mul’tikanal’nyy diskurs. Chast’ II. Razrabotka korpusa i napravleniya issledovaniy // Psikhologicheskiy zhurnal, 2018b. Vol. 39 (2), P. 78—89.
- Kibrik A.A., Podlesskaya V. I. (Eds.). Rasskazy o snovideniyakh: korpusnoe issledovanie ustnogo russkogo diskursa. Moscow: YaSK, 2009.
- Kodzasov S.V. Issledovaniya v oblasti russkoi prosodii. Moscow: YaSK. 2009.
- Litvinenko A.O., Nikolaeva Yu.V., Kibrik A.A. Annotirovanie russkikh manual’nykh zhestov: teoreticheskie i prakticheskie voprosy. Komp’yuternaya lingvistika i intellektual’nye tekhnologii: Po materialam ezhegodnoi Mezhdunarodnoi konferentsii «Dialog 2017», Moscow, 2017. P. 271—286.
- Fedorova O.V., Kibrik A.A., Korotaev N.A., Litvinenko A.O., Nikolaeva Yu.V. Vremennaya koordinatsiya mezhdu zhestovymi i rechevymi edinitsami v mul’timodal’noi kommunikatsii. Komp’yuternaya lingvistika i intellektual’nye tekhnologii: Po materialam ezhegodnoi Mezhdunarodnoi konferentsii Dialog (Moskva, 1—4 iyunya 2016). Vyp. 15 (22). Moscow: Izd-vo RGGU, 2016. P. 159—170.
- Fedorova O.V. Raspredelenie zritel’nogo vnimaniya sobesednikov v estestvennoi kommunikatsii: 50 let spustya. E.V. Pechenkova, M.V. Falikman (red.) Kognitivnaya nauka v Moskve: novye issledovaniya. Materialy konferentsii 15 iyunya 2017. Moscow: BukiVedi, IPPiP, 2017. P. 370—375.
- Yazykov S.A., Fedorova O.V., Filimonova I.V., Potanina Yu.D., Pogodaev I.S. Issledovanie napravleniya vzora v dialoge na materiale mul’timodal’nogo korpusa «Rasskazy i razgovory o grushakh». Sed’maya mezhdunarodnaya konferentsiya po kognitivnoi nauke: Tezisy dokladov. Svetlogorsk, 20—24 iyunya 2016. Moscow: Institut psikhologii RAN, 2016. P. 682—683.
- Abuczki A., Esfandiari B.G. An overview of multimodal corpora, annotation tools and schemes // Argumentum, 2013. Vol. 9. P. 86—98.
- Beattie G., Webster K., Ross J. The fixation and processing of the iconic gestures that accompany talk // Journal of Language and Social Psychology. 2010. Vol. 29. P. 194—213.
- Brône G., Oben B. InSight Interaction. A multimodal and multifocal dialogue corpus // Language Resources and Evaluation. 2015. Vol. 49(1). P. 195—214.
- Carletta J. Announcing the AMI Meeting Corpus // The ELRA Newsletter. 2003. Vol. 11 (1), January- March. P. 3—5.
- Chafe W. (Ed.). The pear stories: Cognitive, cultural, and linguistic aspects of narrative production. Norwood, 1980.
- Engbert R. Microsaccades: A microcosm for research on oculomotor control, attention, and visual perception // Progress in Brain Research, 2006. Vol. 154. P. 177—192.
- Foster M.E., Oberlander J.Corpus-based generation of head and eyebrow motion for an embodied conversational agent // Language Resources and Evaluation, 2007. Vol. 41 (3/4). P. 305—323.
- Gullberg M. Gestures, referents, and anaphoric linkage in learner varieties // C. Dimroth, M. Starren (Eds.) Information Structure, Linguistic Structure and the Dynamics of Language Acquisition. Amsterdam: John Benjamins, 2003. P. 311—328.
- Gullberg M., Holmqvist K. Keeping an eye on gestures: Visual perception of gestures in face-to-face communication // Pragmatics and Cognition, 1999. Vol. 7. P. 35—63.
- Gullberg M., Holmqvist K. Visual attention towards gestures in face-to-face interaction vs. on screen // I. Wachsmuth and T. Sowa (Eds.) Gesture and Sign Language in Human—Computer Interaction. Berlin Heidelberg: Springer, 2002. P. 206—214.
- Gullberg M., Holmqvist K. What speakers do and what addressees look at: Visual attention to gestures in human interaction live and on video // Pragmatics & Cognition, 2006. Vol. 14 (1), P. 53—82.
- Gullberg M., Kita S. Attention to speech-accompanying gestures: Eye movements and information uptake // Journal of Nonverbal Behavior, 2009. Vol. 33. P. 251—277.
- Holler J., Kendrick K.H.Unaddressed participants’ gaze in multi-person interaction: Optimizing recipiency // Frontiers in Psychology, 2015. Vol. 6 (98).
- Hostetter A.B. When do gestures communicate? A meta-analysis, Psychological Bulletin, 2001. Vol. 137 (2). P. 297—315.
- Jacob R., Karn K. Eye tracking in human-computer interaction and usability research: Ready to deliver the promises // R. Radach, J. Hyona, and H. Deubel (Eds.). The mind’seye: Cognitive and applied aspects of eye movement research. Oxford: Elsevier Science, 2003.
- Jokinen K. Turn taking, Utterance Density, and Gaze Patterns as Cues to Conversational Activity // Proceedings of The International Conference on Multimodal Interaction, Alicante, Spain, 2011.
- Kendon A. Some functions of gaze direction in social interaction // Acta Psychologica, 1967. Vol. 26. P. 22—63.
- Kendon A. Gesture. Visible action as utterance. Cambridge, 2004.
- Kita S. How representational gestures help speaking // D. McNeill (ed.) Language and gesture, Cambridge University Press; Cambridge, UK, 2000. P. 162—185.
- Kita S., Ozyurek A. What does cross-linguistic variation in semantic coordination of speech and gesture reveal?: evidence for an interface representation of spatial thinking and speaking // Journal of Memory and Language, 2003. Vol. 48 (1). P. 16—32.
- Knight D. Multimodality and active listenership: A corpus approach, Bloomsbury, London, 2011.
- Krauss R.M., Chen Y., Gottesman R. Lexical gestures and lexical access: A process model // D. McNeill D (ed.) Language and gesture, Cambridge University Press, Cambridge, UK, 2000. P. 261—283.
- McNeill D. Hand and Mind: What Gestures Reveal about Thought. The University of Chicago Press, Chicago, 1992.
- McNeill D. Gesture and thought, Chicago, 2005.
- Mol L., Krahmer E., Maes A., Swerts M. Seeing and being seen: The effects on gesture production // Journal of Computer-Mediated Communication, 2011. Vol. 17. P. 77—100.
- Mondada L.Challenges of multimodality: Language and the body in social interaction // Journal of sociolinguistics, 2016. Vol. 20& P. 336—366.
- Müller C., Fricke E., Cienki A., McNeill D. (Eds.). Body — Language — Communication, Mouton de Gruyter, Berlin, 2014.
- Scollon R. Multimodality and the language of politics // K. Brown (ed.) Encyclopedia of language and linguistics, Elsevier, 2006. Vol. 9. P. 386—387.
- Smith D.T., Schenk T.The premotor theory of attention: time to move on? // Neuropsychologia, 2012. Vol. 50, P. 1104—1114.
- Taylor M. The Structure of Multimodal Dialogue. Amsterdam: Elsevier, 1989.
- Vertegaal R., Slagter R., Van der Veer G., Nijholt A. Eye gaze patterns in conversations: There is more to conversational agents than meets the eyes // Proceedings of the Conference on Human Factors in Computing Systems, 2001.
Information About the Authors
Metrics
Views
Total: 2456
Previous month: 23
Current month: 1
Downloads
Total: 1105
Previous month: 5
Current month: 1