Accuracy of the Formation of Spatial Representations of Dynamic Scenes in Working Memory

The cognitive representation of the environment is formed using cognitive systems that process data on spatial representations of two types: egocentric, encoding the position of environmental objects relative to the observer, and allocentric, encoding the position of objects relative to each other, regardless of the position of the observer. Data on spatial representations were studied mainly in problems of memorization and reconstruction of static scenes. However, the task of processing information about dynamic scenes in everyday life has a higher ecological validity. We used HMD virtual reality technologies to study the accuracy of the formation of egocentric and allocentric spatial representations of static and dynamic scenes in working memory. The subjects were presented 8 three-dimensional virtual scenes of 4 objects each for 10 seconds in static and dynamic conditions for memorization and reconstruction. Identification accuracy (number of correctly reconstructed objects) and localization accuracy (accuracy of spatial scene reconstruction) were assessed. Localization accuracy was assessed in topological units, corresponding to the accuracy of the representation of the general configuration of objects in the scene (global topological information), and in metric units, corresponding to the accuracy of the representation of the spatial coordinates of each object (local metric information). The results showed that object identification accuracy was similar in static and dynamic conditions; the processes of encoding metric local information during the formation of both types of representations of dynamic scenes worsen compared to static ones; the accuracy of encoding topological global information remains stable compared to the static condition. We can conclude that the visual and spatial systems operate independently as part of a general cognitive system that processes data on spatial representations in time-limited working memory, as well as the redistribution of its resource in dynamic condition for supporting topological data of the holistic configuration of moving objects more, than metric data. The results highlight the importance of topological spatial characteristics of spatial representations for processes of early spatial perception, decision making, and action in the environment.

1. Velichkovskij B.B. Rabochaya pamyat' cheloveka: Struktura i mekhanizmy. M.: Kogitocentr, 2015. 247 p. (In Russ.).
2. Gusev A.N. Psihofizika sensornyh zadach: Sistemno-deyatel'nostnyj analiz povedeniya cheloveka v situacii neopredelennosti / A.N. Gusev. M.: Izd-voMosk. un-ta; UMK «Psihologiya», 2004. 316 p. (In Russ.).
3. Men'shikova G.YA., Velichkovskij B.B., Bugrij G.S., Savel'eva O.A. Tochnost' formirovaniya prostranstvennyh reprezentacij v rabochej pamyati. Kognitivnaya nauka v Moskve: novye issledovaniya. M.: OOO «BukiVedi», IPPiP, 2021. Pp. 285—290. (In Russ.).
4. Men'shikova G.YA., Savel'eva O.A., Kovyazina M.S. Ocenka uspeshnosti vosproizvedeniya egocentricheskih i allocentricheskih prostranstvennyh reprezentacij pri ispol'zovanii sistem virtual'noj real'nosti. Nacional'nyj psihologicheskij zhurnal, 2018. 2(30), pp. 113—122. (In Russ.).
5. Men'shikova G.YA., Savel'eva O.A., Velichkovskij B.B., Bugrij G.S. Formirovanie egocentricheskih i allocentricheskih prostranstvennyh reprezentacij v rabochej pamyati. Voprosy psihologii, 2020. 6, pp. 131—140. (In Russ.).
6. Minskij M. Frejmy dlya predstavleniya znanij. Energiya, 1979. (In Russ.).
7. Najser U. Poznanie i real'nost'. "Progress", 1981. (In Russ.).
8. Savel'eva O.A., Men'shikova G.YA. Ocenka uspeshnosti formirovaniya prostranstvennyh predstavlenij pri ispol'zovanii sistem virtual'noj real'nosti. Tomsk: Tomskij gosudarstvennyj universitet, 2017. Pp. 190. (In Russ.).
9. Utochkin I.S., YUrevich M.A., Bulatova M.E. Zritel'naya rabochaya pamyat': metody, issledovaniya, teorii. Rossijskij zhurnal kognitivnoj nauki, 2016. Vol. 3, 3, pp. 58—76. (In Russ.).
10. Baddeley A. The episodic buffer: a new component of working memory? Trends in cognitive sciences, 2000. Vol. 4, 11, pp. 417—423.
11. Baddeley A. Working memory: theories, models, and controversies. Annual review of psychology, 2012. Vol. 63, pp. 1—29. DOI:1146/annurev-psych-120710-100422
12. Baddeley A.D. Working memory. New York: Oxford University Press, 1986.
13. Bartlett F.C.Remembering: A study in experimental and social psychology. Cambridge, UK Cambridge University Press, 1932.
14. Blalock L.D., Clegg B.A. Encoding and representation of simultaneous and sequential arrays in visuospatial working memory. Quarterly Journal of Experimental Psychology, Vol. 63, no. 5, pp. 856—862.
15. Borst G., Ganis G., Thompson W.L., &Kosslyn S.M. Representations in mental imagery and working memory: Evidence from different types of visual masks. Memory & cognition, Vol. 40, pp. 204—217.
16. Clark B.J., Simmons C.M., Berkowitz L.E., Wilber A.A. The retrosplenial-parietal network and reference frame coordination for spatial navigation. Behavioral Neuroscience, Vol. 132(5), pp. 416—429.
17. Cocchi L., et al. Visuo-spatial processing in a dynamic and a static working memory paradigm in schizophrenia. Psychiatry research, 2007. Vol. 152, no. 2-3, pp. 129—142.
18. Colombo D., et al. Egocentric and allocentric spatial reference frames in aging: A systematic review. Neuroscience & Biobehavioral Reviews, 2017. Vol. 80, pp. 605—621.
19. Coluccia E. Learning from maps: the role of visuo‐spatial working memory. Applied Cognitive Psychology: The Official Journal of the Society for Applied Research in Memory and Cognition, 2008. Vol. 22, no. 2, pp. 217—233.
20. Coluccia E., Martello A. Ilruolodellamemoria di lavoro visuo-spaziale nell'orientamento geografico: uno studio correlazionale. Giornaleitaliano di psicologia, 2004. Vol. 31, no. 3, pp. 523—552.
21. Cowan N. The magical mystery four: How is working memory capacity limited, and why? Current directions in psychological science, 2010. Vol. 19, no. 1, pp. 51—57.
22. Cowan N. The magical number 4 in short-term memory: A reconsideration of mental storage capacity. Behavioral and Brain Sciences, 2001. Vol. 24(1), pp. 87—185. DOI:1017/S0140525X01003922
23. Ekstrom A.D., Isham E.A. Human spatial navigation: Representations across dimensions and scales. Current opinion in behavioral sciences, Vol. 17, pp. 84—89.
24. Endress A.D., Wood J.N. From movements to actions: Two mechanisms for learning action sequences. Cognitive psychology, Vol. 63, no. 3, pp. 141—171.
25. Engle R.W. Working memory capacity as executive attention. Current directions in psychological science, 2002. Vol. 11, no. 1, pp. 19—23.
26. Ganis G., Schendan H.E. Visual imagery. Wiley Interdisciplinary Reviews: Cognitive Science, 2011. Vol. 2, no. 3, pp. 239—252.
27. Gupta S., Davidson J., Levine S., Sukthankar R., Malik J. Cognitive Mapping and Planning for Visual Navigation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2017. Pp. 2616—2625.
28. Hollingworth A. Object-position binding in visual memory for natural scenes and object arrays. Journal of Experimental Psychology: Human Perception and Performance, 2007. Vol. 33, no. 1, pp. 31.
29. Jiang Y., Chun M.M., Olson I.R. Perceptual grouping in change detection. Perception & Psychophysics, Vol. 66, no. 3, pp. 446—453.
30. Jiang Y., Olson I.R., and Chun M.M. Organization of visual short-term memory.  Exp. Psychol. Learn. Mem. Cogn., 2000. Vol. 26(3), pp. 683—702. DOI:10.1037/0278-7393.26.3.683
31. Kahneman D. Attention and effort. Englewood Cliffs, NJ: Prentice-Hall, 1973. Vol. 1063, pp. 218—226.
32. Klatzky R.L. Allocentric and egocentric spatial representations: Definitions, distinctions, and interconnections. Spatial cognition. Springer, Berlin, Heidelberg, 1998. Pp. 1—17.
33. Kosslyn S.M. Seeing and imagining in the cerebral hemispheres: a computational approach. Psychological review, 1987. Vol. 94, no. 2, pp. 148.
34. Levinson S.C. Frames of reference and Molyneux’s question: Crosslinguistic evidence. Language and space, Vol. 109, pp. 169.
35. Levinson S.C. Space in language and cognition: Explorations in cognitive diversity. Cambridge University Press, 2003. No. 5.
36. Lopez A., Postma A., Bosco A. Categorical & coordinate spatial information: Can they be disentangled in sketch maps? Journal of Environmental Psychology, 2020. Vol. 68, pp. 101392.
37. Luck S.J., Vogel E.K. The capacity of visual working memory for features and conjunctions. Nature, Vol. 390, no. 6657, pp. 279—281.
38. May J., Barnard P. Comprehending Dynamic Scenes: Cognitive Lessons from Cinematography, 2018.
39. McAfoose J., Baune B.T. Exploring visual—spatial working memory: A critical review of concepts and models. Neuropsychology review, 2009. Vol. 19, no. 1, pp. 130—142.
40. McKeefry D.J., Burton M.P., Vakrou C. Speed selectivity in visual short term memory for motion. Vision research, 2007. Vol. 47, no. 18, pp. 2418—2425.
41. Milner A.D., Goodale M.A. Oxford psychology series, No. 27. 1995.
42. Milner A.D., Goodale M.A. Two visual systems re-viewed. Neuropsychologia, Vol. 46, no. 3, pp. 774—785.
43. Moraresku S., Vlcek K. The use of egocentric and allocentric reference frames in static and dynamic conditions in humans. Physiological Research, Vol. 69, no. 5.
44. Nardini M., et al. A viewpoint-independent process for spatial reorientation. Cognition, 2009. Vol. 112, no. 2, pp. 241—248.
45. Oberauer K., et al. What limits working memory capacity? Psychological bulletin, 2016. Vol. 142, no. 7, pp. 758.
46. Pazzaglia F., Cornoldi C. The role of distinct components of visual-spatial working memory in the processing of texts. Memory, 1999. Vol. 7(1), pp.19—41.
47. Piaget J. The construction of reality in the child (M. Cook, Trans.). New York, NY, US,
48. Piaget J., Inhelder B. Die Psychologie des Kindes. Fischer-Taschenbuch-Verlag, 1983.
49. Piaget J., Inhelder В. L'image mental chez l'enfant. Paris: PUF, 1966.
50. Pick H.L. Comparative and developmental approaches to spatial cognition / Pick H.L., Acredolo L.P (eds.). Spatial orientation: Theory, research, and application.Y.: Plenum, 1983. Pp. 73—78.
51. Ruggiero G., et al. Egocentric metric representations in peripersonal space: A bridge between motor resources and spatial memory. British Journal of Psychology, 2021. Vol. 112, no.2, pp. 433—454.
52. Ruotolo F., Iachini T., Postma A., van der Ham I.J. Frames of reference and categorical and coordinate spatial relations: a hierarchical organization. Exp Brain Res, Vol. 214(4), pp. 587—595.
53. Ruotolo F., Iachini T., Ruggiero G., van der Ham I.J.M., Postma A.Frames of reference and categorical/coordinate spatial relations in a “what was where” task. Exp Brain Res, 2016. Vol. 234, pp. 2687—2696.
54. Ruotolo F., van der Ham I., Postma A., Ruggiero G., Iachini T.How coordinate and categorical spatialrelations combine with egocentric and allocentric reference frames in a motor task: effects of delay and stimuli characteristics. Behav Brain Res, 2015. Vol. 284, pp. 167—178.
55. Ruotolo F., van der Ham I.J.M., Iachini T., Postma A.The relationship between allocentric and egocentric frames of reference and categorical and coordinate spatial relations. Q J Exp Psychol, 2011. Vol. 64(6), pp. 1138—1156.
56. Ruotolo F., Claessen M.H.G., van der Ham I.J.M. Putting emotions in routes: the influence of emotionally laden landmarks on spatial memory. Psychological research, Vol. 83, no. 5, pp. 1083—1095.
57. Shepard R.N., Metzler J. Mental rotation of three-dimensional objects. Science, 1971. 171, no. 3972, pp. 701—703.
58. Shooner C., et al. High-capacity, transient retention of direction-of-motion information for multiple moving objects. Journal of Vision, 2010. Vol. 10, no. 6, pp. 8.
59. Sligte I.G., Scholte H.S., Lamme V.A.F. Are there multiple visual short-term memory stores? PLOS one, 2008. Vol. 3, no. 2, pp. e1699.
60. Sun Z., et al. How to break the configuration of moving objects? Geometric invariance in visual working memory. Journal of experimental psychology: human perception and performance, 2015. Vol. 41, no. 5, pp. 1247.
61. Tolman E.C. Purposive behavior in animals and men. Univ of California Press, 1932.
62. Wen W., Ishikawa T., Sato T. Individual differences in the encoding processes of egocentric and allocentric survey knowledge. Cognitive science, Vol. 37, no. 1, pp. 176—192.
63. Wen W., Ishikawa T., Sato T. Working memory in spatial knowledge acquisition: Differences in encoding processes and sense of direction. Applied Cognitive Psychology, 2011. Vol. 25, no. 4, pp. 654—662.
64. Wheeler M.E., Treisman A.M. Binding in short-term visual memory. Journal of Experimental Psychology: General, Vol. 131, no. 1, pp. 48.
65. Wood J.N. When do spatial and visual working memory interact?  Percept. Psychophys., 2011. Vol.73, pp. 420—439. DOI:10.3758/s13414-010-0048-8
66. Yan-yan S.U.N. Visual Working Memory in Different Scenes. Journal of Weifang Engineering Vocational College, 2013.
67. Zhang W., Luck S.J. Discrete fixed-resolution representations in visual working memory. Nature, Vol. 453, no. 7192, pp. 233—235.
68. Zokaei N., et al. Precision of working memory for visual motion sequences and transparent motion surfaces. Journal of vision, 2011. Vol. 11, no. 14, pp. 2.