Ways of Acting with an Object as Part of its Representation

Goal. A representation is a multisensory model of an object that may or may not be represented in direct perceptual experience. In this regard, the question arises about the possibility of including in the representation information about the way of action with an object (motor programs). The purpose of this work is to analyze the existing theoretical concepts and empirical studies of the thesis about the inclusion of motor knowledge in the representation of an object and the influence of this knowledge on other cognitive processes. Methods. The work examines such theoretical approaches as the theory of the dual visual system, the model of embedded representation, as well as approaches based on manipulation and intention. In particular, the effect of compatibility is analyzed, which consists in reaction time reduction in the case of a congruence of the executed motor program and the perceived object. Results. The compatibility effect can be found in naming, categorization, and visual search tasks. The conditions for the occurrence of the compatibility effect in visual search are considered. Conclusions. It is assumed that there are two alternative explanations for the compatibility effect in the visual search — the conflict resolution in working memory or the inhibition of affordances hypothesis.

1. Anufrieva A.A., Gorbunova E.S. Affordansy kak chast' protsessa identifikatsii ob"ekta v zritel'nom poiske [Affordances as part of the process of object identification in visual search]. Rossiiskii psikhologicheskii zhurnal = Russian psychological journal, 2022. Vol. 19, no. 2, pp. 188—200. DOI:10.21702/rpj.2022.2.14 (In Russ.).
2. Anufrieva A.A., Gorbunova E.S. Rol' aktivatsii motornykh programm v zritel'nom poiske [Role of Activation of Motor Programs in Visual Search] [Electronic resourse]. In Vladimirov I.Yu., Korovkin S.Yu. (eds.), Materialy Vserossiiskoi nauchnoi konferentsii pamyati Dzh.S. Brunera “Psikhologiya poznaniya” [Proceedings of the All-Russian Scientific Conference in Memory of J.S. Bruner “Psychology of Cognition”]: Yaroslavl', 16—17 dekabrya 2022 g. Yaroslavl: Filigran', 2023, pp. 21—25. URL: https://www.elibrary.ru/item.asp?id=50741579 (Accessed 04.04.2024). (In Russ.).
3. Anufrieva A.A., Sapronov F.A., Gorbunova E.S. Effekt sovmestimosti v zadache zritel'nogo poiska [Compatibility effect in visual Search Task] [Electronic resource]. In Vladimirov I.Yu., Korovkin S.Yu. (eds.), Materialy Vserossiiskoi nauchnoi konferentsii pamyati Dzh.S. Brunera “Psikhologiya poznaniya” [Proceedings of the All-Russian Scientific Conference in Memory of J.S. Bruner “Psychology of Cognition”]: Yaroslavl', 01—03 dekabrya 2023 goda. Yaroslavl: Filigran', 2024, pp. 27—30. URL: https://elibrary.ru/item.asp?id=62489998 (Accessed 04.04.2024). (In Russ.).
4. Kotov A.A., Nosov A.V. Affordansy i kategorii: odinakov li effekt sovmestimosti po otnosheniyu k ob"ektam s raznym kategorial'nym statusom? [Accessibility and categories: the same or compatibility effect when looking at objects with different categorical status?] [Electronic resource]. Rossiiskii zhurnal kognitivnoi nauki [Russian Journal of Cognitive Science], 2017. Vol. 4, no. 2—3, pp. 39—48. URL: https://publications.hse.ru/pubs/share/direct/215667559.pdf (Accessed 04.04.2024). (In Russ.).
5. Mahon B.Z., Milleville S.C., Negri G.A.L., Rumiati R.I., Caramazza A., Martin A. Action-related properties shape object representations in the ventral stream. Neuron, 2007. Vol. 55, no. 3, pp. 507—520. DOI:10.1016/j.neuron.2007.07.011
6. Borghi A.M., Bonfiglioli C., Lugli L., Ricciardelli P., Rubichi S., Nicoletti R. Are visual stimuli sufficient to evoke motor information?: Studies with hand primes. Neuroscience Letters, 2007. Vol. 411, no. 1, pp. 17—21. DOI:10.1016/j.neulet.2006.10.003
7. Azaad S., Laham S.M., Shields P. A meta-analysis of the object-based compatibility effect. Cognition, 2019. Vol. 190, pp. 105—127. DOI:10.1016/j.cognition.2019.04.028
8. Bamford L.E., Klassen N.R., Karl J.M. Faster recognition of graspable targets defined by orientation in a visual search task. Experimental Brain Research, 2020. Vol. 238, no. 4, pp. 905—916. DOI:10.1007/s00221-020-05769-z
9. Borghi A.M. Object concepts and action. In Pecher D., Zwaan R.A. (eds.), Grounding cognition: The role of perception and action in memory, language, and thinking. N.Y.: Published by Cambridge University Press, 2005, pp. 8—34.
10. Popp M., Trumpp N.M., Sim E.J., Kiefer M. Brain activation during conceptual processing of action and sound verbs. Advances in Cognitive Psychology, 2019. Vol. 15, no. 4, pp. 236—255. DOI:10.5709/acp-0272-4
11. Bub D.N., Masson M.E.J., Lin T. Features of planned hand actions influence identification of graspable objects. Psychological Science, 2013. Vol. 24, no. 7, pp. 1269—1276. DOI:10.1177/0956797612472909
12. Chen Q., Garcea F.E., Mahon B.Z. The representation of object-directed action and function knowledge in the human brain. Cerebral Cortex, 2016. Vol. 26, no. 4, pp. 1609—1618. DOI:10.1093/cercor/bhu328
13. Creem S.H., Proffitt D.R. Defining the cortical visual systems:“what”,“where”, and “how”. Acta psychologica, 2001. Vol. 107, no. 1—3, pp. 43—68. DOI:10.1016/S0001-6918(01)00021-X
14. Adamo S.H., Gereke B.J., Shomstein S., Schmidt J. From “satisfaction of search” to “subsequent search misses”: a review of multiple-target search errors across radiology and cognitive science. Cognitive Research: Principles and Implications, 2021. Vol. 6, article ID 59. 19 p. DOI:10.1186/s41235-021-00318-w
15. Greco A. Spatial and Motor Aspects in the “Action-Sentence Compatibility Effect”. Frontiers in Psychology, 2021. Vol. 12, article ID 647899. 16 p. DOI:10.3389/fpsyg.2021.647899
16. Haddad L., Wamain Y., Kalénine S. Stimulus—response compatibility effects during object semantic categorisation: Evocation of grasp affordances or abstract coding of object size? Quarterly Journal of Experimental Psychology, 2023. Vol. 77, no. 1, pp. 29—41. DOI:10.1177/17470218231161310
17. Cohen H., Lefebvre C. (eds.), Handbook of Categorization in Cognitive Science. Amsterdam: Elsevier Science, 2005. 1136 p.
18. Hayward W.G. Whatever happened to object-centered representations? Perception, 2012. Vol. 41, no. 9, pp. 1153—1162. DOI:10.1068/p7338
19. Caligiore D., Borghi A., Parisi D., Ellis R., Cangelosi A., Baldassarre G. How affordances associated with a distractor object affect compatibility effects: A study with the computational model TRoPICALS. Psychological Research, 2013. Vol. 77, pp. 7—19. DOI:10.1007/s00426-012-0424-1
20. Flumini A., Barca L., Borghi A.M., Pezzulo G. How do you hold your mouse? Tracking the compatibility effect between hand posture and stimulus size. Psychological research, 2015. Vol. 79, pp. 928—938. DOI:10.1007/s00426-014-0622-0
21. Kozuch B. Conscious vision guides motor action—rarely. Philosophical Psychology, 2023. Vol. 36, no. 3, pp. 443—476. DOI:10.1080/09515089.2022.2044461
22. Kriegel U. Two notions of mental representation. In Kriegel U., Current controversies in philosophy of mind. New York: Routledge, 2013, pp. 161—179. DOI:10.4324/9780203116623
23. Lacey S., Sathian K. Representation of object form in vision and touch [Electronic resource]. In Murray M.M., Wallace M.T. (eds.), The neural bases of multisensory processes. Boca Raton: CRC Press/Taylor & Francis, 2012, pp. 179—190. URL: https://www.ncbi.nlm.nih.gov/books/NBK92850/ (Accessed 04.04.2024).
24. Mahon B.Z., Hickok G. Arguments about the nature of concepts: Symbols, embodiment, and beyond. Psychonomic bulletin & review, 2016. Vol. 23, pp. 941—958. DOI:10.3758/s13423-016-1045-2
25. Martin A. GRAPES—Grounding representations in action, perception, and emotion systems: How object properties and categories are represented in the human brain. Psychonomic bulletin & review, 2016. Vol. 23, pp. 979—990. DOI:10.3758/s13423-015-0842-3
26. Maxfield J.T., Zelinsky G.J. Searching through the hierarchy: How level of target categorization affects visual search. Visual cognition, 2012. Vol. 20, no. 10, pp. 1153—1163. DOI:10.1080/13506285.2012.735718
27. McKellar P. Imagination and thinking: A psychological analysis. New York: Basic Books, 1957. 248 p.
28. Milner A.D., Goodale M.A. Visual pathways to perception and action. Progress in brain research, 1993. Vol. 95, pp. 317—337. DOI:10.1016/S0079-6123(08)60379-9
29. Moise N. Getting a Handle on Meaning: Planned Hand Actions' Influence on the Identification of Handled Objects: a thesis submitted in partial fulfillment of the requirements for the degree of Master of science in the Department of Psychology [Electronic resource]. Victoria, 2022. 38 p. URL: http://hdl.handle.net/1828/14285 (Accessed 04.04.2024).
30. Moretti S., Greco A. Assessing with the head: a motor compatibility effect. MOCO '18: Proceedings of the 5th International Conference on Movement and Computing (Genoa, June 28—30, 2018). New York: Association for Computing Machinery, 2018. Article ID 35. 4 p. DOI:10.1145/3212721.3212853
31. Zang A., Wang H., Guo H., Wang Y. Motor Compatibility Effect on the Comprehension of Complex Manual Action Sentences in L2: An ERP Study. Chinese Journal of Applied Linguistics, 2022. Vol. 45, no. 2, pp. 176—193. DOI:10.1515/cjal-2022-0202
32. Ni L., Liu Y., Yu W. The dominant role of functional action representation in object recognition. Experimental brain research, 2019. Vol. 237, pp. 363—375. DOI:10.1007/s00221-018-5426-9
33. Osiurak F., Rossetti Y., Badets A. What is an affordance? 40 years later. Neuroscience & Biobehavioral Reviews, 2017. Vol. 77, pp. 403—417. DOI:10.1016/j.neubiorev.2017.04.014
34. D’Ascenzo S., Lugli L., Nicoletti R., Umiltà C. Practice effects vs. transfer effects in the Simon task. Psychological Research, 2020. Vol. 85, pp. 1955—1969. DOI:10.1007/s00426-020-01386-1
35. Lee C.-L., Huang H., Federmeier K.D., Buxbaum L.J. Sensory and semantic activations evoked by action attributes of manipulable objects: Evidence from ERPs. NeuroImage, 2018. Vol. 167, pp. 331—341. DOI:10.1016/j.neuroimage.2017.11.045
36. Sztybel P., Gómez M.A., Snow J.C. Graspable objects grab attention more than images do — even when no motor response is required. Journal of Vision, 2019. Vol. 19, no. 10, article ID 221. DOI:10.1167/19.10.221
37. Harrak M.H., Heurley L., Morgado N., Mennella R., Dru V. The visual size of graspable objects is needed to induce the potentiation of grasping behaviors even with verbal stimuli. Psychological Research, 2022. Vol. 86, no. 7, pp. 2067—2082. DOI:10.1007/s00426-021-01635-x
38. Tucker M., Ellis R. The potentiation of grasp types during visual object categorization. Visual cognition, 2001. Vol. 8, no. 6, pp. 769—800. DOI:10.1080/13506280042000144
39. Wolfe J.M. Asymmetries in visual search: An introduction. Perception & psychophysics, 2001. Vol. 63, pp. 381—389. DOI:10.3758/BF03194406
40. Yamani Y., Ariga A., Yamada Y. Object affordances potentiate responses but do not guide attentional prioritization. Frontiers in integrative neuroscience, 2016. Vol. 9, article ID 74. 6 p. DOI:10.3389/fnint.2015.00074