Peculiarities of Interaction of a Reactive Agent with the Environment: a Model Study

The present work is dedicated to the empirical assessment of discrimination between two classes of methods – methods of stimuli presentation and methods of immersion in an environment. Experimental research was carried out on a computer model of a responsive agent. It has been showed that methods of immersion into an environment allows to explore a more broad class of phenomena than methods of stimuli presentation; for example, to retrace the dependence of an existing situation upon a preceding situation, preceding action, an experience of a subject; assess the ability of a subject to restructure the situation and affect on the complexity of a task by interaction with the environment. Nevertheless, the received data do not dismiss the possibility of an application of stimuli presentation methods for a research in specific forms of interaction of a subject with the environment when he cannot interfere with the situation.

1. Alexandrov I. O. Formirovanie struktury individual’nogo znaniya. (Organization of individual knowledge structure) M.: Izd-vo «IP RAN», 2006.
2. Alexandrov Yu. I. Teoriya funktsional’nyh system i sistemnaya psihofiziologiya (Functional systems theory and systemic psychophysiology) // Systemnie aspekty psihicheskoi deyatel’nosti (Systemic aspects of mental activity) / Pod red. K. V. Sudakova. M.: Edithorial URSS, 1999. S. 96–152.
3. Alexandrov Yu. I. Vvedenie v systemnuyu psihofiziologiyu (Introduction to systemic psychophysiology) // Psihologiya XXI veka. Pod. red. Druzhinina V. N. (Psychology of XXI century. Ed.: Druzhinin V. N.) M.: Per Se, 2004. S. 39–85.
4. Alexandrov Yu. I., Grechenko T. N., Gavrilov V. V., Gorkin A. G., Shevchenko D. G., Grinchenko Yu. V., Alexandrov I. O., Maksimova N. E., Bezdenezhnyh B. N., Bodunov M. V. Zakonomernosti formirovaniya i realizatsii individual’nogo opyta (Mechanisms of organization and realization of individual experience) // Zhurnal vyssh. nervn. deyat. (Journal of Higher Nervous Activity) 1997. T. 47. № 2. S. 243–260.
5. Alexandrov Yu. I., Krylov A. K. Systemnaya metodologiya v psihofiziologii: ot neironov do soznaniya (Systemic methodology in psychophysiology: from neurons to consciousness) // Ideya systemnosti v sovremennoi psihologii (The idea of system in modern Psychology) / Pod red. V. A. Barabanschikova. M.: IP RAN, 2005. S. 119–157.
6. Alexandrov Yu. I., Sozinov A. A., Averkin A. G., Laukka S. Fenomen proaktivnoi interferentsii: svyaz’ s emotsiami i vozmozhnie mozgovie osnovy (The phenomenon of proactive interference: its connection to emotions and probable neuronal basis) // Trudy nauchnogo soveta po eksperimental’noi i prikladnoi fiziologii. T. 14. Morfofunktsional’nie osnovy systemnoi deyatel’nosti. (The proceedings of research counsil in experimental and applied physiology. Vol. 14. Morfofunctional basis of systemic activity) M., 2007. S. 150–166.
7. Alexeev P. V., Panin A. V. Filosofiya. (Philosophy) M.: Prospekt, 1996.
8. Anohin P. K. Filosofskie aspekty teorii funktsional’noi systemy. (Philosophical aspects of functional system theory) M.: Nauka, 1978.
9. Anohin P. K. Iz tetradei P. K. Anohina (From P. K. Anohin's papers) // Psihologicheskij zhurnal. (Psychological journal) 1980. T. 1. № 4. S. 185–188.
10. Asmolov A. G. Osnovnie printsipy psihologicheskoi teorii deyatel’nosti (Basic principles of psychological theory of activity) // A. N. Leont’ev i sovremennaya psihologiya (A. N. Leont’ev and modern psychology) / Pod red. A. V. Zaporozhtsa. M.: Izd-vo Mosk. un-ta, 1983.
11. Asmolov A. G. Printsipy organizatsii pamyati cheloveka. (Principles of human memory organization) M.: Izd-vo Mosk. un-ta, 1985.
12. Barabanschikov V. A. Vospriyatie i sobytie. (Perception and event) SPb.: Aleteiya, 2002.
13. Batuev A. S. Vysshaya nervnaya deyatel’nost’. (Higher nervous activity) M.: Vyssh. shk., 1991.
14. Bezdenezhnyh B. N. Dinamika vzaimodeistviya funktsional’nyh system v strukture deyatel’nosti. (Dynamics of interaction between functional systems in activity structure) M.: Izd-vo «Institut psihologii RAN», 2004.
15. Bernshtein N. A. Ocherki po fiziologii dvizhenij i fiziologii aktivnosti. (Studies in physiology of movements and physiology of activity) M.: Meditsina, 1966.
16. Bernshtein N. A. Fiziologiya dvizhenij i aktivnost’. (Physiology of movements and activity) M.: Nauka, 1990.
17. Brushlinskij A. V. Myshlenie i prognozirovanie. (Thought and prediction) M.: Mysl’, 1979.
18. Vasilyuk F. E. Metodologicheskij analiz v psihologii. (Methodological analysis in Psychology) M.: MGPPU; Smysl, 2003.
19. Vygotsky L. S. Psihologiya razvitiya kak fenomen kul’tury. (Developmental Psychology as a cultural phenomenon) M.: Izdatel’stvo «Institut prakticheskoi psihologii», Voronezh: NPO «Modek», 1996.
20. Gorkin A. G., Shevchenko D. G. Otrazhenie istorii obucheniya v aktivnosti neironov limbicheskoi kory krolikov (Communication history reflection in limbic cortex neurons activity in rabbits) // Zhurn. vyssh. nervn. deyat. (Journal of higher nervous activity) 1993. T. 43. № 1. S. 172–175.
21. Gusev A. N. Psihofizika sensornyh zadach: systemno-deyatel’nostnyi analiz povedeniya cheloveka v situatsii neopredelennosti. (Psychophysics of sensory tasks: systemic activity analysis of human behavior in situations of uncertainty) M.: Izd-vo Mosk. un-ta: UMK «Psihologiya», 2004.
22. Zhuravlev A. L. Osobennosti mezhdisciplinarnyh issledovanij v psihologicheskoi nauke (Features of interdisciplinary studies in psychological science) // Materialy konferentsii «Psihologiya: Sovremennie napravleniya mezhdisciplinarnyh issledovanij». (Proceedings of the Conference "Psychology: modern trends of interdisciplinary studies") M.: Izd-vo «Institut psihologii RAN», 2003. S. 7–20.
23. Kornilova T. V., Smirnov S. D. Metodologicheskie osnovy psihologii. (Methodological basis of Psychology) SPb.: Piter, 2007.
24. Kruglikov R. I. Determinizm, aktivnost’, refleks (Detrminism, activity, reflex) // Metodologicheskie problemy fiziologii vysshei nervnoi deyatel’nosti. (Methodological issues of Physiology of higher nervous activity) M.: Nauka, 1982. S. 47–85.
25. Krylov A. K. Otsenka primenimosti reflektornoi modeli neironnoi seti k povedencheskoi zadache (Estimation of applicability of reflectory neural net model to behavioral task) // Trudy VI Vserossijskoi nauchno-tehnicheskoi konferentsii «Neiroinformatika-2004». (Proceedings of VI All-Russia Scientific and Technical Conference "Neuroinformatics-2004") M.: MIFI, 2004.
26. Krylov A. K. Testovaya povedencheskaya zadacha minimal’no neobhodimoi slozhnosti: skrytaya dinamika (Test behavioral task of baseline difficulty: its latent dynamics) // Trudy mezhdunarodnoi nauchno-tehnicheskoi konferentsii «Intellektual’nie systemy» (Proceedings of International Scientific and Technical Conference "Artificial intelligence Systems") (AIS’05). M.: Fizmatlit, 2005a. T. 1. S. 237–244.
27. Krylov A. K. Neopredelennost’ rezul’tata deistviya v paradigme Reinforcement Learning (Ambiguity in action result in paradigm of Reinforcement Learning) // Trudy III mezhdunarodnogo nauchno-prakticheskogo seminara «Integrirovannie modeli i myagkie vychisleniya v iskusstvennom intellekte». (Proceedings of International Scientific and Applied Seminar "Integrated models and soft computing in artificial intelligence") M.: Fizmatlit, 2005b. S. 238–243.
28. Krylov A. K. Pred’yavlenie stimulov ili pogruzhenie v sredu: model’noe issledovanie paradigm v psihofiziologii (Stimuli presentation or imbedding into environment: a model study of psychophysiological paradigms) // Materialy itogovoi nauchnoi konferentsii IP RAN (1–2 fevralya 2006 g.). (Proceedings of final scientific conference in IP RAS, February 1-2, 2006) M.: Izd-vo «Institut psihologii RAN», 2006. S. 111–120.
29. Krylov V. Yu. Metodologicheskie i teoreticheskie problemy matematicheskoi psihologii. (Methodological and theoretical issues of Mathematical Psychology) M.: Yanus-K, 2000.
30. Kun T. Struktura nauchnyh revolyutsij. (The Structure of Scientific Revolutions) M.: Progress, 1975.
31. Lakatos I. Fal’sifikatsia i metodologiya nauchno-issledovatel’skih programm. (The Falsification and Methodology of Scientific Research Programmes) M.: Medium, 1995.
32. Leont’ev A. N. Deyatel’nost’. Soznanie. Lichnost’. (Activity. Consciousness. Personality.) M. Politizdat, 1975.
33. Lomov B. F. Systemnost’ v psihologii. (Systems in Psychology) M.-Voronezh, 1996.
34. Lomov B. F., Nikolaev V. I., Rubahin V. F. Nekotorie voprosy primeneniya matematiki v psihologii (Several issues of mathematics application in Psychology) // Psihologiya i matematika. (Psychology and mathematics) M.: Nauka, 1976.
35. Luriya A. R. O meste psihologii v ryadu social’nyh i biologicheskih nauk (Concerning the place of Psychology among social and biological sciences) // Voprosy filosofii. (Issues of Philosophy) 1977. № 9. S. 68–76.
36. Menitskij D. N. Nekotorie metodologicheskie voprosy uslovnoreflektornoi teorii (Several methodological issues of conditional reflex theory) // Metodologicheskie voprosy teoreticheskoi meditsiny. (Methodological issues of theoretical medicine) L., 1975. S. 70–86.
37. Naisser U. Poznanie i real’nost’. (Cognition and reality) M.: Progress, 1981.
38. Pavlov I. P. Izbrannie proizvedeniya. (Selected works) M.: Izd. AN SSSR, 1949.
39. Petrenko V. F. Shkola A. N. Leont’eva v semanticheskom prostranstve psihologicheskoi mysli. (A. N. Leontyev school in semantic space of psychological thoughts) // Traditsii i perspektivy deyatel’nostnogo podhoda v psihologii: shkola A. N. Leont’eva. (Traditions and prospects of activity theory in Psychology: A. N. Leontyev school) M.: Smysl. 1999. S. 11–37.
40. Petrovskij A. V., Yaroshevskij M. G. Osnovy teoreticheskoi psihologii. (Basics of theoretical Psychology) M.: Infra-M, 1998.
41. Ponomarev Ya. A. Psihologiya tvoreniya. (Psychology of creation) M.: Moskovskij psihologo-social’nyi institut; Voronezh: Izd-vo NPO «MODEK», 1999.
42. Saltykov A. B., Toloknov A. V., Hitrov N. K. Povedenie i neopredelennost’ sredy. (Behavior and environment uncertainty) M.: Meditsina, 1996.
43. Skotnikova I. G. Sovremennoe sostoyanie sub’ektnoi psihofiziki (Subjectivу psychophysics: state of the art) // Materialy konferentsii «Psihologiya: Sovremennie napravleniya mezhdisciplinarnyh issledovanij». (Proceedings of the conference "Psychology. Modern trends of interdiscilinary studies") M.: Izd-vo «Institut psihologii RAN», 2003. S. 433–442.
44. Sozinov A. A., Laukka S., Averkin R. G., Alexandrov Yu. I. Usloviya i mozgovoe obespechenie interferentsii pri formirovanii systemnoi struktury individual’nogo opyta (Conditions anв neural support of interference during individual experience system structure developing) // Tendentsii razvitiya sovremennoi psihologicheskoi nauki. Otv. red. A. L. Zhuravlev, V. A. Kol’tsova. (Trends in modern psychological science. Ed.: A. L. Zhuravlev, V. A. Kol’tsova) M.: «Institut psihologii RAN», 2007. Ch. 2, S. 343–346.
45. Sokolov E. N., Vaitkyavichyus G. G. Neirointellekt: ot neirona k neirokomp’yuteru. (Neurointelligence: from neuron to neurocomputer) M.: Nauka, 1989.
46. Sudakov K. V. Refleks i funktsional’naya systema.(Reflexes and functional systems) Novgorod: NovGU, 1997.
47. Shvyrkov V. B. Neirofiziologicheskoe izuchenie systemnyh mehanizmov povedeniya. (Neurophysiological study of behavior system mechanisms) M.: Nauka, 1978.
48. Shvyrkov V. B. Vvedenie v ob’ektivnuyu psihologiyu: Neironal’nie osnovy psihiki / Pod red. Yu. I. Alexandrova. (Introduction to objective psychology: neuronal basis of mind) M.: Institut psihologii RAN, 1995.
49. Shingarov G. H. Teoriya i metod poznaniya I.P.Pavlova (Theory and research method of I.P. Pavlov) // Metodologicheskie problemy fiziologii vysshei nervnoi deyatel’nosti. Otv. red. R. I. Kruglikov. (Methodological problems of physiology of higher nervous activity. Ed.: R. I. Kruglikov) M.: Nauka, 1982.
50. Sherrington Ch. Integrativnaya deyatel’nost’ nervnoi systemy. (Integrative nervous system activity) L.: Nauka, 1969.
51. Yurevich A. V. Metodologicheskij liberalizm v psihologii (Methodological liberalism in Psychology) // Voprosy psihologii. (Issues of Psychology) 2001. № 5. S. 3–18.
52. Baldassarre G. Needs and motivations as mechanisms of learning and control of behaviour: Interference problems with multiple tasks // Cybernetics and Systems 2000 – Proceedings of the Fifteenth European Meeting on Cybernetics and Systems Research / Ed. R. Trappl. Vienna, 2000. P. 677–682.
53. Baldassarre G. Cultural evolution of “guiding criteria” and behavior in a population of neural networks agents // Journ. of memetics – Evolutionary models of Information Transmission. 2001. V. 4. Online journal.
54. Baldassarre G. A modular neural network model of the basal ganglia’s role in learning and selecting motor behaviours // Journ. of Cognitive Systems Research. 2002. V. 3. P. 5–13.
55. Beer R. D. Framing the debate between computational and dynamical approaches to cognitive science (commentary on The dynamical hypothesis in cognitive science by Tim van Gelder) // Behavioral and Brain Sciences. 1998. V. 21(5). P. 630.
56. Beer R. D. Dynamical approaches to cognitive science // Trends in Cognitive Sciences. 2000. V. 4(3). P. 91–99.
57. Brembs B., Heisenberg M. The Operant and the Classical in Conditioned Orientation of Drosophila melanogaster at the Flight Simulator // Learning&Memory. 2000. V. 7. № 2. P. 104–115.
58. Brooks R. A. Intelligence without representation // Artificial Intelligence. 1991а. V. 47. P. 139–159.
59. Brooks R. A. New approaches to robotics // Science. 1991б. V. 253. P. 1227–1232.
60. Colombetti M., Dorigo M. Training agents to perform sequential behavior. Adaptive behavior. MIT Press, 1994. 2(3). P. 247–275.
61. Houk C. J., Davis L. J., Beiser G. D. (eds.). Models of Information Processing in the Basal Ganglia. Cambridge, Mass.: The MIT Press, 1995.
62. Krylov A. K. The smartest agent is not optimal in a motivationally driven actor/critic model //Proceedings of the International Scientific Conferences Intelligent Systems (IEEE AIS’04) and Intelligent CAD’s (CAD-2004), 2004. P. 21–26.
63. Lombrozo T., Carey S. Functional explanation and the function of explanation // Cognition. 2005. Jun 4. P. 1–38.
64. Mirus C. V. Aristotle’s teleology and modern mechanics. Ph.D. thesis, 2004.
65. Nolfi S., Parisi D. Self-selection of input stimuli for improving performance // Neural Networks and Robotics / Ed. G. A. Bekey. Kluwer Academic Publisher, 1993. P. 403–441.
66. Redgrave P., Prescott T. J. and Gurney K. The basal ganglia: a vertebrate solution to the selection problem? // Neuroscience. 1999. V. 89. P. 1009–1023.
67. Seth A. K. Evolving action selection and selective attention without actions, attention, or selection // Proceedings of the Fifth International Conference on the Simulation of Adaptive Behaviour. Cambridge, MA. MIT Press, 1998. P. 139–147.
68. Sutton R., Barto A. Reinforcement Learning: An introduction. MIT Press, 1998.
69. Velichkovsky B. M. Heterarchy of cognition: The depths and the highs of a framework for memory research // Memory. 2002. V. 10 (5/6). P. 405–419.