Assessing the Aircraft Crew Activity Basing on Video Oculography Data

364

Abstract

Mathematical models and methods for crew training level assessing based on video oculography data are presented. The results obtained are based on comparing the studied fragments of oculomotor activity of pilots with comparable patterns of video oculography data of various types and performance quality contained in a pre-formed specialized database. To obtain estimates, a complex combination of random process analysis and multivariate statistical analysis is used. The “intelligence” of diagnostic tools is contained in empirical data and can flexibly change as they accumulate. The considered example of determining the flight mode and pilot qualification based on video oculography data allows us to talk about the possibility of significant discrimination of the gaze movement trajectories of pilots at different flight phases and significant discrimination of the gaze movement trajectories of experienced and inexperienced pilots at certain phases of flight. An important new component of the presented results is a discriminant analysis for solving the problem of flight exercises classification, based on the principles of quantum computing. The scope of the considered approach is not limited to aviation applications and can be extended to tasks that are similar in content.

General Information

Keywords: crew training level assessing, video oculography, Discriminant Analysis, Multidimensional Scaling, Cluster Analysis, oculomotor activity indexes.

Journal rubric: Psychology of Labor and Engineering Psychology

DOI: https://doi.org/10.17759/exppsy.2021140110

Funding. This work was performed as part of the “SAFEMODE” project (grant # 814961) with the financial support of the Ministry of Science and Higher Education of the Russian Federation (UID RFMEFI62819X0014 project)

For citation: Kuravsky L.S., Yuryev G.A., Zlatomrezhev V.I., Greshnikov I.I., Polyakov B.Y. Assessing the Aircraft Crew Activity Basing on Video Oculography Data. Eksperimental'naâ psihologiâ = Experimental Psychology (Russia), 2021. Vol. 14, no. 1, pp. 204–222. DOI: 10.17759/exppsy.2021140110.

References

  1. Barabanschikov V.A., Zhegallo A.V. Registraciya i analyz napravlennosty vzora cheloveka [Registration and analysis of the orientation of a person’s gaze]. Moscow Pub-l: Institute of psychology of the Russian Academy of Sciences, 2013, P. 316. (In Russ.).
  2. Zheltov S.Yu., Fedosov E.A., Chuyanov G.A., Zlatomregev V.I., Greshnikov I.I. etc. Patent No. 101331 Kompleks oborudovaniya (stend) prototipirovaniya interfeisa cabiny vozdushnogo sudna / Pravoobladateli FGUP GosNIIAS (Russia). Zayavka 2016500077; Zayav. 15.01.2016; Zaregistr. 15.12.2016.(ROSPATENT). (In Russ.).
  3. Krasil’shchikov M.N., Evdokimenkov V.N., Bazlev D.A. Individualno-adaptirovannye bortovye sistemy kontrolya tehnicheskogo sostoyaniya samlyota i podderzhki ypravlyayuschikh deistviy letchika [Individually adapted on-Board systems for monitoring the technical condition of the aircraft and supporting the pilot’s control actions]. Moscow: MAI Pub-l, 2011. P. 438. (In Russ.).
  4. Kuravsky L.S., Yuriev G.A. Svidetel’stvo o gosudarstvennoi registratsii programmy dlyaEHVM №2018660358 Intelligent System for Flight Analysis v1.0 (ISFA#1.0) / Pravoobladateli Kuravskii L.S., Yur’ev G.A. (Russia). Zayavka №2018617617; Zayav. 18.07.2018; Zaregistr.22.08.2018. (ROSPATENT). (In Russ.).
  5. Report on applied research on the topic “Development of human factor risk models and recommendations for creating a human-machine interface for the aircraft crew cabin” (intermediate), stage 1, State program of the Russian Federation “Development of the aviation industry for 2013-2025”, grant agreement dated 21.10.2019. No. 075-11-2019-018, state registration no. RFMEFI62819X0014. (In Russ.).
  6. Report on applied research on the topic “Development of human factor risk models and recommendations for creating a human-machine interface for the aircraft crew cabin” (final), stage 2, State program of the Russian Federation “Development of the aviation industry for 2013—2025”, grant agreement dated 21.10.2019. No. 075-11-2019-018, state registration no. RFMEFI62819X0014. (In Russ.).
  7. Kuravsky L.S., Yuryev G.A. A novel approach for recognizing abnormal activities of operators of complex technical systems: three non-standard metrics for comparing performance patterns, International Journal of Advanced Research in Engineering and Technology (IJARET), 11 (4), 2020, pp. 119—136. http://www. iaeme.com/IJARET/issues.asp?JType=IJARET&VType=11&IType=4. (Accessed 20.11.2020)
  8. Kuravsky L.S. Discriminant Analysis Based on the Approaches of Quantum Computing. Lobachevskii Journal of Mathematics, 2020, Vol. 41, no. 12, pp. 2338—2344.
  9. Kuravsky L.S., Yuryev G.A. The intelligent system to support condition monitoring for activities of operators of complex technical systems. — In: Proc. 16th International Conference on Condition Monitoring and Asset Management, Glasgow, UK, June 2019. DOI: 10.1784/cm.2019.108 17 pp.
  10. Kuravsky L.S., Yuryev G.A., Detecting Abnormal Activities of Operators of Complex Technical Systems and their Causes Basing on Wavelet Representations. International Journal of Civil Engineering and Technology (IJCIET), 10(2), 2019, pp. 724—742. http://www.iaeme.com/IJCIET/ issues.asp?JType=IJCIE T&VType=10&IType=2.
  11. Kuravsky L.S., Yuryev G.A., Zlatomrezhev V.I. New approaches for assessing the activities of operators of complex technical systems. Eksperimental’naya psikhologiya = Experimental psychology (Russia), 2019, vol. 12, no. 4, pp. 27—49. doi:10.17759/exppsy.2019120403.
  12. Kuravsky L.S., Yuryev G.A., Zlatomrezhev V.I., Yuryeva N.E. Assessing the Aircraft Crew Actions with the Aid of a Human Factor Risk Model. Eksperimental’naya psikhologiya = Experimental Psychology (Russia), 2020. Vol. 13, no. 2, pp. 153—181. DOI: https://doi.org/10.17759/ exppsy.2020130211.
  13. Aircraft trajectory clustering techniques using circular statistics. Yellowstone Conference Center, Big Sky, Montana, 2016. IEEE.
  14. Bastani V., Marcenaro L., Regazzoni C. Unsupervised trajectory pattern classification using hierarchical Dirichlet Process Mixture hidden Markov model // 2014 IEEE International Workshop on Machine Learning for Signal Processing (MLSP) / IEEE. 2014. Pp. 1—6.
  15. Enriquez M. Identifying temporally persistent flows in the terminal airspace via spectral clustering // Tenth USA/Europe Air Traffic Management Research and Development Seminar (ATM2013) / Federal Aviation Administration (FAA) and EUROCONTROL. Chicago, IL, USA: 2013. June 10-13.
  16. Enriquez M., Kurcz C. A Simple and Robust Flow Detection Algorithm Based on Spectral Clustering // International Conference on Research in Air Transportation (ICRAT) / Federal Aviation Administration (FAA) and EUROCONTROL. Berkeley, CA, USA: 2012. May 22—25.
  17. Faure C., Bardet J.M., Olteanu M., Lacaille J. Using Self-Organizing Maps for Clustering and Labelling Aircraft Engine Data Phases. In: WSOM (2017): 96—103.
  18. Gaffney S., Smyth P. Joint probabilistic curve clustering and alignment // In Advances in Neural Information Processing Systems. Vol. 17. Cambridge, MA: MIT Press, 2005. Pp. 473—480.
  19. Gaffney S., Smyth P. Trajectory clustering with mixtures of regression models // Proceedings of the fifth ACM SIGKDD international conference on Knowledge discovery and data mining. 1999. Pp. 63—72.
  20. Laxhammar R., Falkman G. Online learning and sequential anomaly detection in trajectories // IEEE Transactions on Pattern Analysis and Machine Intelligence. 2014. Vol. 36, no. 6. Pp. 1158—1173.
  21. Rintoul M., Wilson A. Trajectory analysis via a geometric feature space approach // Statistical Analysis and Data Mining: The ASA Data Science Journal. 2015.
  22. Wilson A., Rintoul M., Valicka C. Exploratory Trajectory Clustering with Distance Geometry // International Conference on Augmented Cognition /Springer. 2016. Pp. 263—274.

Information About the Authors

Lev S. Kuravsky, Doctor of Engineering, professor, Dean of the Computer Science Faculty, Moscow State University of Psychology and Education, Moscow, Russia, ORCID: https://orcid.org/0000-0002-3375-8446, e-mail: l.s.kuravsky@gmail.com

Grigory A. Yuryev, PhD in Physics and Matematics, Associate Professor, Head of Department of the Computer Science Faculty, Leading Researcher, Youth Laboratory Information Technologies for Psychological Diagnostics, Moscow State University of Psychology and Education, Moscow, Russia, ORCID: https://orcid.org/0000-0002-2960-6562, e-mail: g.a.yuryev@gmail.com

Valentin I. Zlatomrezhev, Head of Laboratory, State Research Institute of Aviation Systems (GosNIIAS), Moscow, Russia, ORCID: https://orcid.org/0000-0003-1776-6881, e-mail: vizlatomr@2100.gosniias.ru

Ivan I. Greshnikov, PhD in Engineering, Lead Engineer, State Research Institute of Aviation Systems (GosNIIAS), Graduate Student, Moscow State University of Psychology and Education (MSUPE), Moscow, Russia, ORCID: https://orcid.org/0000-0001-5474-3094, e-mail: vvanes@mail.ru

Borislav Y. Polyakov, Junior Researcher, Research Assistant, Laboratory of Mathematical Psychology and Applied Software of the Center for Information Technologies for Psychological Research, Moscow State University of Psychology and Education, Moscow, Russia, ORCID: https://orcid.org/0000-0002-6457-9520, e-mail: deslion@yandex.ru

Metrics

Views

Total: 860
Previous month: 11
Current month: 4

Downloads

Total: 364
Previous month: 5
Current month: 2