Assessing the Aircraft Crew Activity Basing on Video Oculography Data

Mathematical models and methods for crew training level assessing based on video oculography data are presented. The results obtained are based on comparing the studied fragments of oculomotor activity of pilots with comparable patterns of video oculography data of various types and performance quality contained in a pre-formed specialized database. To obtain estimates, a complex combination of random process analysis and multivariate statistical analysis is used. The “intelligence” of diagnostic tools is contained in empirical data and can flexibly change as they accumulate. The considered example of determining the flight mode and pilot qualification based on video oculography data allows us to talk about the possibility of significant discrimination of the gaze movement trajectories of pilots at different flight phases and significant discrimination of the gaze movement trajectories of experienced and inexperienced pilots at certain phases of flight. An important new component of the presented results is a discriminant analysis for solving the problem of flight exercises classification, based on the principles of quantum computing. The scope of the considered approach is not limited to aviation applications and can be extended to tasks that are similar in content.

1. Barabanschikov V.A., Zhegallo A.V. Registraciya i analyz napravlennosty vzora cheloveka [Registration and analysis of the orientation of a person’s gaze]. Moscow Pub-l: Institute of psychology of the Russian Academy of Sciences, 2013, P. 316. (In Russ.).
2. Zheltov S.Yu., Fedosov E.A., Chuyanov G.A., Zlatomregev V.I., Greshnikov I.I. etc. Patent No. 101331 Kompleks oborudovaniya (stend) prototipirovaniya interfeisa cabiny vozdushnogo sudna / Pravoobladateli FGUP GosNIIAS (Russia). Zayavka 2016500077; Zayav. 15.01.2016; Zaregistr. 15.12.2016.(ROSPATENT). (In Russ.).
3. Krasil’shchikov M.N., Evdokimenkov V.N., Bazlev D.A. Individualno-adaptirovannye bortovye sistemy kontrolya tehnicheskogo sostoyaniya samlyota i podderzhki ypravlyayuschikh deistviy letchika [Individually adapted on-Board systems for monitoring the technical condition of the aircraft and supporting the pilot’s control actions]. Moscow: MAI Pub-l, 2011. P. 438. (In Russ.).
4. Kuravsky L.S., Yuriev G.A. Svidetel’stvo o gosudarstvennoi registratsii programmy dlyaEHVM №2018660358 Intelligent System for Flight Analysis v1.0 (ISFA#1.0) / Pravoobladateli Kuravskii L.S., Yur’ev G.A. (Russia). Zayavka №2018617617; Zayav. 18.07.2018; Zaregistr.22.08.2018. (ROSPATENT). (In Russ.).
5. Report on applied research on the topic “Development of human factor risk models and recommendations for creating a human-machine interface for the aircraft crew cabin” (intermediate), stage 1, State program of the Russian Federation “Development of the aviation industry for 2013-2025”, grant agreement dated 21.10.2019. No. 075-11-2019-018, state registration no. RFMEFI62819X0014. (In Russ.).
6. Report on applied research on the topic “Development of human factor risk models and recommendations for creating a human-machine interface for the aircraft crew cabin” (final), stage 2, State program of the Russian Federation “Development of the aviation industry for 2013—2025”, grant agreement dated 21.10.2019. No. 075-11-2019-018, state registration no. RFMEFI62819X0014. (In Russ.).
7. Kuravsky L.S., Yuryev G.A. A novel approach for recognizing abnormal activities of operators of complex technical systems: three non-standard metrics for comparing performance patterns, International Journal of Advanced Research in Engineering and Technology (IJARET), 11 (4), 2020, pp. 119—136. http://www. iaeme.com/IJARET/issues.asp?JType=IJARET&VType=11&IType=4. (Accessed 20.11.2020)
8. Kuravsky L.S. Discriminant Analysis Based on the Approaches of Quantum Computing. Lobachevskii Journal of Mathematics, 2020, Vol. 41, no. 12, pp. 2338—2344.
9. Kuravsky L.S., Yuryev G.A. The intelligent system to support condition monitoring for activities of operators of complex technical systems. — In: Proc. 16th International Conference on Condition Monitoring and Asset Management, Glasgow, UK, June 2019. DOI: 10.1784/cm.2019.108 17 pp.
10. Kuravsky L.S., Yuryev G.A., Detecting Abnormal Activities of Operators of Complex Technical Systems and their Causes Basing on Wavelet Representations. International Journal of Civil Engineering and Technology (IJCIET), 10(2), 2019, pp. 724—742. http://www.iaeme.com/IJCIET/ issues.asp?JType=IJCIE T&VType=10&IType=2.
11. Kuravsky L.S., Yuryev G.A., Zlatomrezhev V.I. New approaches for assessing the activities of operators of complex technical systems. Eksperimental’naya psikhologiya = Experimental psychology (Russia), 2019, vol. 12, no. 4, pp. 27—49. doi:10.17759/exppsy.2019120403.
12. Kuravsky L.S., Yuryev G.A., Zlatomrezhev V.I., Yuryeva N.E. Assessing the Aircraft Crew Actions with the Aid of a Human Factor Risk Model. Eksperimental’naya psikhologiya = Experimental Psychology (Russia), 2020. Vol. 13, no. 2, pp. 153—181. DOI: https://doi.org/10.17759/ exppsy.2020130211.
13. Aircraft trajectory clustering techniques using circular statistics. Yellowstone Conference Center, Big Sky, Montana, 2016. IEEE.
14. Bastani V., Marcenaro L., Regazzoni C. Unsupervised trajectory pattern classification using hierarchical Dirichlet Process Mixture hidden Markov model // 2014 IEEE International Workshop on Machine Learning for Signal Processing (MLSP) / IEEE. 2014. Pp. 1—6.
15. Enriquez M. Identifying temporally persistent flows in the terminal airspace via spectral clustering // Tenth USA/Europe Air Traffic Management Research and Development Seminar (ATM2013) / Federal Aviation Administration (FAA) and EUROCONTROL. Chicago, IL, USA: 2013. June 10-13.
16. Enriquez M., Kurcz C. A Simple and Robust Flow Detection Algorithm Based on Spectral Clustering // International Conference on Research in Air Transportation (ICRAT) / Federal Aviation Administration (FAA) and EUROCONTROL. Berkeley, CA, USA: 2012. May 22—25.
17. Faure C., Bardet J.M., Olteanu M., Lacaille J. Using Self-Organizing Maps for Clustering and Labelling Aircraft Engine Data Phases. In: WSOM (2017): 96—103.
18. Gaffney S., Smyth P. Joint probabilistic curve clustering and alignment // In Advances in Neural Information Processing Systems. Vol. 17. Cambridge, MA: MIT Press, 2005. Pp. 473—480.
19. Gaffney S., Smyth P. Trajectory clustering with mixtures of regression models // Proceedings of the fifth ACM SIGKDD international conference on Knowledge discovery and data mining. 1999. Pp. 63—72.
20. Laxhammar R., Falkman G. Online learning and sequential anomaly detection in trajectories // IEEE Transactions on Pattern Analysis and Machine Intelligence. 2014. Vol. 36, no. 6. Pp. 1158—1173.
21. Rintoul M., Wilson A. Trajectory analysis via a geometric feature space approach // Statistical Analysis and Data Mining: The ASA Data Science Journal. 2015.
22. Wilson A., Rintoul M., Valicka C. Exploratory Trajectory Clustering with Distance Geometry // International Conference on Augmented Cognition /Springer. 2016. Pp. 263—274.