New approaches for assessing the activities of operators of complex technical systems



Presented are new approaches for supporting the outcome grading for activities of operators of complex technical systems, which are based on comparisons of current exercises with the activity database patterns in both the wavelet representation metric associated with time series of activity parameters and the likelihood metric of eigenvalue trajectories for these parameters transforms as well as on probabilistic assessments of skill class recognition using sample distribution functions of exercise distances to cluster centers in a scaling space and Bayesian likelihood estimations with the aid of probabilistic profile of staying in activity parameter ranges. These techniques have demonstrated the capabilities of recognizing sets of abnormal exercises and detection of parameters characterizing operator mistakes to reveal the causes of abnormality. The techniques in question overcome limitations of existing methods and provide advantages over manual data analysis since they greatly reduce the combinatorial enumeration of the options considered.

General Information

Keywords: operators of complex technical systems, discrete wavelet transform, skill class recognition, Principal Components Analysis, Multidimensional Scaling, Cluster Analysis, Discriminant Analysis, eigenvalue trajectories

Journal rubric: Cognitive Psychology

Article type: scientific article


Funding. This work was performed as a part of the “SAFEMODE” Project (Grant Agreement No 814961) with the financial support of the Ministry of Science and Higher Education of the Russian Federation (Project UID RFMEFI62819X0014).

For citation: Kuravsky L.S., Yuryev G.A., Zlatomrezhev V.I. New approaches for assessing the activities of operators of complex technical systems. Eksperimental'naâ psihologiâ = Experimental Psychology (Russia), 2019. Vol. 12, no. 4, pp. 27–49. DOI: 10.17759/exppsy.2019120403.

A Part of Article

An objective assessment of activity performance is essential for training process of operators of complex technical systems (OCTS). One of the critical aspects here is development of the training evaluation criteria.


  1. Aircraft trajectory clustering techniques using circular statistics. Yellowstone Conference Center, Big Sky, Montana, 2016. IEEE.
  2. Bastani V., Marcenaro L., Regazzoni C. Unsupervised trajectory pattern classification using hierarchical Dirichlet Process Mixture hidden Markov model //2014 IEEE International Workshop on Machine Learning for Signal Processing (MLSP) / IEEE. 2014. Pp. 1—6.
  3. Borg, I., Groenen, P. J. F. Modern Multidimensional Scaling Theory and Applications. — Springer, 2005. P. 140.
  4. Bress, Thomas J. Effective LabVIEW Programming: NTS Press, 2013. ISBN 1-934891-08-8.
  5. Cottrell M., Faure C., Lacaille J., Olteanu M. Anomaly Detection for Bivariate Signals. IWANN (1) 2019: 162—173.
  6. Cramer H. Mathematical Methods of Statistics. Princeton: Princeton University Press. 1999. — 575 pp.
  7. Eerland W.J., Box S. Trajectory Clustering, Modelling and Selection with the focus on Airspace Protection // AIAA Infotech@ Aerospace. _ AIAA, 2016. Pp. 1—14.
  8. Enriquez M. Identifying temporally persistent flows in the terminal airspace via spectral clustering // Tenth USA/Europe Air Traffic Management Research and Development Seminar (ATM2013) / Federal Aviation Administration (FAA) and EUROCONTROL. Chicago, IL, USA: 2013. June 10—13.
  9. Enriquez M., Kurcz C. A Simple and Robust Flow Detection Algorithm Based on Spectral Clustering // International Conference on Research in Air Transportation (ICRAT) / Federal Aviation Administration (FAA) and EUROCONTROL. — Berkeley, CA, USA: 2012. May 22—25.
  10. Faure C., Bardet J.M., Olteanu M., Lacaille J. Using Self-Organizing Maps for Clustering and Labelling Aircraft Engine Data Phases. In: WSOM (2017): 96—103.
  11. Gaffney S., Smyth P. Joint probabilistic curve clustering and alignment // In Advances in Neural Information Processing Systems. Vol. 17. Cambridge, MA: MIT Press, 2005. Pp. 473—480.
  12. Gaffney S., Smyth P. Trajectory clustering with mixtures of regression models // Proceedings of the fifth ACM SIGKDD international conference on Knowledge discovery and data mining. 1999. Pp. 63—72.
  13. Gariel M., Srivastava A., Feron E. Trajectory clustering and an application to airspace monitoring // IEEE Transactions on Intelligent Transportation Systems. 2011. Vol. 12, no. 4. Pp. 1511—1524.
  14. Grevtsov N. Synthesis of control algorithms for aircraft trajectories in time optimal climb and descent // Journal of Computer and Systems Sciences International. 2008. Vol. 47, no. 1. Pp. 129—138.
  15. Hung C., Peng W., Lee W. Clustering and aggregating clues of trajectories for mining trajectory patterns and routes // The VLDB Journal — The International Journal on Very Large Data Bases. 2015. Vol. 24, no. 2. Pp. 169—192.
  16. Krasilshchikov M.N., Evdokimenkov V.N., Bazlev D.A. Individually adapted airborne systems for monitoring the aircraft technical condition and supporting the pilot control actions. Moscow, MAI Publishing House, 2011. — 440 pp (in Russian).
  17. Kuchar J. K., Yang L. C. A review of conflict detection and resolution modeling methods // IEEE Transactions on Intelligent Transportation Systems. 2000. Vol. 1, no. 4. Pp. 179—189.
  18. Kuravsky L.S., Artemenkov S.L., Yuriev G.A., Grigorenko E.L. New approach to computer-based adaptive testing. Experimental Psychology. 2017. Vol. 10. No. 3. Pp. 33—45. doi:10.17759/exppsy.2017100303
  19. Kuravsky L.S., Margolis A.A., Marmalyuk P.A., Panfilova A.S. , Yuriev G.A. Mathematical aspects of the adaptive simulator concept — Psychological Science and Education. 2016. Vol. 21. No. 2. Pp. 84—95. doi: 10.17759/pse.2016210210 (in Russian).
  20. Kuravsky L.S., Margolis A.A., Marmalyuk P.A., Panfilova A.S., Yuryev G.A., Dumin P.N. A Probabilistic Model of Adaptive Training. — Applied Mathematical Sciences, Vol. 10, 2016, no. 48, 2369—2380, http://
  21. Kuravsky L.S., Marmalyuk P.A., Yurev G.A. Diagnostics of Professional Skills Based on Probability Distributions of Oculomotor Activity. — RFBR Journal, No. 3 (91), 2016, pp. 72—82 (Supplement to “Information Bulletin of RFBR” No. 24, in Russian).
  22. Kuravsky L.S., Marmalyuk P.A., Yuryev G.A. and Dumin P.N. A Numerical Technique for the Identification of Discrete-State Continuous-Time Markov Models — Applied Mathematical Sciences. Vol. 9, 2015, No. 8, pp. 379—391. URL: 2015.410882
  23. Kuravsky L.S., Marmalyuk P.A., Yuryev G.A., Belyaeva O.B. and Prokopieva O.Yu. Mathematical Foundations of Flight Crew Diagnostics Based on Videooculography Data. — Applied Mathematical Sciences, Vol. 10, 2016, no. 30, 1449—1466,
  24. Kuravsky L.S., Marmalyuk P.A., Yuryev G.A., Dumin P.N., Panfilova A.S. Probabilistic modeling of CM operator activity on the base of the Rasch model. — In: Proc. 12th International Conference on Condition Monitoring & Machinery Failure Prevention Technologies, Oxford, UK, June 2015.
  25. Kuravsky L.S., Yuriev G.A. Probabilistic method of filtering artefacts in adaptive testing. — Experimental Psychology, Vol.5, No. 1, 2012, pp. 119—131 (in Russian).
  26. Kuravsky L.S., Yuryev G.A. Certificate of state registration of the computer program № 2018660358 Intelligent System for Flight Analysis v1.0 (ISFA#1.0). Application №2018617617; declared 18 July 2018; registered 22 August 2018. (ROSPATENT).
  27. Kuravsky L.S., Yuryev G.A. Detecting abnormal activities of operators of complex technical systems and their causes basing on wavelet representations // International Journal of Civil Engineering and Technology (IJCIET). Vol. 10(2). P. 724—742. URL: IJCIET_10_02_070/IJCIET_10_02_070.pdf. (Accessed 17.09.2019).
  28. Kuravsky L.S., Yuryev G.A. On the approaches to assessing the skills of operators of complex technical systems. In: Proc. 15th International Conference on Condition Monitoring & Machinery Failure Prevention Technologies, Nottingham, UK, September 2018. 25 pp.
  29. Laxhammar R., Falkman G. Online learning and sequential anomaly detection in trajectories // IEEE Transactions on Pattern Analysis and Machine Intelligence. 2014. Vol. 36, no. 6. Pp. 1158—1173.
  30. Li Z., et al. Incremental clustering for trajectories // Database Systems for Advanced Applications. Lecture Notes in Computer Science. 2010. Vol. 5982. Pp. 32—46.
  31. Liu H., Li J. Unsupervised multi-target trajectory detection, learning and analysis in complicated environments // 2012 21st International Conference on Pattern Recognition (ICPR) / IEEE. 2012. Pp. 3716—3720.
  32. Markov models in the diagnostics and prediction problems: Textbook. /Edited by L.S. Kuravsky. 2nd Edition, Enlarged. — Moscow: MSUPE Edition, 2017. — 203 pp. (in Russian).
  33. Neal P.G. Multiresolution Analysis for Adaptive Refinement of Multiphase Flow Computations. University of Iowa, 2010.116 pp.
  34. René Vidal, Yi Ma, Shankar Sastry. Generalized Principal Component Analysis. Springer-Verlag: New York, 2016.URL: us/book/9780387878102
  35. Rintoul M., Wilson A. Trajectory analysis via a geometric feature space approach // Statistical Analysis and Data Mining: The ASA Data Science Journal. 2015.
  36. Trevor F. Cox, M.A.A. Cox. Multidimensional Scaling, Second Edition. Chapman & Hall/CRC, 2001. P. 299.
  37. Wei J., et al. Design and Evaluation of a Dynamic Sectorization Algorithm for Terminal Airspace // Journal of Guidance, Control, and Dynamics. 2014. Vol. 37, no. 5. Pp. 1539—1555.
  38. Wilson A., Rintoul M., Valicka C. Exploratory Trajectory Clustering with Distance Geometry // International Conference on Augmented Cognition /Springer. 2016. Pp. 263—274.
  39. Xiangyu Kong, Changhua Hu, Zhansheng Duan. Principal Component Analysis Networks and Algorithms. Springer, 2017. URL:

Information About the Authors

Lev S. Kuravsky, Doctor of Engineering, professor, Dean of the Computer Science Faculty, Moscow State University of Psychology and Education, Moscow, Russia, ORCID:, e-mail:

Grigory A. Yuryev, PhD in Physics and Matematics, Associate Professor, Head of Department of the Computer Science Faculty, Leading Researcher, Youth Laboratory Information Technologies for Psychological Diagnostics, Moscow State University of Psychology and Education, Moscow, Russia, ORCID:, e-mail:

Valentin I. Zlatomrezhev, Head of Laboratory, State Research Institute of Aviation Systems (GosNIIAS), Moscow, Russia, ORCID:, e-mail:



Total: 839
Previous month: 2
Current month: 2


Total: 342
Previous month: 0
Current month: 0