|
|
Automatic engagement detection in the education: critical review 56
Kasatkina D.A.
Doctor of Psychology, Expert of the Scientific Research Laboratory of Personality Development and Health Protection, Moscow City University, Moscow, Russia
ORCID: https://orcid.org/0000-0002-5248-5367
e-mail: darianemesis@gmail.com Kravchenko A.M.
Leading Research Associate of the Scientific Research Laboratory of Personality Development and Health Protection, , Moscow City University, Moscow, Russia
ORCID: https://orcid.org/0000-0001-8095-7141
e-mail: kravchenkoam@mgpu. ru Kupriyanov R.B.
Deputy Head of the Information Technology Department, Moscow City University, Moscow, Russia
ORCID: https://orcid.org/0000-0001-5153-4334
e-mail: kupriyanovrb@mgpu.ru Nekhorosheva E.V.
PhD in Education, Head of the scientific research laboratory of personality development and health protection, Moscow City University, Moscow, Russia
ORCID: https://orcid.org/0000-0002-1243-4223
e-mail: nehoroshevaev@mgpu.ru
This paper reviews the key research of the automatic engagement detection in education. Automatic engagement detection is necessary in enhancing educational process, there is a lack of out-of-the-box technical solutions. Engagement can be detected while tracing learning-centered affects: interest, confusion, frustration, delight, anger, boredom, and their facial and bodily expressions. Most of the researchers reveal these emotions on video using Facial Action Coding System (FACS). But there doesn’t exist a set of ready-made criteria to detect engagement and many scientists use additional techniques like self-reports, audio-data, physiological indicators and others. In this paper we present a review of most recent researches in the field of automatic affect and engagement detection and present our theoretical model of engagement in educational process based on the learning-centered affects’s detection. Engagement is understood as an affective and cognitive state, accompanying learning process. While reaching optimal engagement students experience various affects, where highly positive and negative feelings mean that a student is close to be engaged in the learning process.
- Il'in E.P. Psikhofiziologiya sostoyanii cheloveka [Psychophysiology of
human states]. St. Petersburg: Piter, 2005. 412 p.(In Russ.).
- Kupriyanov R.B. Primenenie tekhnologii komp'yuternogo zreniya dlya
avtomaticheskogo sbora dannykh ob emotsiyakh obuchayushchikhsya vo vremya
gruppovoi raboty [Application of computer vision technologies for automatic
collection of data about students' emotions during group work]. Informatika
i obrazovanie [Informatics and Education],2020. Vol. 314,
no. 5,pp. 56–63. DOI:10.32517/0234-0453-2020-35-5-56-63(In
Russ.).
- Altuwairqi K. et al. A new emotion–based affective model to detect
student’s engagement. Journal of King Saud University – Computer and
Information Sciences, 2019. In Press. DOI:10.1016/j.jksuci.2018.12.008
- Zeng Z. et al. A survey of affect recognition methods: Audio, visual, and
spontaneous expressions. IEEE Transactions on Pattern Analysis and Machine
Intelligence, 2009. Vol. 31, no. 1, pp. 39–58.
DOI:10.1109/TPAMI.2008.52
- Craig S. et al. Affect and learning: An exploratory look into the role of
affect in learning with AutoTutor. Journal of Educational Media, 2004.
Vol. 29, no. 3, pp. 241–250.
DOI:10.1080/1358165042000283101
- Ainley M. Connecting with learning: Motivation, affect and cognition in
interest processes. Educational Psychology Review, 2006. Vol. 18,
pp. 391–405. DOI:10.1007/s10648-006-9033-0
- Bosch N. et al. Automatic detection of learning-centered affective states
in the wild. In Proceedings of the 20th International Conference on
Intelligent User Interfaces. New York: Association for Computing Machinery,
2015, pp. 379–388. DOI:10.1145/2678025.2701397
- Baker R.S.J., Rodrigo M.M.T., Xolocotzin U.E. The Dynamics of Affective
Transitions in Simulation Problem-Solving Environments. International
Conference on Affective Computing and Intelligent Interaction. Affective
Computing and Intelligent Interaction. Lisbon: ACII, 2007,
pp. 666–677. DOI:10.1007/978-3-540-74889-2_58
- Beck J. Engagement tracing: using response times to model student
disengagement. In Chee-Kit Looi (ed.), Artificial Intelligence in Education:
Supporting Learning Through Intelligent and Socially Informed Technology.
Amsterdam: IOS Press, 2005, pp. 88–95.
- Bosch N. Detecting student engagement: Human versus machine Association for
Computing Machinery. In Vassileva J. et al. (eds.), UMAP '16: Proceedings of
the 2016 Conference on User Modeling Adaptation and Personalization. New
York: Association for Computing Machinery, 2016, pp. 317–320.
DOI:10.1145/2930238.2930371
- Bosch N., Chen Y., D’Mello S. It’s Written on Your Face: Detecting
Affective States from Facial Expressions while Learning Computer Programming.
Intelligent Tutoring Systems. Lecture Notes in Computer Science, 2014.
Vol. 8474, pp. 39–44. DOI:10.1007/978-3-319-07221-0_5
- Calvo M.G., Nummenmaa L. Processing of Unattended Emotional Visual Scenes.
Journal of Experimental Psychology: General, 2007. Vol. 136,
no. 3, pp. 347–369. DOI:10.1037/0096-3445.136.3.347
- Calvo R.A., D’Mello S. Affect detection: An interdisciplinary review of
models, methods, and their applications. IEEE Transactions on Affective
Computing, 2010. Vol. 1, no. 1, pp. 18–37.
DOI:10.1109/T-AFFC.2010.1
- Christenson S.L., Wylie C., Reschly A.L. Handbook of Research on Student
Engagement. New York: Springer, 2012.
840 p.DOI:10.1007/978-1-4614-2018-7
- Conati C., MacLaren H. Empirically building and evaluating a probabilistic
model of user affect.User Modeling and User-Adapted Interaction, 2009.
Vol. 19, pp. 267–303. DOI:10.1007/s11257-009-9062-8
- D’Mello S. A selective meta-analysis on the relative incidence of discrete
affective states during learning with technology. Journal of Educational
Psychology, 2013. Vol. 105, no. 4, pp. 1082–1099.
DOI:10.1037/a0032674
- D’Mello S., Graesser A. Dynamics of affective states during complex
learning. Learning and Instruction, 2012. Vol. 22, no. 2,
pp. 145–157. DOI:10.1016/j.learninstruc.2011.10.001
- Dixson M.D. Measuring student engagement in the online course: The online
student engagement scale (OSE).Journal of Asynchronous Learning Network,
2015. Vol. 4, no. 19. DOI:10.24059/olj.v19i4.561
- Ekman P., Freisen W.V., Ancoli S. Facial signs of emotional experience.
Journal of Personality and Social Psychology, 1980. Vol. 39,
no. 6, pp. 1125–1134. DOI:10.1037/h0077722
- Ekman P., Friesen W. V. Facial Action Coding System: A Technique for the
Measurement of Facial Movement. California: Consulting Psychologists Press,
1978. 197 p.
- Arroyo I. et al. Emotion Sensors Go to School. In Dimitrova V. et al.
(eds.), Artificial Intelligence in Education. Amsterdam: IOS Press,
2009, pp. 17–24. (Frontiers in Artificial Intelligence and
Applications).
- Bergdahl N. et al. Engagement, disengagement and performance when learning
with technologies in upper secondary school [Elektronnyi resurs]. Computers
& Education, 2020. Vol. 149, Article number 103783. URL:
https://www.sciencedirect.com/science/article/abs/pii/S0360131519303331
(Accessed 18.08.2020).
- Fredricks J.A., Blumenfeld P.C., Paris A.H. School Engagement: Potential of
the Concept, State of the Evidence. Review of Educational Research,
2004. Vol. 74, no. 1, pp. 59–109.
DOI:10.3102/00346543074001059
- Frenzel A.C., Pekrun R., Goetz T. Perceived learning environment and
students’ emotional experiences: A multilevel analysis of mathematics
classrooms. Learning and Instruction, 2007. Vol. 17, no. 5,
pp. 478–493. DOI:10.1016/j.learninstruc.2007.09.001
- Frijda N.H. Emotion, cognitive structure, and action tendency. Cognition
and Emotion, 1987. Vol. 1, no. 2, pp. 115–143.
DOI:10.1080/02699938708408043
- Kupriyanov R. et al. Intelligent tool for developing student’s social
skills [Elektronnyi resurs]. Proceedings of Edulearn20 Conference: 6th-7th
July 2020. Mallorca, 2020, pp. 2408–2413. URL:
http://132.66.16.6/~ilia1/744.pdf (Accessed 18.08.2020).
- Kapoor A., Burleson W., Picard R.W. Automatic prediction of frustration.
International Journal of Human Computer Studies, 2007. Vol. 65,
no. 8, pp. 724–736. DOI:10.1016/j.ijhcs.2007.02.003
- Marks H.M. Student Engagement in Instructional Activity: Patterns in the
Elementary, Middle, and High School Years. American Educational Research
Journal, 2000. Vol. 37, no. 1, pp. 153–184.
DOI:10.3102/00028312037001153
- Matthews G., Davies D.R. Individual differences in energetic arousal and
sustained attention: A dual-task study. Personality and Individual
Differences, 2001. Vol. 31, no. 4, pp. 575–589.
DOI:10.1016/S0191-8869(00)00162-8
- Appleton J.J. et al. Measuring cognitive and psychological engagement:
Validation of the Student Engagement Instrument. Journal of School
Psychology, 2006. Vol. 44, no. 5, pp. 427–445.
DOI:10.1016/j.jsp.2006.04.002
- Mota S., Picard R. Automated Posture Analysis for detecting Learner’s
Interest Level [Elektronnyi resurs].Conference on Computer Vision and
Pattern Recognition Workshop. Vol. 5. Madison: IEEE, 2003.,
pp. 49. URL: https://affect.media.mit.edu/pdfs/03.mota-picard.pdf
(Accessed 18.08.2020).
- Newmann F., Wehlage G., Lamborn S.D. The significance and sources of
student engagement. In Newman F. Student Engagement and Achievement in
American Secondary School. New York: Teachers College Press, 1992,
pp. 11–39. URL: https://files.eric.ed.gov/fulltext/ED371047.pdf#page=16
(Accessed 18.08.2020).
- Pantic M., Rothkrantz L. Toward an affect-sensitive multimodal
human-computer interaction. Proceedings of the Institute of Electrical and
Electronics Engineers, 2003, pp. 1370–1390.
DOI:10.1109/JPROC.2003.817122
- Pontual Falcão T. et al. Participatory methodologies to promote student
engagement in the development of educational digital games. Computers and
Education, 2018. Vol. 116, pp.161–175.
DOI:10.1016/j.compedu.2017.09.006
- Matthews G. et al. Profiling task stress with the dundee state
questionnaire. In Cavalcanti L., Azevedo S. (eds.), Psychology of
Stress. New York: Nova Science Publishers, 2013, pp. 49–91.
- Roseman I. Cognitive determinants of emotion: A structural theory.
Review of Personality & Social Psychology, 1984. Vol. 5,
pp. 11–36.
- Russell J.A. Core Affect and the Psychological Construction of Emotion.
Psychological Review, 2003. Vol. 110, no. 1, pp. 145–172.
DOI:10.1037/0033-295X.110.1.145
- Salovey P. Introduction: Emotion and Social Processes. In Davidson R.J.,
Scherer K.R., Goldsmith H.H. (eds.),Series in affective science. Handbook of
affective sciences. Oxford: Oxford University Press, 2003,
pp. 3–7.
- Scherer K.R. Facets of emotion: recent research. London: Psychology Press,
1988. 280 p.
- Schutz P., Pekrun R. Emotion in Education. London: Academic Press, 2007.
368 p.
- Whitehill J. et al. The faces of engagement: Automatic recognition of
student engagement from facial expressions.IEEE Transactions on Affective
Computing, 2014. Vol. 5, no. 1, pp. 86–98.
DOI:10.1109/TAFFC.2014.2316163
- Xie K., Heddy B.C., Greene B.A. Affordances of using mobile technology to
support experience-sampling method in examining college students’
engagement.Computers and Education, 2019. Vol. 128,
pp. 183–198. DOI:10.1016/j.compedu.2018.09.020
|
|
