A model for predicting affective characteristics of color palette images

The study uses computer vision and machine learning technologies to assess the relationship between the emotional labeling of images and the characteristics of their color palette. The color palette of images in the HSV space was estimated using a modified image clustering algorithm, in which the cluster centroids were determined arbitrarily. The colors of the Luscher test were taken as the centroids of the clusters. The stimulus material for training the random forest model was taken from the Open affective standardized image set (OASIS) database. The parameters of the final random forest model for the test set: AUC = 0.77, Accuracy = 0.744, Kappa = 0.489; for the training set: AUC = 0.969, Accuracy = 0.904, Kappa = 0.808. The achieved classification accuracy can be interpreted as sufficient, provided that the original corpus labeling, on which the training took place, has emotional labeling that considers not only low-level characteristics of images, but also the semantics of scenes. To test the validity of the model, we: 1) used a pre-trained random forest model to estimate the emotional valence of a base of artistic photographs and a base of abstract images; 2) statistically assessed the quality of valence assessment of specific emotions in artistic photographs and abstract
images; 3) compared the results obtained for both bases. The obtained results allow us to conclude that the proposed random forest model is applicable to solving problems of image classification by color palette characteristics. If the confidence in classifying images as positive or negative is more than 60 %, we can predict that the color palette of the stimulus material will induce amusement, awe, excitement, fear, sad and content. The quality of recognition of negative emotions such as anger and disgust are not good enough. The proposed model is recommended for hybrid labeling of stimulus material, especially in cases of primary assessment of the emotional valence of images.

Bartoszek, G., Cervone, D. (2017). Toward an implicit measure of emotions: Ratings of abstract images reveal distinct emotional states. Cognition and Emotion, 31 (7), 1377–1391.

Bazyma, B. A. (2007). Psychology of color: Theory and practice. St. Petersburg, Rech Publ (In Russian)

Benjamini, Yo., Yekutieli, D. (2001). The control of the false discovery rate in multiple testing under dependency. Annals of statistics, 29 (4), 1165–1188.

Dubrovskaya, O. F. (2008). Manual on the use of the eight-color Lusher test. Moscow, Cogito-Centre Publ (In Russian)

Haight, M., Gruzdev, A. (2019). Learning pandas. Moscow, DMK Publ (In Russian)

Kurdi, B., Lozano, S., Banaji, M. R. (2017). Introducing the open affective standardized image set (OASIS). Behavior Research Methods, 49, 457–470.

Lüscher, M. ([1969]). Psychologie der Farben. English The Lüscher color test, transl. and ed. by I. A. Scott. New York, Random House

Machajdik, J., Hanbury, A. (2010). Affective image classification using features inspired by psychology and art theory. In: Proceedings of the 18th ACM international conference on Multimedia (MM ‘10) (pp. 83–92). New York, ACM https://doi.org/10.1145/1873951.1873965

Mastitsky, S. E., Shitikov, V. K. (2015). Statistical analysis and data visualisation using R. Moscow, DMK Publ (In Russian)

McKinney, W. (2020). Python and data analysis. Moscow, DMK Publ (In Russian)

Morozova, S. V. (2023). Text and image analysis in psychological research using the library ‘psyscaling’. In: Anan’evskie chteniia — 2023. 60 let social’noi psikhologii v SPbGU: Chelovek v sovremennom mire: potentsialy i perspektivy psikhologii razvitiia. Kirillitsa Publ (In Russian)

Sokolov, E. N., Boucsein, W. (2000). A psychophysiological model of emotion space. Integrative Physiological and Behavioral Science, 35, 81–119.

Solem, J. (2016). Programming computer vision in the Python language. Moscow, DMK Publ (In Russian)

Yeo, I. K., Johnson, R. A. (2000). A new family of power transformations to improve normality or symmetry. Biometrika, 87 (4), 954–959. https://doi.org/10.1093/biomet/87.4.954