Criteria Based Content Analysis (CBCA) as a tool for decision-making on message credibility: diagnostic and practical value of the method

CBCA (Criteria-Based Content Analysis) as a method for assessing the credibility of verbal statements is not universally recognized. The professional community lacks consensus regarding its reliability and validity. While this method is established within German-speaking forensic science, it is approached with caution in English-speaking and domestic contexts. Although it has not yet found widespread application in practice, CBCA holds potential for both conceptual research and practical implementation. Despite the relatively high diagnostic indicators of CBCA, they do not compensate for its inconvenience and the probability of error when applied in practice. This conclusion was drawn based on a field study of CBCA conducted in a police department during investigations of real incidents. During the new study, the distribution of CBCA criteria in messages was measured, and statistical methods were used to identify criteria whose presence differs between truthful and deceptive messages. Four new combinations of the identified criteria — formulas for interpreting the results and deciding on the trustworthiness of a message — were developed. The diagnostic indicators of the present study — sensitivity, specificity, and accuracy — were compared with those from previous studies. The diagnostic indicators of the new formulas were compared with three formulas proposed by foreign authors as well as with the results of canonical discriminant analysis. As a result, the reliability of CBCA in the new study was confirmed; however, its validity remains in question: the most practical formula turned out to consist of only one criterion, which undermines CBCA as a method. From a practical standpoint, CBCA is not recommended for making judicial decisions, and its applied value is auxiliary, intended primarily for use during the inquiry and preliminary investigation stages, and mainly for verifying deceptive messages.

Avramenko, A. P. (2023). Large language models in linguistics and linguodidactics. Moscow, KDU Publ., Dobrosvet Publ. https://doi.org/10.31453/kdu.ru.978-5-7913-1343-0-2023-129 (In Russian)

Aleksandrova, O. V. (2024). On approaches to assessing the reliability of testimony in investigative and judicial practice in modern Germany. Chelovek: prestuplenie i nakazanie, 32 (2), 204–215. Available at: https://www.elibrary.ru/item.asp?id=67916935

Baranov, A. N. (2007). Linguistic expertise of the text: theory and practice: textbook. Moscow, Flinta Publ., Nauka Publ. (In Russian)

Bagmet, A. M., Gusev, A. N., Engalychev, V. F., Kravtsova, G. K., Sedin, V. I., Kholopova, E. N. (2017). Methodology for identifying signs of veracity/falsity of information reported by participants in criminal proceedings (Based on video recordings of investigative actions and operational investigative measures): A scientific and practical manual. Moscow, FGBU GNTS FMBTs im. A. I. Burnazyana Press, Akademiya SK Rossii Press. (In Russian)

Danilin, M. D., Shestopalova, D. A. (2023). Using large language models in decision support systems. Strizh: Studencheskii elektronnyi zhurnal, 6 (53), 66–70. (In Russian)

Dozortseva, E. G., Afanasieva, A. G. (2015). Assessment of juveniles’ testimonies validity. Journal of Modern Foreign Psychology, 4 (3), 47–56. https://doi.org/10.17759/jmfp.2015040306 (In Russian)

Engalychev, V. F., Kravtsova, G. K., Kholopova, E. N. (2016). Forensic psychological examination for identifying signs of veracity/falsity of information reported by participants in criminal proceedings (Based on video recordings of investigative actions and operational investigative measures): monograph. Moscow, Yurlitinform Publ. (In Russian)

Fedorov, V. O., Polyakov, R. A. (2023). Large language models with search-enhanced generation: A review and prospects. Original’nye issledovaniia, 13 (12), 43–47. (In Russian)

Intelligent Decision Support Systems — A Brief Overview. Habr. URL: https://habr.com/ru/companies/ods/articles/359188/ (accessed: 17.11.2024). (In Russian)

Kozma, R., Puljic, M., Freeman, W. J. (2016). Thermodynamic model of criticality in the cortex based on EEG/ECoG data. In: Cognitive Phase Transitions in the Cerebral Cortex — Enhancing the Neuron Doctrine by Modeling Neural Fields (pp. 67–87). Berlin, Springer International Publishing. https://doi.org/10.1007/978-3-319-24424-7_4

Lapshina, A. V. (2022). Identification of speech signs of the veracity and falsity of information in linguistic expertise: master's thesis. Voronezh, Voronezhskii gosudarstvennyi universitet Press. (In Russian)

Obraztsov, V. A., Bogomolova, N. N. (2022). Forensic psychology. Moscow, Unity-Dana Publ. (Ser. “Pravo i zakon”) (In Russian)

Pozdnyakov, R. V. (2018). Structured and narrative interviews as methods in the structure of the methodology “assessment of the validity of statements”. In: M. V. Musiychuk, E. I. Shuleva (eds). Professionalizm i tvorchestvo. Sbornik materialov nauchno-prakticheskoi konferentsii (pp. 76–78). Moscow, MGTU im. Nosova. (In Russian)

Pozdnyakov, R. V., Gorbunov, I. A. (2023). Diagnostic informativeness of the methodology “Assessment of the validity of statements” when making a decision on trust in a speech message. In: Anan’evskie chteniia — 2023. Chelovek v sovremennom mire: potentsialy i perspektivy psikhologii razvitiia: Materialy mezhdunarodnoi nauchnoi konferentsii, Sankt-Peterburg, 17–20 oktiabria 2023 goda (p. 168). Moscow, Soiuzknig Publ., Kirillitsa Publ. (In Russian)

Prokopenko, N. Yu. (2017). Decision support systems based on Deductor Studio Academic 5.3. Nizhnii Novgorod, NNGASU Press. Available at: https://bibl.nngasu.ru/electronicresources/uch-metod/programming/866785.pdf (accessed: 17.11.2024). (In Russian)

Sporer, S. L., Masip, J. A. (2024). A systematic review of the validity of Criteria-based Content Analysis in child sexual abuse cases and other field studies. Psychology, Crime & Law, 1–42. https://doi.org/10.1080/1068316X.2024.2335971

Steller, M., Kohnken, G. (1989). Criteria-Based Content Analysis. In: Psychological methods in criminal investigation and evidence (pp. 217–245), D. C. Raskin (ed.). Berlin, Springer-Verlag.

Undeutch, U. (1967). Beurteilung der Glaubhaftigkeit von Aussagen. In: U. Undeutsch (Hrsg.). Handbuch der Psychologie, Forensische Psychologie. Bd. 11. (pp. 26–181). Göttingen, Verlag für Psychologie.

Volbert, R., Steller, M. (2023). Glaubhaftigkeit. In: T. Bliesener, K-P. Dahle, F. Losel (eds). Lehrbuch Rechtspsychologie (pp. 387–404), 2. Auflage. Bern, Verlag Hans Huber.

Vrij, A. (2005). Criteria-based content analysis: a qualitative review of the first 37 Studies. Psychology, Public Policy, and Law, 11 (1), 3–41. https://doi.org/10.1037/1076-8971.11.1.3

What is a Large Language Model (LLM)? Amazon Web Services. Available at: https://aws.amazon.com/ru/what-is/large-language-model (accessed: 17.11.2024). (In Russian)