Application of the m* and d* Statistics for Assessing the Quality of Data in Psychological Research Using Benford's Law (Illustrated with Reaction Time Measurements)

The objective of the proposed study was to examine the properties of statistics used to assess the conformity of the distribution of the first significant digit to Benford's Law, m* and d*, with relatively modest sample sizes (10≤n ≤70). A simulation study was conducted to achieve this goal. Data were simulated following a log-normal distribution with parameters that mimic the distribution of reaction time measurements. The distribution of the first significant digit was examined in standardized values raised to the power of γ; 5≤γ≤100. It was found that the statistic m* does not depend on the power of the number, unlike d*. Critical values were established for samples ranging from 10 to 70 observations with an increment of h=10. It turned out that for small n, the critical values of the statistic d* are close to asymptotic, while the critical values of the statistic m* are significantly larger. The functionality of the established critical values was tested within the framework of an experimental study: one respondent performed the Stroop cognitive test in accordance with the instructions (control case), while another violated them (experimental case). It was discovered that the statistic d* does not allow for differentiation in the behavior of subjects. Conversely, m* proved sensitive to changes in respondent behavior, and in the experimental case, it significantly more often allowed for the rejection of the null hypothesis regarding the conformity of the distribution of the first significant digit of the standardized value of reaction time to Benford's Law compared to the control. Thus, a preliminary conclusion is made that the statistic m* is more functional compared to d* in studying the quality of data on reaction time with small n.

1. Zajcev A.V., Lupandin V.I., Surnina O.E. Vremya reakcii v teoreticheskih i prikladnyh issledovaniyah. Psihologicheskij vestnik Ural'skogo gosudarstvennogo universiteta. Vyp. 3, 2002. Available at: https://elar.urfu.ru/bitstream/10995/3964/3/pv-01-03.pdf (Accessed: 03.03.2024). (In Russ.).
2. Zenkov A.V. Otkloneniya ot zakona Benforda i raspoznavanie avtorskih osobennostej v tekstah // Komp'yuternye issledovaniya i modelirovanie=Computer Research and Modeling, 2015. Vol. 7, no. 1, pp. 197–201. DOI:10.20537/2076-7633-2015-7-1-197-201 (In Russ.).
3. Karpenko L.A., Petrovskij A.V., YAroshevskij M.G. Kratkij psihologicheskoj slovar'. Rostov-na-Donu, «FENIKS», 1998. 505 p. (In Russ.).
4. Kuvakina L.V., Dolgopolova A.F. Zakon Benforda: sushchnost' i primenenie. Sovremennye naukoemkie tekhnologii, 2013. no. 6, pp. 74–76. (In Russ.).
5. Kulikova A. A., Prohorov YU. V. Odnostoronnie ustojchivye raspredeleniya i zakon Benforda. Teoriya veroyatnostej i ee primeneniya, 2004. Vol. 49, no. 1, pp. 178–184. DOI:10.4213/tvp244 (In Russ.).
6. Lone M. Teorema zontika, ili iskusstvo pravil'no smotret' na mir cherez prizmu matematiki. Bombora, 2022. 352 p. (In Russ.).
7. Osin E.N. Problema social'noj zhelatel'nosti v issledovaniyah lichnostnogo potenciala. 2011. Available at: https://www.hse.ru/data/2012/08/28/1242770673/Osin%202011.pdf?ysclid=lt46qcsjlj468944400 (Accessed: 27.02.2024) (In Russ.).
8. Popina O.YU., Savel'eva M.YU., Borodina YU.B. Ocenka kachestva otchetov o dvizhenii denezhnyh sredstv rossijskih organizacij s ispol'zovaniem zakona Benforda. Nauchnye issledovaniya: ot teorii k praktike, 2016. no. 4-2, pp. 187–190. (In Russ.).
9. Ushakova T.N., CHuprikova N.I. (Eds.) Psihologiya vysshih kognitivnyh processov. Moscow: In-t psihologii RAN, 2004. 303 p. (In Russ.).
10. Starunova O.A., Rudnev S.G., Ivanova A.E., Semenova V.G., Starodubov V.I. Primenenie zakona Benforda dlya ocenki kachestva dannyh profilakticheskogo skrininga. Matematicheskaya biologiya i bioinformatika=Mathematical biology and bioinformatics, 2022. Vol 17, no. 2. pp. 230–249. DOI:10.17537/2022.17.230 (In Russ.).
11. Antipkina I., Ludlow L. H. Parental involvement as a holistic concept using Rasch/Guttman scenario scales. Journal of Psychoeducational Assessment, 2020. Vol. 38, no. 7, pp. 846–865. DOI:10.1177/0734282920903164
12. Benford F. The law of anomalous numbers. Proceedings of the American philosophical society, 1938. pp. 551–572.
13. Campanelli L. On the Euclidean distance statistic of Benford’s law. Communications in Statistics-Theory and Methods, 2024. Vol. 53, no. 2. pp. 451–474. DOI:10.1080/03610926.2022.2082480
14. Chen H., Cohen P., Chen S. How big is a big odds ratio? Interpreting the magnitudes of odds ratios in epidemiological studies. Communications in Statistics—simulation and Computation®, 2010. Vol. 39, no. 4, pp. 860–864.DOI:10.1080/03610911003650383
15. Dixon P. The p-value fallacy and how to avoid it // Canadian Journal of Experimental Psychology. Revue canadienne de psychologie expérimentale, 2003. Vol. 57, no. 3, pp. 189–202. DOI:10.1037/h0087425
16. Dutilh G. et al. The quality of response time data inference: A blinded, collaborative assessment of the validity of cognitive models. Psychonomic Bulletin & Review, 2019. Vol. 26, pp. 1051–1069. DOI:10.3758/s13423-017-1417-2
17. Lachaud C. M., Renaud O. A tutorial for analyzing human reaction times: How to filter data, manage missing values, and choose a statistical model. Applied Psycholinguistics, 2011. Vol. 32, no. 2, pp. 389–416.DOI:10.1017/S0142716410000457
18. Marmolejo-Ramos F. et al. On the efficacy of procedures to normalize Ex-Gaussian distributions. Frontiers in Psychology, 2015. Vol. 5. DOI:10.3389/fpsyg.2014.01548
19. Marszalek J. M., Barber C., Kohlhart J., & Cooper B. H. Sample size in psychological research over the past 30 years. Perceptual and Motor Skills, 2011. Vol. 112, no. 2, pp. 331–348. DOI:10.2466/03.11.PMS.112.2.331-348
20. Microsoft Corporation. Microsoft Excel. 2018. Available at: https://office.microsoft.com/excel (Accessed: 02.03.2024)
21. Morrow J. Benford's Law, Families of Distributions and a Test Basis. CEP Discussion Papers dp1291, Centre for Economic Performance, LSE, 2014. Available at: https://cep.lse.ac.uk/pubs/download/dp1291.pdf (Accessed: 12.03.2024)
22. Peng K., Nisbett R. E., Wong N. Y. C. Validity problems comparing values across cultures and possible solutions. Psychological Methods, 1997. Vol. 2, no. 4, pp. 329–344. DOI:10.1037/1082-989X.2.4.329
23. Primi R. et al. Anchoring Vignettes: Can They Make Adolescent Self-Reports of Social-Emotional Skills More Reliable, Discriminant, and Criterion-Valid? European Journal of Psychological Assessment, 2016. Vol. 32, no. 1, pp. 39–51. DOI:10.1027/1015-5759/a000336
24. R Core Team. R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria. 2021. Available at: https://www.R-project.org/ (Accessed: 02.03.2024)
25. Stoet G. PsyToolkit: A novel web-based method for running online questionnaires and reaction-time experiments. Teaching of Psychology, 2017. Vol. 44, no. 1, pp. 24–31. DOI:10.1177/0098628316677643
26. Stroop J. R. Studies of interference in serial verbal reactions. Journal of Experimental Psychology, 1935. Vol. 18, no. 6, pp. 643–662.
27. The jamovi project. jamovi (Version 2.3) [Computer Software]. 2023. Available at: https://www.jamovi.org (Accessed: 02.03.2024)
28. Whelan R. Effective analysis of reaction time data. The Psychological Record, 2008. Vol. 58, pp. 475–482.DOI:10.1007/BF03395630
29. Zhang Q., Kong L., Jiang Y. The interaction of arousal and valence in affective priming: behavioral and electrophysiological evidence. Brain Research, 2012. Vol. 1474, pp. 60–72. DOI:10.1016/j.brainres.2012.07.023