Easy as pie. How not to treat statistics

The article discusses the methodological difficulties associated with the spread of statistical methods of data processing in psychological works. In particular, there is a simplification of the understanding of statistics by researchers, which increasingly leads to the adoption of the hypothesis of the presence of the effect, in a situation where it is not. There is a brief review of publications and examples where this issue is the subject of critical analysis. Presented a research which showed that in the repetition of experiments published in reputable journals, the significance accepted in the scientific community was observed only in 39% of the works — this effect was called the “publication shift”. The article discusses ways to solve this problem, in particular, meta-analysis, through which it is possible to confirm the results of a series of experiments relating to one problem — subject to the availability of information about the design and openness of raw data. An understanding of the probabilistic nature of statistical results must be mandatory for the evaluation of studies

1. Korneev A.A., Rasskazova E.I., Krichevec A.N., Kojfman A.YA. Kritika metodologii proverki nulevoj gipotezy: ogranicheniya i vozmozhnye puti vyhoda. CHast’ I. [Elektronnyi resurs] [Criticism of the methodology for testing the null hypothesis: limitations and possible ways out. Part I.]. Psihologicheskie issledovaniya [Psychological research], 2016. Vol. 9, no. 45, 1. URL: http://psystudy.ru/index.php/num/2016v9n45/1231-korneev45. html (Accessed 16.06.2018).
2. Korneev A.A., Rasskazova E.I., Krichevec A.N., Kojfman A.YA. Kritika metodologii proverki nulevoj gipotezy: ogranicheniya i vozmozhnye puti vyhoda. CHast’ II. [Criticism of the methodology for testing the null hypothesis: limitations and possible ways out. Part II.]. Psihologicheskie issledovaniya [Psychological research], 2016. Vol. 9, no. 47, 6. URL: http:// psystudy.ru/index.php/num/2016v9n47/1282-korneev47.html (Accessed 16.06.2018).
3. Nuzzo R. Statistical errors. Nature, 2014. Vol. 506, no. 13, pp. 150—152. doi:10.1038/506150a
4. Open Science Collaboration. Estimating the reproducibility of psychological science. Science, 2015. Vol. 349, ISSUE 6251 aac4716-1. doi: 10.1126/science.aac4716
5. Wasserstein R.L. Lazar N.A. The ASA’s statement on p-values: context, process, and purpose. The American Statistician, 2016. doi: 10.1080/00031305.2016.1154108