Prediction of Students’ Academic Performance: the Correlation between the Results of the Unified State Exam and Academic Success

Introduction

Since the introduction of the Unified State Exam (USE) into the Russian education system, debates about its impact on student performance and academic success in universities have remained at the center of attention of the educational community. Attitudes towards this standardized test are divided into opposing views: while some see it as an objective tool for assessing knowledge, others express concerns about its impact on the quality of education and the process of knowledge formation among students [Kochetkova, 2020; Min'ko, 2017; Prakhov, 2012; Roshchina, 2006; Shchapov, 2009].

Most publications on the influence of the USE on university education can be divided into three groups:

1. Identifying regressors and the influence of factors on university preparation – on GPA or grades in individual subjects, primarily mathematics [Vlasova, 2018; Zamkov, 2013; Kudelin, 2021; Pereyaslavskaya, 2014; Rochev, 2022; Timofeeva, 2015].

2. Predicting university performance based on USE results, primarily using neural network models [Bogdanov, 2015; Poldin, 2011; Rusakov, 2018; Semerikov, 2021].

3. Theoretical understanding and description of psychological factors influencing the USE on university education [Azizova, 2019; Osipov, 2024; Osipov, 2022; Subbotin, 2019].

Alternative approaches to predicting academic achievements include:

• Comparing predictions based on entrance exam results and psychological characteristics, as discussed in the works of A.L. Duckworth, V.G. Erofeeva, and S.K. Nartova-Bochaver [Erofeeva, 2020; Duckworth, 2019].

• Reviewing international experience in assessment tests, as conducted in the works of D. Opposs, J.A. Baird, and others [Opposs, 2020].

• The growing popularity of using machine learning to identify hidden patterns [24; 26; 27; 30; 31].

Let us consider some studies closely related to ours in more detail.

In the study by N.A. Chernyshova [Chernyshova, 2017], data from 6,000 students over 4 years were analyzed. It was noted that the correlation between total USE scores and students' GPA is quite strong, leading to the conclusion that the USE is valid for monitoring the quality of school graduates' education and assessing applicants' knowledge.

O.O. Zamkov and A.A. Peresetsky [Zamkov, 2013] provide a detailed analysis of international research on the impact of national school exams on university education. Based on data from 505 students, they note that USE results are an adequate indicator for selecting students for the MIEF program.

L.B. Pereyaslavskaya and V.I. Pereyaslavsky [Pereyaslavskaya, 2014] examine correlations between USE scores in mathematics and academic performance in general education subjects among 394 students at two universities. The article reveals a significant change in performance compared to school results and shows that grades in mathematics at both universities correlate better with USE scores in Russian than with USE scores in mathematics, indicating problems with school mathematics education in the corresponding region.

E.A. Vlasova, N.M. Mezhennaya, and V.S. Popov [Vlasova, 2018] also note that the relationship between exam grades and USE results is statistically significant and more pronounced than the relationship with total test scores, despite the fact that applicants with high USE scores often demonstrate superficial subject knowledge, which hinders their ability to solve basic-level problems in entrance tests.

O.V. Poldin [Poldin, 2011], based on data from 157 students, studies the dependence of student dropout rates on their USE scores and notes that mathematics scores are most strongly associated with the dropout of economics students.

A.V. Semerikov and M.A. Glazyrin [Semerikov, 2021] propose predicting student success based on USE scores using a neural network model, demonstrating its 65% effectiveness based on data from 36,830 students.

S.V. Rusakov, O.L. Rusakova, and K.A. Posokhina [Rusakov, 2018], after studying data from 274 students using a neural network, suggest creating a "profile" of students at risk of dropout and low academic performance.

The relationship between academic performance and various factors is shown in articles [Kudelin, 2021; Prakhov, 2012a; Rochev, 2022; Oliveira, 2017].

In our study, we tested the extent to which the total USE score reflects individual students' learning abilities, which are later expressed in their performance in various university subjects. The main question requiring investigation is the extent to which USE results truly correlate with subsequent academic success in higher education institutions.

In this regard, the following research hypotheses were formulated:

• The total USE score correlates with subsequent academic success of students in higher education institutions.

• High USE scores indicate the presence of basic academic competencies and developed cognitive skills necessary for successful university education.

• USE results have predictive value for assessing future academic achievements of students.

These assumptions are based on the following facts:

• The exam is standardized and objectively measures students' knowledge and skills, minimizing the influence of external factors.

• High USE scores indicate the presence of basic academic competencies and developed cognitive skills necessary for successful university education.

• Previous studies show a positive correlation between USE results and students' academic achievements, confirming its predictive value.

Research Organization and Methods

Since 2006, UGNTU has operated a system for recording the activities of students and teachers. Based on data collected in the UGNTU information system, our study examined the relationship between USE results and academic performance: GPA, grades in individual subjects, and diploma grades.

Key concepts necessary for justifying the research scheme are defined:

1. Abilities: General (affecting performance in all subjects) and special (needed for success in a specific subject).

2. Competencies and learning outcomes: Competencies (knowledge, skills, and abilities for performing activities) and learning outcomes (specific knowledge, skills, and abilities to be achieved).

3. Learnability and training level: Learnability (a person's ability to acquire new knowledge) and training level (the level of knowledge, skills, and abilities achieved as a result of training).

4. Factors: Academic performance (degree of achievement of learning goals), motivation (factors driving activity), level of preparation (possession of knowledge, skills, and abilities for learning), socio-economic factors (family living standards, parents' education, etc.), and USE specifications (determining the content, structure, and format of the exam).

USE results should reflect abilities, competencies, learnability, training level, and the influence of the motivational-volitional sphere, which affect students' subsequent academic performance.

Research methods:

• Correlation analysis: Relationship between two variables.
• Regression analysis: Predicting the value of one variable based on another.
• Experimental method: Cause-and-effect relationship between two variables.

Research limitations:

• Difficulty in establishing cause-and-effect relationships.
• Imperfection of research methods.
• Influence of other factors.

The analysis was based on a database containing information on student performance over 13 years [Rochev]. Only data from students with both USE results and diploma grades were considered. Student grades were squared to enhance deviations.

The average score (quadratic) was calculated as:

Where:

O – average score (quadratic),

OE – exam grade (excluding credits),

n – number of student grades.

Where:

USE – normalized USE score,

E – USE exam score,

n – number of USE exams.

Based on diploma grades and calculated average scores, their dependence on the USE score was analyzed using a linear regression model:

Where:

y – output parameters,

x – input parameters,

b – regression coefficients.

For a more detailed study, a cohort analysis was performed (4). For this purpose, the whole data set was divided into specialty groups (5).

Where: y - output parameters,

x - input parameters,

b - coefficients of the regression equation.

For a more detailed study, cohort analysis was conducted. The entire dataset was divided into groups of specialties.

The study examined data from 45,743 students, but the final analysis included 9,520 students with USE results, full-time education, and successful diploma defense. Among them, 3,733 were female and 5,785 were male. Additional information is provided in Table 1.

The research was conducted using several software tools: Microsoft Excel, Statistica, the university management system, and a custom data analysis program developed for student activity analytics.

Table 1. Descriptive statistics

Results

First, let us consider the general picture of the relationship between various factors and academic performance.

Table 2. Correlation of various factors affecting students' academic performance and grade point average

Note: FEMIT - Faculty of Economics, Management and IT; TF - Faculty of Technology; NGF - Faculty of Oil and Gas.

Table 2 presents the results of correlation analysis between various factors and students' GPA. For all cases except the "Correspondence students" column, data from full-time students were considered. Results for students from specific faculties are also shown: Faculty of Economics, Management, and IT (FEMIT), Technological Faculty (TF), and Oil and Gas Faculty (NGF), as they differ significantly in their specifics. As can be seen from the table, the strongest correlation with GPA among the considered factors is shown by the USE score.

Table 3 presents similar data on the correlation between USE scores and various factors.

Table 3. Correlation of USE score and various factors affecting students' academic performance

Note: FEMIT - Faculty of Economics, Management and IT; TF - Faculty of Technology; NGF - Faculty of Oil and Gas.

As shown in Figures 1-3, there is a significant correlation between students' GPA and their USE scores upon admission. The correlation is 0.48, with 0.24 of the GPA explained by the USE score (significance above 0.001). Diploma grades also correlate with USE results, but to a lesser extent – 0.25, with only 0.07 of the variation in diploma grades explained by the USE score (significance above 0.001).

Normalized total USE score Average score squared

Fig. 1. Dependence of GPA on USE scores for all groups

 

Fig. 2. Dependence of GPA on USE scores for MON (Machinery and Equipment for Oil and Gas Complex – minimum significance of explained variation)

Fig. 3. Dependence of GPA on USE scores for FC (Finance and Credit – maximum significance of explained variation)

Table 4. Summary statistics on USE results by specialty groups, percentage of GPA and diploma grade variation explained by the regression equation from USE scores

Note: * – significance at 95%, ** – significance at 99%, *** – significance at 99.9%.

As is known, different specialties have different levels of competition and ranges of admission scores. It is logical to hypothesize that the greater the variation in USE scores among applicants, the greater the influence of this difference on subsequent academic performance. Since with a small variation in USE scores, the difference in academic performance among former schoolchildren is also small, they differ less at the start of their studies, and this difference is determined by further university education and other factors. To test this hypothesis, let us examine the obtained data in more detail – we will evaluate the regression of the difference between the minimum and maximum USE scores of students admitted to different specialties on the R-squared correlation between USE scores and their GPA (see Table 4) (5). As can be seen in Fig. 4, this regression is quite noticeable and amounts to 0.34 for GPA and 0.23 for diploma grades.

Where:

R – average score (quadratic),

r – R-squared correlation between USE scores and GPA or diploma grades,

n – number of students in the specialty,

m – number of specialties in the specialty group.

Thus, it can be concluded that the range of USE scores upon admission affects average academic performance at a significance level of 0.001 and diploma defense results at a significance level of 0.011.

Average standard deviation by USE

Percentage of variation in mean score explained by variation in USE

Fig. 4. Influence of the standard deviation of USE scores on the degree of correlation between USE scores and students' GPA

Conclusion

Based on the analysis of the presented data, we obtained significant dependencies of both GPA and diploma grades on USE results. Moreover, the greater the variation in USE scores among admitted students, the more accurately they determine subsequent academic performance. Undoubtedly, besides USE scores, there are many other factors that can significantly influence the quality of education. However, on average, most of the considered factors correlate with academic performance (as one of the main measurable parameters of education quality) significantly less than the abilities and motivational-volitional characteristics of students reflected in USE scores. Therefore, when building models for assessing education quality, it is advisable to include USE results as additional explanatory variables.

1. Azizova L.R. Edinyy gosudarstvennyy ekzamen (EGE) kak pedagogicheskiy eksperiment: plyusy i minusy [Unified State Exam (USE) as a pedagogical experiment: pros and cons]. Izvestiya Dagestanskogo gosudarstvennogo pedagogicheskogo universiteta. Psikhologo-pedagogicheskie nauki [Bulletin of the Dagestan State Pedagogical University. Psychological and Pedagogical Sciences], 2019, no. 13, 3, pp. 5–9. (In Russ.).
2. Bogdanov E.P., Sukhanov A.V. O prognozirovanii uspevaemosti studentov po rezul'tatam EGE i atributam sotsial'nogo statusa [On predicting student performance based on USE results and social status attributes]. Aktual'nye napravleniya nauchnykh issledovaniy XXI veka: teoriya i praktika [Current Directions of Scientific Research in the 21st Century: Theory and Practice], 2015. Vol. 3, no. 7-3(18-3), pp. 382–386. EDN: TKLXCH, DOI:10.12737/15210 (In Russ.).
3. Vlasova E.A., Mezhennaia N.M., Popov V.S. Sravnitel'nyy analiz rezul'tatov EGE, testa po proverke ostatocnykh znaniy i uspevaemosti pervokursnikov po matematike [Comparative analysis of USE results, residual knowledge test and freshmen’s academic performance in mathematics]. Internet-zhurnal "Mir nauki" [Internet Journal "World of Science"], 2018, no. 5. URL: https://mir-nauki.com/PDF/71PDMN518.pdf (open access). Title from the screen. Lang. Rus., Eng. EDN: YTUYNF. (In Russ.).
4. Erofeeva V.G., Nartova-Bochaver S.K. Chto takoe "Grit" i pochemu ona mozhet byt' lichnostnym resursom? [What is "Grit" and why it may be a personal resource?]. Sovremennaya zarubezhnaya psikhologiya [Modern Foreign Psychology], 2020. Vol. 9, no. 4, pp. 22–31. DOI:10.17759/jmfp.2020090402 (In Russ.).
5. Zamkov O.O., Peresetskiy A.A. EGE i akademicheskie uspehi studentov bakalavriata MIEF NIU VShE [USE and academic achievements of undergraduate students at ICEF HSE]. Prikladnaya ekonometrika [Applied Econometrics], 2013, no. 30(2). EDN: RYGOTT. (In Russ.).
6. Kochetkova T.N., Sashenko A.K. Mnogoaspektnost' problem i vozmozhnostey EGE [The multifaceted nature of problems and opportunities of USE]. Perspektivy nauki [Perspectives of Science], 2020, no. 5, pp. 95–100. (In Russ.).
7. Kudelin A.G., Shakirov D. Informatsionno-analiticheskaya sistema otsenki uchebnogo protsessa FGBOU VO "UGTU" [Information and analytical system for assessing the educational process of USTU]. Informatsionnye tekhnologii v upravlenii i ekonomike [Information Technology in Management and Economics], 2021, no. 2. EDN: RCWUEP. Available at: http://itue.ru/Issue/Article/151 (In Russ.).
8. Min'ko E.V., Min'ko A.E. Kachestvo i vostrebovannost' obrazovatel'nykh uslug [The quality and demand for educational services]. Uchebnoe posobie [Study Guide], Saratov, 2017. (In Russ.).
9. Osipov M.Yu. Mozhet li edinyy gosudarstvennyy ekzamen byt' effektivnym instrumentom otbora abiturientov v vuzy (soderzhatel'nye aspekty)? [Can the Unified State Exam be an effective tool for selecting university applicants (content aspects)?]. Alma Mater (Vestnik vysshey shkoly) [Alma Mater (Bulletin of Higher Education)], 2024, no. 1, pp. 62–68. (In Russ.).
10. Osipov M.Yu. Otsenka effektivnosti EGE kak instrumenta otbora studentov v vuzy (psikhologicheskiy aspekt) [Assessing the effectiveness of USE as a tool for university selection (psychological aspect)]. Vysshee obrazovanie segodnya [Higher Education Today], 2022, no. 9, pp. 90–94. DOI:10.18137/RNU.HET.22.09.P.090 (In Russ.).
11. Pereyaslavskaya L.B., Pereyaslavskiy V.I. Issledovanie korrelyatsiy mezhdu rezul'tatami EGE po matematike abiturientov i ikh uspevaemost'yu v vuze [Study of correlations between USE results in mathematics and university performance of applicants]. Vestnik Assotsiatsii vuzov turizma i servisa [Bulletin of the Association of Universities of Tourism and Services], 2014. Vol. 8, no. 4, pp. 49–56. EDN: SZDJRR, DOI:10.12737/6475 (In Russ.).
12. Poldin O.V. Prognozirovanie uspevaemosti v vuze po rezul'tatam EGE [Predicting university performance based on USE results]. Prikladnaya ekonometrika [Applied Econometrics], 2011, no. 1(21), pp. 56–69. EDN: SLPPNH, DOI:10.32609/0042-8736-2014-9-112-126 (In Russ.).
13. Prakhov I.A. Edinyy gosudarstvennyy ekzamen i determinanty rezul'tativnosti abiturientov: rol' investitsiy v podgotovku k postupleniyu [Unified State Exam and determinants of applicants’ success: the role of investments in preparation for admission]. Prikladnaya ekonometrika [Applied Econometrics], 2012, no. 3(27), pp. 86–108. EDN: PCBQYV. (In Russ.).
14. Prakhov I.A., Yudkevich M.M. Vliyanie dokhoda domokhozyaystv na rezul'taty EGE i vybor vuza [The influence of household income on USE results and university choice]. Voprosy obrazovaniya [Education Issues], 2012, no. 1, pp. 126–147. EDN: OXIMQB, DOI:10.17323/1814-9545-2012-1-126-147 (In Russ.).
15. Rochev K.V., Bazarova A.M. Otsenka faktorov, vliyayushchikh na uspevaemost' studentov na osnove bazy dannykh informatsionnoy sistemy vuza [Evaluation of factors influencing student performance based on the university’s information system database]. Pedagogicheskaya informatika [Pedagogical Informatics], 2022, no. 4, pp. 131–139. EDN: HKYNWR. (In Russ.).
16. Rochev K.V., Kudelin A.G. (2024) Analiz svjazi mezhdu rezul'tatami edinogo gosudarstvennogo jekzamena i akademicheskimi uspehami studentov [The analysis of the relationship between Unified State Exam results and students' academic achievements]. Dataset. DOI:10.48612/MSUPE/65u5-gf9t-45u3
17. Roshchina Ya.M. Povyshaet li EGE dostupnost' vysshego obrazovaniya? [Does the USE increase accessibility to higher education?]. Ekonomika obrazovaniya [Economics of Education], 2006, no. 3(34), pp. 27–50. EDN: HSSASX. (In Russ.).
18. Rusakov S.V., Rusakova O.L., Posokhina K.A. Neyrosetevaya model' prognozirovaniya gruppy riska po uspevaemosti studentov pervogo kursa [Neural network model for predicting the risk group of first-year students' performance]. Sovremennye informatsionnye tekhnologii i IT-obrazovanie [Modern Information Technologies and IT-Education], 2018, no. 14(4), pp. 815–822. EDN: JWGGFH, DOI:10.25559/SITITO.14.201804.815-822 (In Russ.).
19. Semerikov A.V., Glazyrin M.A. Prognozirovanie rezultata uspeshnosti zaversheniya obucheniya potentsial'nym studentom universiteta [Predicting the success of potential students in completing university education]. Informatsionnye tekhnologii v upravlenii i ekonomike [Information Technology in Management and Economics], 2021, no. 4. Available at: http://itue.ru/Issue/Article/181 (In Russ.).
20. Subbotin V.Ya., Khairullina N.G. Pokolenie EGE: posledstviya sovremennogo obrazovaniya dlya budushchego Rossii [The USE generation: the consequences of modern education for the future of Russia]. Vestnik Surgutskogo gosudarstvennogo pedagogicheskogo universiteta [Bulletin of the Surgut State Pedagogical University], 2019, no. 4(61), pp. 155–163. (In Russ.).
21. Timofeeva A.Yu., Avrunev O.E. Otchisleniya studentov v protsesse obucheniya: ob"yasnitel'naya sila EGE [Student dropouts during the learning process: explanatory power of the USE]. In the book: My prodolzhaem traditsii rossiyskoy statistiki. Materialy I Otkrytogo rossiyskogo statisticheskogo kongressa [We Continue the Traditions of Russian Statistics. Materials of the First Open Russian Statistical Congress], 2015, pp. 570–571. (In Russ.).
22. Chernyshova N.A. Svyaz' rezul'tatov EGE i akademicheskikh uspekh studentov v selskokhozyaystvennom vuze [The relationship between USE results and academic success of students in an agricultural university]. Vestnik Nizhegorodskogo universiteta im. N.I. Lobachevskogo. Seriya: Sotsial'nye nauki [Bulletin of N.I. Lobachevsky University of Nizhny Novgorod. Series: Social Sciences], 2017, no. 1(45), pp. 171–177. (In Russ.).
23. Shchapov A.N. Rezul'tat EGE i uspeshnost' obucheniya v vuze [USE results and university academic success]. Khimiya v shkole [Chemistry at School], 2009, no. 6, pp. 43–44. (In Russ.).
24. Alshanqiti A., Namoun A. Predicting student performance and its influential factors using hybrid regression and multi-label classification. IEEE Access, 2020, no. 8, p. 203827-203844. DOI:10.1109/ACCESS.2020.3036572
25. Duckworth A.L. Grit: Why Passion and Persistence are the Secrets to Success. London, 2019. 440 p.
26. Kardaş K., Güvenir A. Analysis of the effects of Quizzes, homeworks and projects on final exam with different machine learning techniques. EMO Journal of Scientific, 2020. Vol. 10(1), pp. 22–29.
27. Musso M.F., Hernández C.F.R., Cascallar E.C. Predicting key educational outcomes in academic trajectories: A machine-learning approach. Higher Education, 2020. Vol. 80(5), pp. 875–894. DOI:10.1007/s10734-020-00520-7
28. Oliveira I.M., Taveira M.D., Porfeli E.J. Career Preparedness and School Achievement of Portuguese Children: Longitudinal Trend Articulations. Frontiers in Psychology, 2017, no. 8. 618 p. DOI:10.3389/fpsyg.2017.00618
29. Opposs D., Baird J.-A., Chankseliani M., Johnson D., Stobart G., Kaushik A., McManus H. Governance structure and standard setting in educational assessment. Assessment in Education, 2020. Vol. 27, no. 2, pp. 192–214. DOI:10.1080/0969594X.2020.1730766
30. Waheed H., Hassan S.U., Aljohani N.R., Hardman J., Alelyani S., Nawaz R. Predicting academic performance of students from VLE big data using deep learning models. Computers in Human Behavior, 2020, no. 104, p. 106189. DOI:10.1016/j.chb.2019.106189
31. Yağcı M. Educational data mining: prediction of students' academic performance using machine learning algorithms. Smart Learning Environments, 2022. Vol. 9, no. 1, pp. 1–19. DOI:10.1186/s40561-022-00192-z