Student Underperformance in Azerbaijan: Who is to Blame and What to Do?

This study provides an overview of education reforms in Azerbaijan since gaining independence, as well as a review of the dynamics of school success in the country based on data from final exams in lower secondary schools. The results of these exams show that despite the implemented reforms, students' academic success is not improving. The authors present factors that leading national education experts participating in research believe play a decisive role in students' success in school. These factors include the quality of teachers, their education, and professional development; standardized testing of students upon graduation from school; as well as school evaluations. The authors present a comparative analysis conducted on the identified "influence" factors in relation to data from international studies, as well as changes in national education policies and practices in terms of their effectiveness in ensuring successful school learning. The diagnosis of the national education system, using the RISE Systems Framework instrument (Research on Improving Education Systems), revealed that the alignment of all system elements in the reform process around academic success raises questions. The research findings may be useful for education policymakers, both in analyzing past reforms and in developing subsequent coherent changes in education policies aimed at enhancing students' success.

Introduction

National and international research on assessing the success of school education are important tools for analyzing and improving educational policies and practices. According to data from the State Examination Center (SEC) [9; 12], there is no decrease in the number of students demonstrating low results in graduation exams in the 9th grade of secondary schools (STEs).

The fact of the low success among 9th-grade graduates is of particular research interest, considering that the country has been undergoing educational reforms for more than two decades. For example, within the framework of the first State Education Reform Program [8], a curriculum reform was implemented from 2003 to 2013 [23]. In 2013, the State Program for the Development of Education was approved [31]. The need to improve the academic achievement of all students is emphasized in both strategic documents.

However, at the national level, there has been no research conducted to investigate the reasons for the low success of Azerbaijani schoolchildren. This study is pioneering both in terms of its research question and the novelty of the obtained results.

Problem Statement

In the graduation exam, the knowledge and skills of 9th-grade students are assessed in the subjects of the language of instruction (Azerbaijani or Russian), mathematics, and a foreign language. In this study, students with low academic performance are considered those who received a "2" or "3" on a 5-point scale in their school graduation exams. The grades 2 and 3 on the 5-point scale constitute 40% and 60% of the overall assessment, respectively.

Figure 1. Percentage of Underperforming Students by Subject from 2012 to 2020.

As evident from the reports of the State Examination Center (SEC) on the results of 9th-grade graduation exams [30], the percentage of students with low academic performance did not decrease below certain levels during the observation period. Specifically, in Azerbaijani language, it remained above 66.74% (2018), in Russian language above 46.2% (2014), and in Mathematics above 63.32% (2016). What exactly, and which factors are causing both a significant proportion of students with low academic performance and the overall decline in the success of graduates from an incomplete secondary education?

It is important to note that in 2006, when Azerbaijan first participated in the international PISA study, the country's results showed that Azerbaijani students performed relatively well in Mathematics compared to other countries with similar levels of economic development and even outperformed highly developed countries like the United States [20]. What changes in the country's education system since then have contributed to the decline in student success?

Research Method

To identify factors that can be considered priorities in influencing student success, the study employed an expert assessment method [13].

Individual in-depth interviews were conducted with leading national experts in the field of education. The experts participating in the study have extensive experience in the school education system (at least 20 years), have been involved in the development of the curriculum reform program or its implementation, and have held various leadership positions within the education system. The study involved 6 experts, including 2 men and 4 women, with their ages ranging from 55 to 75 years.

Expert Views on the Causes of Low Student Success

As factors exerting the greatest influence on the low academic performance of school graduates, experts unanimously identified the following:

Topic 1: Teacher Quality

Experts identified the low professionalism of teachers as the main problem in the educational process. According to their observations, teachers predominantly implement a reproductive teaching process based on memory development and rote learning, rather than a student-centered approach.

"Our teacher still perceives themselves as the authority in the classroom, and it is the duty of students to follow their instructions and obey," an expert stated.

Domestic teachers are oriented towards the average, standardized student and do not individualize education. When characterizing the professional qualities of contemporary teachers, experts used the following definitions: "Our teacher does not know or understand their student", "Our teacher does not develop critical thinking, empathy, or a democratic spirit in students."

The analysis of experts' assessments suggests that the problem also lies in the values required of teachers in their professional activities. "We have very few teachers who perceive the student as an individual and treat them with respect," "In the classroom, on average, 30% succeed, and the teacher does not think about the 70% who do not succeed," "The teacher is not motivated to teach the student... Everything depends on the teacher's level of professionalism, their level of culture." Teaching practices and the nature of teacher-student relationships are recognized as potentially powerful factors influencing student motivation and performance [33; 28].

Recognizing the importance of the personnel issue, the Ministry of Science and Education (MSE), starting from 2012, initiated several initiatives to strengthen the human resources of schools, such as verifying the authenticity of teachers' diplomas, introducing a pension threshold for teaching activities, prohibiting the employment of non-certified teachers in schools, teacher certification, and centralized hiring based on testing (MIG).

The experts tend to blame not the teachers themselves but the teacher training system for the current situation:

"It is necessary to start with pedagogical universities... with the preparation of pedagogical personnel. The most important factors determining the current state of affairs are gaps in the initial training of pedagogical personnel and the professional development system. Because a teacher is prepared for what MIG (MSE) demands. And MIG demands what the pedagogical university provided to the teacher", an expert stated.

In this view, the root of the problem lies in the preparation and education of teachers in pedagogical universities, which should align more closely with the requirements and standards set by the MSE. The experts emphasize the importance of improving the teacher education and the teacher professional development system to address the issues in the education system effectively.

Subtopic 1.1: Teacher Education

"Children's achievements largely depend on what teachers know and can do" [6, p. 11]. Such an approach helps identify the characteristics of teachers that influence student learning and, as a result, allows for possible recommendations for teacher education [3].

Experts consider the preparation of a new generation of teachers and the changing of the existing teacher preparation system to be of the highest priority. The main problems in this regard are the limited capacity of the teaching staff working in pedagogical universities. Another problem is outdated educational programs and the methods of their implementation.

"Our future teachers are taught various theories that students memorize but are not prepared for what they will encounter in practice, in the classroom. Our pedagogical education... does not develop the competencies necessary for teachers", an expert noted.

However, it is necessary to acknowledge that the teacher training system has not been the focus of education reforms in the country, including the curriculum reform.

Subtopic 1.2: Professional Development of Teachers

The importance of continuous and quality education for the professional growth and skill development of educators is recognized by its inclusion as a target indicator in the United Nations Sustainable Development Goals [34].

Experts believe that without reforming the teacher training system, professional development (PD) will not be effective, as teachers must be prepared for new approaches, and they must recognize the importance of PD for themselves. According to experts, the state allocates sufficient funds for PD, but its effectiveness is not high enough:

"For example, since 2003, the curriculum has been introduced. All teachers have undergone training. However, they all learned the theoretical part but cannot implement it in practice", an expert explained.

According to experts, teachers should be provided with diverse opportunities for PD, and it should be designed in a way that teachers can perceive its positive impact on themselves.

Moreover, it's important to note that the PD system has undergone significant changes in recent years. For example, starting in 2014, the Ministry of Education began conducting the diagnostic assessments of teachers, mandatory for every teacher every five years, which gradually evolved into teacher certification [19]. Certification is done through a test exam that assesses the knowledge of job functions, pedagogical knowledge, subject knowledge, and teaching method [32]. Teachers who successfully pass certification have had their salaries increased twofold, and their teaching load reduced by 1.5 times [18; 11].

Additionally, since 2010, the MSE initiated centralized electronic teacher recruitment based on competition (MIG) [17]. This process includes document authenticity verification, testing, the selection of vacant positions by participants, and personal interviews. During the testing phase, participants have the opportunity to answer questions related to their subject of instruction (40 questions) and teaching methodology (20 questions).

Both certification and competitive hiring have stimulated interest in PD among teaching staff. However, according to experts, to raise the professionalism of teachers, it is necessary to do so at the systemic level:

"...today teachers prepare for certification. Training courses are overcrowded, and there are no places. There are both online and face-to-face courses... teachers sign up and pay a certain amount... What are they learning there? Only theories. Are they learning anything practical? No, they learn how to answer questions on tests", an expert pointed out.

PD based on schools is implemented on a limited scale, mainly at the initiative of school principals. This is indirectly confirmed by the results of a survey of teachers in the country as part of an international study [21].

Figure 2. The Number of Teachers Who Participated in Professional Development Programs in the Last Three Months (Prior to the Survey).

Subtheme 3: Teacher Professional Standards

At the level of education policy, teacher certification and licensing serve as the foundation on which teacher education is built. Certification and licensing should be based on a set of agreed-upon national professional standards.

"We need professional teachers. But there is not a single document that clearly defines what this professionalism means", an expert said.

According to experts, the understanding of a teacher's professionalism still largely revolves around their subject knowledge, as well as their theoretical knowledge of pedagogy and teaching methods. However, the creation of professional teacher standards will help clarify what is expected of them and in which direction they should develop.

Topic 2: Standardized Testing (STE) of Students upon Leaving School

This theme is related to the impact of the final state exams conducted in 9th grade on the teaching and learning process. The general consensus among experts was that STE creates a demand for teaching only the specific subjects that are used to test students' knowledge. This, in turn, has some important consequences.

First and foremost, parents try to enroll their children in private tutoring for these specific subjects. Furthermore, parents even categorize subjects into two groups: important subjects that their children will study for university entrance exams and other subjects as less important.

"On the graduation exam, there are three different subjects. To prepare well for the exam, parents usually enroll their children in private tutoring… In the 9th grade, parents approached me and asked not to require their children to attend classes for less important subjects", an expert said.

Secondly, seeing the demand for private tutoring, teachers start teaching for the test rather than fostering the learning process in schools.

"Once, I asked English teachers why they spend the whole day preparing students for the 75 questions on the STE. They told me clearly that it's because there's a demand for it. Society demands it from us", an expert said.

Moreover, school teachers also start offering private tutoring services themselves:

"In reality, a teacher should prepare for the next class after lessons. They should work with students… Some students grasp things quickly, some are slower… A teacher should work on that… But here, the teacher comes, conducts the lessons, and then, until 8 p.m., does private tutoring", an expert said.

Last but not least, students lose interest in studying other subjects that are not part of the upcoming university entrance exam.

"Imagine that a student who doesn't attend school classes receives private tutoring to prepare for the upcoming test. This student successfully passes the exam. As a result, parents are satisfied. The student is very satisfied because it was easy. The teacher is also happy because they earn money. So, who is to complain?" an expert said.

The negative impact of private tutoring on the educational process in schools has been extensively researched in academic literature. It is believed that private tutoring can hinder the implementation of the curriculum and disrupt the classroom teaching process [26; 27].

Furthermore, the widespread practice of private tutoring deepens educational inequality for children from families with different socio-economic statuses (SES) and diminishes the role of schools as social lifts for some children:

"Not all families can afford private tutors for their children. For example, more than 300 students study in a 9th grade at a school. Only 50 or 60 of them go to private tutors. When those students take the exams, their scores will be satisfactory, even if they were not high-achieving students in previous years. However, those students who do not go to private tutors and were not high-achieving students in previous years typically show low results on the exams", an expert said.

Theme 3: School Evaluation

Schools "bear the direct and immediate responsibility for... improving the quality of teaching and achieving successful results in education and upbringing" [10, Article 5.4.4]. When assessing schools, sustained progress in the improvement of their results over several years is also taken into account. The MoSE compiles an overall ranking of schools based on several criteria, including the average score on the final exams in 9th grade [1].

"School principals work on a contract basis, which is signed for a year... Their work is assessed, and then a decision is made to extend the contract for another year... If the school's results are low... the Ministry terminates the contract with the school director...", an expert said.

As a result, many school principals, in order to achieve good results on the final exams in 9th grade, create groups of underachieving students at the beginning of the school year and start preparing them for the STE tests, ignoring the 9th-grade school curriculum. According to experts, this is how many schools improve their results.

Analysis and Discussion

The authors of the article attempted to visualize the expert perspective on the reasons for low student success, which is presented in the figure below.

Figure 3. Expert Perception of the Main Factors Affecting the Low Academic Performance of 9th-grade Graduates.

It would be tempting to point to any of the factors as the direct determinant of low learning outcomes [22] and propose corresponding solutions. For example, if teachers are involved in private tutoring, it is necessary to increase their salaries so that they can focus on their primary job. To attract the best educators from universities, you can stimulate the recruitment of top-performing students by offering them increased scholarships. Or, if certification is conducted effectively, it can help rid schools of unprepared and unprofessional teachers, and so on. All of these solutions have already been tested in practice in Azerbaijan. However, addressing the symptoms of the problem may lead to a false conclusion about the cause of the problem and a solution that has little impact on the quality of education [7].

As a first step toward understanding how the national education system can be transformed into a system capable of providing high-quality education for all, precise and comprehensive diagnosis of why the system has been producing low performance results for several years is necessary [24]. To achieve this, it is recommended to conduct an analysis of systemic inconsistencies in the interaction between parts of the system, which often lead to education systems failing to achieve learning outcomes [25].

Alignment with Goals Other than Learning

Educational systems provide education when strong relationships are built around the goal of education in all of its elements [4]. For example, in Azerbaijan, the "point of the curriculum was to replace the traditional approach that focused on acquiring academic knowledge and that was primarily oriented toward students with scientific abilities... to promote the development of students' creativity, critical thinking..." [16, p. 126]. However, this reform was not accompanied by an exam reform that would "measure" the very "soft skills" that the new curriculum was developing; instead, the previously introduced exam system of 1995 continued to measure academic knowledge, performing the task of final student assessment [12].

Another example of systemic inconsistency could be the professional role of teachers. Within the curriculum reform, teachers were supposed to adapt to a new vision of students as active participants in the learning process, having their own interests and needs, characterized by an individual learning style [16]. However, the curriculum reform was not accompanied by a reform of the teacher training system that would prepare the new teacher for their new role.

Another example of inconsistency at the education system level could be teacher professional development. "Curriculum reform also envisaged innovations in teacher professional development... Monitoring the process of implementing the new curriculum showed that teachers encountered numerous difficulties..." [16, p. 127]. The curriculum reform was accompanied by the continuous reform of the teacher professional development system, which eventually led to its elimination at the systemic level in the form inherited from Soviet times and its replacement with various training programs or courses offered by various providers, mainly on a commercial basis. At the same time, the idea of professional development for more effective student learning was gradually replaced by the idea of obtaining the right to be a teacher (certification) and the idea of employment (placement).

As a result of these systemic inconsistencies, teachers and schools in the country, aware of the importance of graduation exams, focus on academic content and aim at academically successful students, as was revealed in the case of curriculum reform in other countries [2].

Alignment with selectivity regarding students

The system in the country works to select the best-performing students for further education at the next level, thus operating as a "filtering" system [25], that considers exam scores as the main argument for filtering. To ensure better selectivity for their children, parents with high and medium socio-economic status enter the process by paying for the services of private tutors.

It is possible that as a result of these identified intra-system inconsistencies, the country faces difficulties in ensuring successful learning for all students.

Conclusion

The analysis of expert assessments of the factors influencing the low success of 9th-grade graduates in Azerbaijan indicates that the quality of teachers, their pedagogical education, continuous professional development, the use of standardized testing for graduation exams, and the existing school evaluation system are considered priority factors. As a result, the public school system, unable to fulfill its primary task of ensuring teaching quality and achieving successful learning outcomes, loses the trust of parents, students' motivation, and their ability to learn. Concurrently, a "shadow" school is emerging, based on private tutoring, which, unlike the public school, is only accessible to students from families with higher socio-economic status. Consequently, this situation creates and perpetuates inequality among students.

This study, due to its limited number of participants and its qualitative nature, cannot serve as an objective analysis of the overall situation. However, it can serve as a foundation for further, more comprehensive research. In particular, it opens the possibility of continuing the initiated research with the involvement of a larger number of participants representing all the stakeholders in the educational process and considering a wider range of possible influence factors. Additionally, a more comprehensive diagnosis of the education system can provide a better understanding of how to transform it to provide a quality education for all students.

The analysis of the results and recommendations can be used to inform education policymakers.

1. Микаилова У.Т. Опыт Азербайджана // В сборнике «Оценка работы школ в постсоветских странах» / Составители и научные редакторы В.А. Болотов, Р.В. Горбовский, Е.А. Ленская. М.: Издательский дом Высшей школы экономики, 2020. Mikayilova U.T. Opit Azerbaydjana [Experience of Azerbaijan]. Book “School assessment in post-Soviet countries”. In Bolotov V., Gorbovskiy R.V., Lenskaya E.A. (eds.). Moscow: Higher School of Economics Publishing House, 2020.

2. Anand G. Entrepreneurship Education and Teacher Training in Rwanda: Findings of the Qualitative Research Study. 2019.
3. Cochran-Smith M. et al. Researching Teacher Education in Changing Times: Politics and Paradigms of New Hampshire // Studying Teacher Education. Routledge, 2009. P. 81–122.
4. Crouch L. Systems implications for core instructional support lessons from Sobral (Brazil), Puebla (Mexico), and Kenya // RISE Insight Note. 2020.
5. De Ree J. et al. Double for nothing? Experimental evidence on an unconditional teacher salary increase in Indonesia // The Quarterly Journal of Economics. 2018. Vol. 133. № 2. P. 993–
6. Darling-Hammond L. Target Time Toward Teachers // Journal of staff Development. 1999. Vol. 20. № 2. P. 31–
7. Darling-Hammond L. Teacher quality and student achievement // Education policy analysis archives. 2000. Vol. 8. P. 1–
8. Glewwe P., Kremer M., Moulin S. Many children left behind? Textbooks and test scores in Kenya // American Economic Journal: Applied Economics. 2009. Vol. 1. № 1. P. 112–
9. Harlen W., Crick R.D. Testing and Motivation for Learning // Assessment in Education: Principles, Policy & Practice. 2003. Vol. 10(2). P. 169–207. DOI:10.1080/0969594032000121270
10. Horn C. High-Stakes Testing and Students: Stopping or Perpetuating a Cycle of Failure? // Theory Into Practice. Published by Ohio State University College of Education. 2003. Vol. 42. № 1. P. 30–41.
11. E-Qanun [E-Law]. Education Reform Program of the Republic of Azerbaijan. 1999. Available at: https://e-qanun.az/framework/5363
12. E-Qanun [E-Law]. On the approval of the Regulation on the State Commission for Student Admission of the Republic of Azerbaijan. Available at: https://e-qanun.az/framework/9510
13. E-Qanun [E-Law]. A Recommended Charter of Public General Schools. Available at: https://e-qanun.az/framework/21148
14. E-Qanun [E-Law]. On Increasing the Teaching Load And Salary of Teachers Whose Knowledge and Skills are Assessed Diagnostically in State General Educational Institutions. Available at: https://e-qanun.az/framework/29132
15. E-Qanun [E-Law]. On the creation of a public legal entity, the State Examination Center of the Republic of Azerbaijan. Available at: https://e-qanun.az/framework/32559
16. Klas C.P. Expert evaluation methods // Dobreva et al. 2012. P. 75–
17. Kellaghan T., Madaus G., Raczek A. The Use of External Examinations to Improve Student Motivation. Washington DC, AERA, 1996.
18. Madaus G., Russell M. Paradoxes of High-Stakes Testing // The Journal of Education. URBAN EDUCATION and OPPORTUNITY (2009/2010). Vol. 190. № 1/2. P. 21–30. Published by: Sage Publications, Inc. URL: https://www.jstor.org/stable/42744178
19. Meadows D.H. Thinking in systems: A primer. Chelsea green publishing, 2008.
20. Merriam S.B., Tisdell E.J. Qualitative research: A guide to design and implementation. John Wiley & Sons, 2015.
21. Mikayilova U., Kazimzade E. Teachers as reflective learners: teacher perception of professional development in the context of Azerbaijan’s Curriculum Reform // Вопросы образования. 2016. № 2 (eng). P. 125–
22. Ministry of Science and Education of the Republic of Azerbaijan [MoSE]. Yearly Report. Available at: https://edu.gov.az/uploads/Hesabat/TN-hesabat-2015.pdf
23. Ministry of Science and Education of the Republic of Azerbaijan [MoSE]. Yearly Report. Available at: https://edu.gov.az/uploads/Hesabat/TN-hesabat-2018.pdf
24. Ministry of Science and Education of the Republic of Azerbaijan [MoSE]. On the Continuation of Diagnostic Assessment of Teachers' Knowledge and Skills. Available at: https://edu.gov.az/az/esas-senedler/13698
25. Organization for Economic Cooperation and Development [OECD]. PISA 2006: Volume 2: Data, PISA, OECD Publishing, Paris. 2006.
26. Organization for Economic Cooperation and Development [OECD]. Programme for International Student Assessment (PISA) 2018: Teacher Questionnaire,
27. Pritchett L. Creating education systems coherent for learning outcomes // Research on Improving Systems of Education (RISE), Working Paper Series. 15. 2015. Vol. 5. P. 2–
28. Final observational study. 2012. Available at: http://www.kurikulum.az/index.php/az/tedqiqat/milli
29. Silberstein J. When the Devil’s Not in the Details: The System Failure of a Large-Scale School Management Reform in India // RISE Programme. 2020.
30. Silberstein J., Spivack M. Applying Systems Thinking to Education: Using the RISE Systems Framework to Diagnose Education Systems // RISE Insight note. RISE programme, University of Oxford. 2022.
31. Silova I. Private tutoring in Eastern Europe and Central Asia: Policy choices and implications // Compare. 2010. Vol. 40. № 3. P. 327–
32. Silova I., Budiene V., Bray M. Education in a hidden marketplace: Monitoring of private tutoring // New York, Open Society Institute, 2006.
33. Skinner E.A., Belmont M.J. Motivation in the classroom: Reciprocal effects of teacher behavior and student engagement across the school year // Journal of educational psychology. 1993. Vol. 85. № 4. P. 571.
34. Spann P. The Negative Effects of High-Stakes Testing // Education Law and Policy. 2015. URL: https://www.luc.edu/media/lucedu/law/centers/childlaw/childed/pdfs/2015studentpapers/Spann.pdf
35. State Examination Center. Statistical Analysis. Available at: https://dim.gov.az/activities/research/statistika/
36. State Examination Center. Available at: https://dim.gov.az/center/
37. State Strategy for the Development of Education in the Republic of Azerbaijan. Available at: https://president.az/az/articles/view/9779
38. The Institute of the Education of the Republic of Azerbaijan. Teacher Recruitment Competition and Certification. Available at: https://arti.edu.az/f%c9%99aliyy%c9%99timiz/mu%c9%99lliml%c9%99rin-inkisafi/miq-v%c9%99-sertifikasiya/
39. Wentzel K.R., Wigfield A. Academic and social motivational influences on students' academic performance // Educational Psychology Review. 1998. Vol. 10. P. 155–
40. United Nations Summit.GA Res 70th sess, 4th plen mtg, Agenda Items 15 and 116. TRANSFORMING OUR WORLD: THE 2030 AGENDA FOR SUSTAINABLE DEVELOPMENT, GA UN Doc A/RES/70/1 (21 October 2015).