Reading Skills of First Graders in Russia and Kazakhstan: A Cross-Cultural Study

This article assesses the intercultural comparability of reading assessment results taking into account the specifics of the test content in relation to the child’s cultural environment. The reading skills of first graders in two countries were assessed using the reading scale of the computerized instrument “Start”. The sample of students from Kazakhstan included 1102 first-graders from Russian-language schools in the city of Almaty. The sample of students from Russia included 2247 first-graders from the city of Novosibirsk. Pearson reliability and Chronbach’s alpha were in the range from 0.89 to 0.96. Subsequently, Differential Item Functioning analysis was carried out on a combined sample in order to investigate whether the scale tasks work identically for the students from Russia and Kazakhstan when the levels of their reading skills are taken into consideration. Logic regression showed that there are no items with DIF effect size reaching beyond 0,13 (under Zumbo-Thomas classification). The research outcomes may be of interest to international comparative studies of reading skills development.

Introduction

Learning to read proficiently is one of the most important educational goals in early elementary school grades. Reading performance is an essential prerequisite necessary for student school success [1; 15]. How reading develops is one of the major themes in educational psychology. According to the simple view of reading, it consists of two parts: decoding and linguistic comprehension [18]. Much research has been focused on whether these two skills are dissociable. Correlation studies have indicated that these two skills are separable and that there can be reading comprehension problems in children with adequate decoding skills [41]. Studies in the English language have shown that the correlation between decoding and reading comprehension is higher in the early grades of elementary school than in the later grades [19]. Also, decoding skills contribute more to reading comprehension at the early grade levels than listening comprehension [23]. However, the exact developmental trajectory of this relationship is still unknown. To better understand how reading develops, it is necessary to understand how beginning readers recognize words accurately and automatically [9]. It is also important to know what factors have effects on reading development. Numerous studies have examined the factors influencing children’s reading [38; 43]. The research focused primarily on cognitive-linguistic factors such as phonological awareness, rapid automatized naming, and working memory and psychological factors such as motivation. Many variables have been identified as having a significant impact on reading such as selective attention [29], orthographic ability [42], homework activities [11], motivation [37], metacognition [31], and many others. Additionally, studies have also found the effects of working memory and processing speed on reading, especially reading comprehension [22; 32; 44].

However, two of the most studied variables concerning reading are phonological awareness (PA) and rapid automatized naming (RAN). These two variables were found to be the most important pathways to reading success [34]. PA can be defined as an awareness of the phonological segments of the speech that are closely represented by an orthography [3]. It is important to note that PA is a complex ability consisting of several components. Høien, Lundberg [17] identified three basic components of PA: 1. phoneme factor, 2. syllable factor, and 3. rhyme factor. Out of these, a phoneme factor was most strongly related to reading outcomes. The relationship between PA, especially at the phoneme level, and word decoding abilities has been firmly established [6]. PA has unanimously been identified as one of the most important predictors of reading, regardless of orthography and whether it is alphabetic or logographic language [20; 28; 39]. In a study of several European alphabetic languages (Finninsh, Hungarian, Dutch, Portuguese, and French), authors found phonological awareness to be the main factor associated with reading performance [50].

Another predictor widely examined in relation to reading was rapid automatized naming (RAN). There were some controversies regarding what RAN tasks measure [7]. On the one hand, some authors consider RAN as a part of phonological processing and defined it as the efficiency of phonological code retrieval [46]. On the other hand, research has shown that RAN is a significant independent predictor of reading [14; 49] and is thus a separable construct from PA. The argumentation behind the claim that RAN is a separable construct from phonological processing stems from the following findings: 1. RAN makes a unique, independent contribution to reading; and 2. poor readers can have RAN deficits only, PA deficits only, and RAN and PA deficits [47]. RAN predicts future reading abilities across different ages and languages [26].

One question that needs further elaboration with reference to the role and effects of PA and RAN as predictors of reading during different periods in a person’s life and across skill levels. For example, are PA and RAN equally strongly related to reading proficiency from Grade 1 to Grade 8? In a study of Finnish, which is the language with a transparent orthography, the authors found only a minor link between early phonological skills and reading at Grade 2 [36]. Similarly, RAN seems to be a more important predictor in children with a higher level of reading performance [34]. Another important question is how PA and RAN, as the most significant predictors of reading, develop in relation to children’s age.

A longitudinal study design is the best way to answer these questions. However, most longitudinal studies have been conducted in English. The question is whether the research in English is relevant to languages with more transparent orthographies [9], that is, whether the skills needed for developing reading are universal across languages [40]. Another question is the size of the relationship of cognitive and linguistic factors such as PA and RAN to decoding skills in different orthographies. One additional problem in answering this question is that studies in different languages have used different measures as proxies for different reading constructs, which makes comparisons with the English language studies only indirectly [10].

Given the importance of PA and RAN on reading achievement, we wanted to examine further the relationship of these variables with decoding skills in the Bosnian language and how they develop over the period of two years. The Bosnian language has a transparent orthography, with each letter pronounced with the same sound regardless of its position in the word. The present study aims to answer how the relationship between PA, RAN, and decoding skills change from Grade 3 to Grade 5 in the Bosnian language. In addition, we wanted to examine how well PA and RAN in Grade 3 predict decoding skills in Grade 5. There is scant research regarding the growth trajectories of these variables among children learning different alphabetic orthographies [5]. Thus, we wanted to answer the following questions:

1. Were there statistically significant improvements in PA, RAN tasks, and decoding skills from Grade 3 to Grade 5?
2. How do the relationships between decoding skills, phoneme deletion tasks, and rapid automatized naming change from Grade 3 to Grade 5?
3. What are the best predictors of decoding skills in Grade 5 assessed in Grade 3?

Method

Participants

The sample for this study consisted of 36 children (16 girls, 20 boys) who were assessed on reading variables in Grade 3 and Grade 5. The mean age of children in Grade 5 was 10.3 years (SD — 0.5 years). These children represented a subsample from a study on predictors of reading speed and comprehension in the Bosnian language [27]. According to the children’s school records, children did not have developmental disabilities or any other neurological or health condition that might influence their learning. None of the children received special educational support at school.

Procedure

We employed a longitudinal study design to assess these reading variables in the same children in Grade 3 and Grade 5. All children in this study attended the same school in Sarajevo. Teachers of the children provided consent forms to the parents. In Grade 3, there were 38 children assessed in this school. In Grade 5, we received 36 consent forms, and these children were tested. Children were tested individually in the morning hours in classrooms available at school. Individual testing lasted about 40 minutes. Testing at Grade 3 took place in November and December 2019 and at Grade 5 — in November and December 2021. The approval for this study was obtained from the Ministry of Education in Canton Sarajevo and the Ethical Board of the Faculty of Educational Sciences at the University of Sarajevo.

Measures

The same tests were used previously in a study of reading predictors in the third-grade children [27]. More specifically, we used the following tests:

1. Word reading. In this task, children were asked to read aloud a list of real words increasing in length. The result is the number of words read correctly in one minute. The test-retest reliability for this kind of task is reported to be high, above .90 [13].
2. Pseudoword reading. As in the previous task, children were asked to read aloud a list of pseudowords increasing in length. The result is the number of words read correctly in one minute. The test-retest reliability for this kind of task was reported to be .84 [8]. These first two tasks served as a measure of decoding skills as they have shown evidence of consistency and validity in predicting reading ability [8].
3. Phoneme deletion task. In this task, children were shown a list of 16 objects and were asked to name the objects without the first sound. Three demonstration items were given prior to the task. The time to name all the objects was used as a result. Faster time indicates better performance. This test measures phonological awareness skills.
4. Rapid automatized naming of Letters (RAN: Letters). This task comprises of five lowercase letters (a, d, o, p, s) that are randomly repeated 10 times in an array of five rows for a total of 50 stimulus items [48]. The psychometric properties of RAN tasks are excellent. According to the RAN manual, interscorer reliability for this task was .98, and test-retest reliability was .90 [48]. Time to name all the letters was recorded, and faster time indicates better performance.
5. Rapid automatized naming of Objects (RAN: Objects). This task comprises of five stimulus items (hand, book, dog, star, and chair) that are randomly repeated 10 times in an array of five rows for a total of 50 stimulus items [48]. Again, the psychometric properties of this task are very good, with test-retest reliability of .84, and interscorer reliability .99 [48]. The time to name all objects was recorded, and faster time indicates better performance.

Statistical analysis

We first presented descriptive data for all the tests in Grade 3 and Grade 5, along with paired t-test measures and effect sizes. We then calculated correlations between the variables for both Grade 3 and Grade 5. In addition, we converted all the variables into ranks and calculated Spearman’s correlation coefficient. As a dependent variable in the prediction models, we used word reading and pseudoword reading in Grade 5, and as predictors we used phoneme deletion task, RAN: Letters, and RAN: Objects in Grade 3. The statistical analysis was conducted using the statistical package IBM SPSS v.26. [21]. An alpha level of .05 was used for all statistical tests.

Results

Table 1 presents mean scores, paired t-tests and effect sizes for all the variables we used in this study.

As can be seen from Table 1, there has been a considerable, statistically significant improvement in all measures from Grade 3 to Grade 5.

Next, we next present correlations between all variables (Table 2).

As shown in Table 2, most of the correlations between reading variables were statistically significant. However, an unexpected finding was the lack of correlation between RAN_objects in Grade 5 and RAN_letters in Grade 3 and Grade 5. To better elucidate the stability of the relationship between the variables, we converted them into ranks and calculated Spearman’s rank correlation coefficient to assess the temporal stability of the measures. These data are shown in Table 3.

As can be seen from Table 3, all reading variables have shown temporal stability over time with word reading being the most stable reading variable.

Lastly, we wanted to evaluate the models predicting word list reading and pseudoword list reading in Grade 5 from phoneme deletion task, RAN_letters, and RAN_objects in Grade 3. The stepwise multiple regression model (backward) for word list reading is presented in Table 4, and the model for pseudoword reading is presented in Table 5. We did not present data for statistically non-significant predictors.

As can be seen from Table 4, statistically significant predictors of reading words at Grade 5 were the RAN_objects and RAN_letters tasks, and the excluded variable was the Phoneme deletion task.

Statistically significant predictors of pseudoword reading at Grade 5 were RAN_objects and phoneme deletion tasks. RAN_letters was not a statistically significant predictor of pseudoword reading.

Table 1

Mean scores, standard deviations, paired t-test,
and Cohen’s effect size for all reading variables

Variable	Mean (SD) Grade 3	Mean (SD) Grade 5	Paired t-testc	Cohen’s effect size
Word list readinga	49.6 (13.8)	62.9 (13.9)	12.0	0.96
Pseudo word readinga	31.8 (8.1)	39.7 (7.3)	9.8	1.02
Phoneme deletionb	68.4 (20.3)	56.2 (17.7)	4.1	0.64
RAN: Lettersb	27.7 (5.9)	23.1 (3.4)	5.5	0.95
RAN: Objectsb	49.6 (8.3)	44.4 (8.3)	4.9	0.63

Note: a number of words read in one minute; b time to complete the task in seconds; c all p values less than .001.

Table 2

Correlations between all reading variables

Grades	Variables	Grade 3					Grade 5
		1	2	3	4	5	1	2	3	4	5
	1. Word list reading	1.00	0.87	-0.54	-0.54	-0.68	0.89	0.79	-0.46	-0.40	-0.60
	2. Pseudoword reading		1.00	-0.55	-0.59	-0.57	0.75	0.81	-0.47	-0.51	-0.53
Grade 3	3. Phoneme deletion task			1.00	0.44	0.36	-0.38	-0.49	0.58	0.33	0.42
	4. RAN letters				1.00	0.44	-0.54	-0.45	0.30	0.54	0.31
	5. RAN objects					1.00	-0.69	-0.56	0.39	0.38	0.71
	1. Word list reading						1.00	0.79	-0.37	-0.50	0.54
	2. Pseudoword reading							1.00	-0.56	-0-56	-0.58
Grade 5	3. Phoneme deletion task								1.00	0.37	0.47
	4. RAN letters									1.00	0.29
	5. RAN objects										1.00

Note. Bold values are not statistically significant. All other p values are statistically significant at p< .05 level except for the relationships: 1. Phoneme deletion at Grade 5 and RAN_Letters at Grade 3; 2. RAN_Objects at Grade 5 and RAN_Letters at Grade 3; and 3. RAN_Objects at Grade 5 and RAN_Letters at Grade 5.

Table 3

Spearman's rank correlation between the reading variables at two time-points

Reading Variables		Spearman's Correlation
Word reading Grade 3	Word reading Grade 5	.90
Pseudoword reading Grade 3	Pseudoword reading Grade 5	.76
Phoneme deletion Grade 3	Phoneme deletion Grade 5	.67
RAN_letters Grade 3	RAN_letters Grade 5	.65
RAN_objects Grade 3	RAN_objects Grade 5	.71

Note. all pꞌs < .01.

Table 4

A stepwise multiple regression predicting word list reading at Grade 5 from Phoneme
deletion task, RAN_letters and RAN_objects at Grade 3

Predictors at Grade 3	B	SEB	β	t	p
Intercept	128.67	10.57	-	12.17	<.001
RAN_objects	-.95	.22	-.56	-4.3	.001
RAN_letters	-.67	.31	-.29	-2.2	.04

Note. R2 = .55 (unadjusted), R2 = .52 (adjusted).

Table 5

A stepwise multiple regression predicting pseudoword list reading at Grade 5 from Phoneme
deletion task, RAN_letters and RAN_objects at Grade 3

Predictors at Grade 3	B	SEB	β	t	p
Intercept	67.05	6.08	-	11.02	<.001
RAN_objects	-.39	.13	-.44	-3.1	.001
Phoneme_deletion	-.12	.05	-.33	-2.3	.03

Note. R2 = .42 (unadjusted), R2 = .37 (adjusted).

Figure 1. Item and Person map. Russia

Figure 2. Item and Person map. Kazakhstan

Discussion

The present paper aimed to examine the developmental trends of decoding abilities in the Bosnian language from Grade 3, a phase of alphabetic readers, to Grade 5, a phase of intermediate readers. This study showed a large improvement in decoding skills from Grade 3 to Grade 5, for both, word reading and pseudoword reading. We also found a large improvement in RAN tasks and in phoneme deletion task. These findings indicate significant improvements in decoding skills happen between 3rd and 5th Grades. In addition to these findings, we have also found great temporal stability in word reading from Grade 3 to Grade 5. Children who were good readers in Grade 3, were also good readers in Grade 5. In the same line, children who were poor readers in Grade 3 were also poor readers in Grade 5.

However, the question remains when this developmental trajectory in decoding abilities reaches its plateau. Previous studies in the English language have reported that the growth trajectory in oral reading fluency sharply increases from Grade 3 to Grade 5 and only a slight increase from Grade 5 to Grade 6 [35]. We have also found the same trend from Grade 3 to Grade 5 in the Bosnian language. However, the increase from Grade 5 to higher grades has yet to be examined in the Bosnian language. The findings of our study suggest that decoding skills in Grade 3 children can accurately tell us how these children will decode in Grade 5. This is very important information from the interventionist perspective so that struggling readers can be identified earlier and be provided with adequate educational support.

As for the predictors of word reading skills at Grade 5, we found that RAN of objects and letters were both significant predictors and explained 52% of the variance in the word reading scores. RAN tasks measure automaticity, a skill necessary for successful reading [30]. It appears that different RAN tasks have a different impact on reading. Our study has shown that RAN of objects had a stronger effect on reading than RAN of letters. This is in line with Meyer, Wood [30] study that showed RAN of colors and objects were better predictors of reading than RAN of numbers and letters. On the other hand, some studies showed a greater effect of alphanumeric RAN tasks than non-alphanumeric RAN task on future reading [26].

Phoneme deletion task and RAN of objects significantly affected pseudoword reading. These two predictors explained 37% of the variance in pseudoword reading task. Phoneme deletion belongs to complex phonological skills related to reading [4]. Although in this study it was not a significant predictor of word reading, other studies conducted in the Bosnian language have found phoneme deletion task to be the most important predictor of reading speed [27].

It is interesting to note a significant improvement in both word reading and pseudoword reading from Grade 3 to Grade 5. In this study, the growth of pseudoword reading was slightly larger than that of word reading. An earlier study by Caravolas [5] reported reverse findings for the Czech and Slovak languages, where the growth was larger for word than pseudoword reading efficiency. However, it is important to note that children in that study were younger (Grades 1 and 2) than the children in our study. Another finding in this study was the high correlation between word reading and pseudoword reading at both times. In Grade 3, the correlation was slightly higher between word reading and pseudoword (r = .87) reading than in Grade 5 (r = .79). This finding is similar to findings of van Setten, Hakvoort [45], in which the authors found a strong association between reading fluency in Grade 3 and reading in Grade 6.

This study pointed to the large improvements in reading variables in children aged 8 to 10 years, and it also showed the importance of PA and RAN as predictors of reading. As demonstrated by previous studies, it seems that models of early literacy development are very similar across different languages and orthographies [2; 12].

It is worth mentioning that the reading predictors we used in this study are susceptible to training. This is especially relevant as the research suggests that decoding skills should be one of the main targets in intervening with poor readers [25]. RAN can be significantly enhanced with training [49]. Similarly, computerized remedial training has shown great potential in improving phonological awareness [33]. Children at risk of phonological difficulties can greatly benefit from teacher-delivered school-based interventions [16]. Timely intervention in these domains can have a positive impact on reading.

This study is not without limitations. The first limitation is the small sample size. We assessed reading variables for only 36 students from the original sample at Grade 5, which significantly limits the generalizability of our results. Due to the small sample, the obtained results may be sample-specific. Thus the study needs to be replicated in different samples with a larger number of participants. We also did not consider some cognitive variables, such as working memory, that impact reading skills [24]. The inclusion of variables such as working memory, processing speed, and vocabulary might have created an even better model of decoding skills. Another limitation of this study is the situation with the COVID-19 pandemic, as the children were attending some portions of their education online. It is hard to tell whether this form of schooling influenced children’s reading development.

Conclusion

Word reading and pseudoword reading improved significantly from Grade 3 to Grade 5. On the one hand, RAN of objects and letters were significant predictors of word reading skills and explained 52% of the variance in word reading skills. On the other hand, RAN of objects and phoneme deletion task were significant predictors of pseudoword reading and explained 37% of the variance in pseudoword reading skills. Given that decoding abilities seem to be stable across time, it would be beneficial to assess reading abilities and reading predictors as early as possible in order to identify children at risk of reading difficulties. Understanding reading development from Grade 3 to Grade 5 can help create timely programs of reading instruction for all readers.

1. Ivanova A.E., Kardanova-Biryukova K.S. Sozdanie russkoyazychnoi versii mezhdunarodnogo instrumenta otsenivaniya rannikh navykov chteniya [Constructing a Russian-Language Version of the international early reading assessment tool]. Voprosy obrazovaniya [Educational Studies. Moscow], 2019, no. 4, pp. 93—115. DOI:10.17323/1814-9545-2019-4-93-115
2. Ivanova A.E., Kardanova E.Yu. Izuchenie vozmozhnosti provedeniya mezhstranovogo sravnitel’nogo issledovaniya navyka chteniya u uchashchikhsya na vkhode v shkolu [Checking the Possibility of an International Comparative Study of Reading Literacy Assessment for Children Starting School]. Voprosy obrazovaniya [Educational Studies. Moscow], 2020, no. 4, pp. 8—36. DOI:10.17323/1814-9545-2020-4-8-36
3. Natsional’nyi doklad o sostoyanii i razvitii sistemy obrazovaniya Respubliki Kazakhstan (po itogam 2018 goda) [National report on the state and development of education systems of the Republic of Kazakhstan (based on the results of 2018)]. M. Atanaeva, M. Amanғazy, G. Nogaĭbaeva, A. Akhmetzhanova, M. Shakenova, G. Karbaeva, Zh. Dzhumabaeva, N. Kasymbekova, M. Dauliev, D. Abdrasheva, A. Kusidenova. Nur-Sultan: MON RK, AO «Informatsionno-analiticheskii tsentr», 2019, pp. 40—41.
4. Nurlanov E.B., Baigulova A.D., Kartpaev B.A., Amangazy M., Sabyruly E., Nogaibaeva G.A. «Rezul’taty Kazakhstana v PIRLS-2016»: natsional’nyi otchet [“Results of Kazakhstan in PIRLS-2016”: national report]. Astana: AO «Informatsionno-analiticheskii tsentr», 2018. 155 p.
5. Avvisati F., Echazarra A., Givord P., Schwabe M. PISA Country note. Kazhakstan [Elektronnyi resurs]. OECD Publishing, 2019. URL: https://www.oecd.org/pisa/publications/PISA2018_CN_KAZ.pdf (Accessed 07.09.2021).
6. Bartholo T.L., Koslinski M.C., Costa M.D., Barcellos T. What do children know upon entry to pre-school in Rio de Janeiro? Ensaio: Avaliação e Políticas Públicas em Educação, 2020. Vol. 28. Issue 107, pp. 292—313. DOI:10.1590/S0104-40362019002702071
7. Belzak W.C.M. Testing differential item functioning in small samples. Multivariate behavioral research, 2020. Vol. 55. Issue 5, pp. 722—747. DOI:10.1080/00273171.2019.1671162
8. Bond T.G., Yan Z., Heene M. Applying the Rasch Model: Fundamental Measurement in the Human Sciences (4th ed.). New York: Routledge, 2020. 376 p. DOI:10.4324/9780429030499
9. Cheema J.R. Cross-country gender DIF in PISA science literacy items. European Journal of Developmental Psychology, 2019. Vol. 16, Issue 2, pp. 152—166. DOI:10.1080/17405629.2017.1358607
10. Ercikan K., Guo H., He Q. Use of Response Process Data to Inform Group Comparisons and Fairness Research. Educational Assessment, 2020. Vol. 25, Issue 3, pp. 179—197. DOI:10.1080/10627197.2020.1804353
11. Fahrmeier E.D. The effect of school attendance on intellectual development in northern Nigeria. Child Development, 1975. Vol. 46, pp. 281—285.
12. Goodrich S., Ercikan K. Measurement Comparability of Reading in the English and French Canadian Populations: Special Case of the 2011 Progress in International Reading Literacy Study. Frontiers in Education, 2019. Vol. 4, Issue 120. URL: https://www.frontiersin.org/articles/10.3389/feduc.2019.00120/full (Accessed 02.11.2021). DOI:10.3389/feduc.2019.00120
13. Grisay A., Gonzalez E., Monseur C. Equivalence of item difficulties across national versions of the PIRLS and PISA reading assessments. In M. von Davier & D. Hastedt (Eds.). IERI monograph series: Issues and methodologies in large scale assessments, 2009. Vol. 2.
14. He J., van de Vijer F. Bias and equivalence in cross-cultural research. Online readings in psychology and culture, 2012. Vol. 2, Issue 2.
15. Ivanova A., Kardanova E., Merrell C., Tymss P., Hawker D. Checking the possibility of equating a mathematics assessment between Russia, Scotland and England for children starting school. Assessment in Education: Principles, Policy and Practice, 2018. Vol. 25, Issue 2, pp. 141—159. DOI:10.1080/0969594X.2016.1231110
16. Magis D., Beland S., Raiche G. Collection of Methods to Detect Dichotomous Differential Item Functioning (DIF). Package ‘difR’. 2020. URL: http://cran.rediris.es/web/packages/difR/difR.pdf (Accessed 01.12.2021).
17. Marsella A.J., Yamada A.M. Culture and mental health: An introduction and overview of foundations, concepts, and issues. In I. Cuéllar & F.A. Paniagua (Eds.). Handbook of multicultural mental health. Academic Press, 2000, pp. 3—24. DOI:10.1016/B978-012199370-2/50002-X
18. Mullis I.V.S., Martin M.O., Sainsbury M. PIRLS 2016 reading framework. PIRLS, 2016, pp. 11—29. URL: https://timss.bc.edu/pirls2016/downloads/P16_FW_Chap1.pdf (Accessed 10.11.2021).
19. Niklas F., Schneider W. Home literacy environment and the beginning of reading and spelling. Contemporary Educational Psychology, 2013. Vol. 38, Issue 1, pp. 40—50. DOI:10.1016/j.cedpsych.2012.10.001
20. Oliveri M.E., Ercikan K., Zumbo B. Analysis of sources of latent class differential item functioning in international assessments. International Journal of Testing, 2013. Vol. 13, Issue 3, pp. 272—293. DOI:10.1080/15305058.2012.738266
21. Oliveri M.E., von Davier M. Toward increasing fairness in score scale calibrations employed in international large-scale assessments. International Journal of Testing, 2014. Vol. 14, Issue 1, pp. 1—21. DOI:10.1080/15305058.2013.825265
22. Sarmurzin Y., Amanzhol N., Toleubayeva K. et al. The impact of OECD research on the education system of Kazakhstan. Asia Pacific Education Review, 2021. Vol. 22, pp. 757—766. DOI:10.1007/s12564-021-09715-8
23. Schleicher A. PISA 2018: Insights and Interpretations [Elektronnyi resurs]. OECD Publishing, 2019. URL: https://eric.ed.gov/?id=ED601150 (Accessed 07.07.2021).
24. Stubbe T.C. How do different versions of a test instrument function in a single language? A DIF analysis of the PIRLS 2006 German assessments. Educational Research and Evaluation, 2011. Vol. 17, Issue 6, pp. 465—481. DOI:10.1080/13803611.2011.630560
25. Tymms P. Baseline assessment, value-added and the prediction of reading. Journal of Research in Reading, 1999. Vol. 22, Issue 1, pp. 27—36.
26. Van de Vijver F.J.R., Poortinga Y.H. Towards an integrated analysis of bias in cross-cultural assessment. European journal of psychological assessment, 1997. Vol. 13, Issue 1, pp. 29—37.
27. Wildy H., Styles I. Measuring what students entering school know and can do: PIPS Australia 2006-2007. Australian Journal of Early Childhood, 2008. Vol. 33, Issue 4, pp. 43—52. DOI:10.1177/183693910803300407
28. Woitschach P., Zumbo B.D., Fernández Alonso R. An ecological view of measurement: Focus on multilevel model explanation of differential item functioning. Psicothema, 2019. Vol. 31, pp. 194—203. DOI:10.7334/psicothema2018.303
29. Zumbo B.D. A Handbook on the Theory and Methods of Differential Item Functioning (DIF): Logistic Regression Modeling as a Unitary Framework for Binary and Likert-Type (Ordinal) Item Scores. Ottawa, ON: Directorate of Human Resources Research and Evaluation, Department of National Defense, 1999. URL: https://faculty.educ.ubc.ca/zumbo/DIF/handbook.pdf (Accessed 01.12.2021).