Introduction
Student project work, or project-based learning, is “a student-centred form of instruction characterized by students’ autonomy, constructive inquiry, goal-setting, collaboration, communication and reflection within real-world practices” (Kokotsaki, Menzies, Wiggins, 2016). Project-based learning is regarded as a popular innovative pedagogical approach in higher education. Within this approach, students seek solutions to non-routine problems by posing and refining questions, discussing ideas, making predictions, developing plans, conducting experiments, collecting and analysing data, drawing conclusions, publicly presenting their findings, generating new hypotheses, and creating products (Yakovleva, 2014; Li, Zhang, He, 2015; Guo et al., 2020; Mota et al., 2025).
Early conceptualizations described project-based learning along two dimensions: (1) whether the purpose of the project was based on academic topics or on “real-world” problems in a social and political context; and (2) whether control over project implementation and outcomes was centred on instructors or students (Morgan, 1983). Activities oriented more strongly toward “real-world” problems and giving students greater control were regarded as more intensive forms of project-based learning. Thus, even early conceptualizations assumed that a learning task could vary in its degree of “projectness”: in other words, it could approximate authentic professional tasks to a greater or lesser extent and could be controlled by students to a greater or lesser extent.
The need to operationalize the notion of “real-world” tasks led to an expansion of the list of characteristics associated with project work. The key features of project-based learning are now considered to include a driving question, the creation of a product, inquiry activity, the need to apply soft skills (critical thinking, communication, and collaboration) and technological skills, the central role of feedback, mandatory public presentation of the project, and high student autonomy (Larmer, Mergendoller, Seven, 2010; Guo et al., 2020; Al-Busaidi, Al-Seyabi, 2021; Husin et al., 2025).
The value of project-based learning lies in shifting the emphasis from mastering theoretical knowledge and discrete abilities/skills toward obtaining a practical result through the application of this knowledge and these abilities/skills. Project outcomes should be obtained by addressing meaningful tasks; moreover, the external outcome of a project should be applicable in practice, while the internal outcome, or experience of activity, becomes part of students’ own resources and part of their system of values and competencies, integrating knowledge and skills (Khalilov, 2012).
The effectiveness of project-based learning in the educational process has been demonstrated across different levels of education and in different countries (Kokotsaki, Menzies, Wiggins, 2016; Granado-Alcón et al., 2020). According to the Federal State Educational Standard,1 the development and implementation of projects is a universal competency that should be developed in undergraduate and master’s students during higher education. Project-based learning has numerous advantages for both instructors and students (Chi, 2023). It develops not only students’ professional competencies but also thinking skills (Husin et al., 2025), soft skills (Guo et al., 2020; Saputri, Maura, 2024), teamwork skills (Crespí, García-Ramos, Queiruga-Dios, 2022; Afzal, Tumpa, 2025), intrinsic motivation to learn (Granado-Alcón et al., 2020), educational initiative (Distenfeld, Paletskaya, 2024), autonomy (Novalia et al., 2025), engagement and satisfaction (Naseer et al., 2025), reflectivity (Kudryavtseva, 2025), self-regulation (Zarouk et al., 2020; Wu, 2024), civic-mindedness (Pevnaya, Boronina, Pochatkova, 2024), social responsibility, readiness for career development (Chistyakov et al., 2023; Domingues et al., 2025; Zaafour, Ait Hammou, Hamdanat, 2025), and adaptability.
The literature describes the process and technologies involved in implementing project-based learning. Shpeizer (2019), for example, discusses the inevitable transformation of instructor and student roles: during a project, students assume greater responsibility for the outcome than in conventional instruction, while instructors become mentors or coordinators. Factors associated with the effectiveness of project-based learning include the use of modern digital technologies, teamwork, instructor assistance and support, a balance between didactic and inquiry-based methods, and the use of formative assessment and feedback (Kokotsaki, Menzies, Wiggins, 2016). In other words, the instructor’s role as an organizer and facilitator of project-based teaching practices is recognized as important (Li, Zhang, He, 2015; Sartika, Syafryadin, Azwandi, 2022). At the same time, project-based teaching from the perspective of the instructor as the organizing agent has been studied less extensively. In project-based learning, instructors pursue instructional, educational, and developmental objectives (Akmurzieva, 2020), maintain their level of practical skills, and create opportunities to exchange experience with practitioners or other instructors (Scott, Weimann, van der Merwe, 2011). Evenddy et al. (2023) note instructors’ resistance to and lack of readiness for implementing project-based learning. In Russian universities, project work has also caused difficulties for instructors because of insufficient time for project implementation and inflexible curricula that do not allow the project-based approach to be used fully (Talanov, Berezin, 2019).
An analysis of existing instruments for assessing university instructors’ attitudes toward project-based learning identified the use of questionnaires with open-ended questions concerning their experience of implementing projects (Lasauskiene, Rauduvaite, 2015). Closed-ended questions in another study were grouped under the themes “learning environment”, “opportunities for project-based learning”, and “skill development” (Harrou, Hellalet, 2021). In Acarol (2019), a semi-structured interview with open-ended questions about the advantages, disadvantages, and intentions to use project-based learning was used to assess Turkish school teachers’ attitudes toward project work (Acarol, 2019). Questionnaires have also been used to assess college instructors’ readiness to implement project-based learning (Ealangov, Jamaludin, 2024), with instructors directly evaluating their knowledge, skills, and attitudes toward project-based learning. A similar approach was demonstrated in the study by Yang, Skelcher, and Gao, which used a questionnaire with direct responses to open-ended questions (Yang, Skelcher, Gao, 2021).
The literature review suggests that there is no generally accepted approach to studying project-based teaching practices (Boronina, Olkhovikova, Baliasov, 2024). Developing a questionnaire of university instructors’ project-based teaching practices would allow the implementation of project-based learning in Russia to be studied more thoroughly.
The aim of the present study was to examine whether the set of project-based teaching practices can be regarded as a single construct comprising indicators differentiated by their level of intensity, using items in an explicit continuum scale format. The following hypotheses were tested:
Hypothesis 1. The use of project-based teaching practices constitutes a distinct, complex competency within teaching activity. This hypothesis will be supported if statistical analysis of the questionnaire results confirms a unidimensional structure of the data.
Hypothesis 2. Instructors with moderate and extensive teaching experience will use project-based teaching practices more actively than instructors with limited experience, because many practices require both accumulated teaching experience (reflection practices and monitoring of student activity) and established relationships with potential project beneficiaries or clients. This hypothesis will be supported if differences in individual practices between groups with different levels of teaching experience are statistically significant and have a medium or large effect size.
Materials and methods
Development of the assessment instrument
The explicit continuum item format was used as the methodology for questionnaire development (Antipkina, Ivanov, Guzhelya, 2025). Explicit continuum scales consist of sets of vignettes describing characters with low, moderate, and high levels of an indicator of the measured construct. Previous studies have shown that this format helps reduce ceiling effects compared with the Likert questionnaire format, even in high-stakes assessment contexts (Antipkina, Ivanov, Guzhelya, 2025). Respondents can evaluate the intensity of an indicator in themselves in comparison with the characters by selecting a position corresponding to one of the characters or between characters arranged in order of increasing intensity. Methodologically, explicit continuum scales are based on two approaches to measurement:
1) Treating the construct as a continuum along which low, moderate, and high levels of its characteristics can be differentiated (Tay, Jebb, 2023). Historically, continuum-based scales, originally proposed by Thurstone (Thurstone, 1928), did not become widespread, although later studies demonstrated their advantages over the Likert format (Drasgow, Chernyshenko, Stark, 2010). Research over the past two decades has established continuum specification as a recommended practice in questionnaire development (Tay, Jebb, 2023) and has led to new scale formats, such as the expanded format (Kam, 2020), which demonstrates highly stable dimensionality. Continuum specification refers to an operationalization of construct indicators that makes it possible to describe behavioral characteristics corresponding to different levels of the construct.
2) Describing indicators through vignettes rather than short statements (Ludlow et al,. 2020). The vignette format is more resistant to ceiling effects (that is, respondents are less likely to choose the highest scores in self-assessment), is stable in its representation of dimensionality, and allows recognizable, realistic descriptions to be created, although it places greater demands on respondents’ reading skills (Ludlow et al., 2014; Ludlow et al., 2020).
Thus, methodologically, we conceptualize project-based teaching practices not as dichotomous discrete variables (“a particular technique is either used or not used”), but as continuous variables in which each selected indicator of the construct “project-based teaching” may be expressed with low, moderate, or high frequency or intensity. Following the literature review, the following eight indicators were selected:
Collaboration with external stakeholders. At one end of this continuum are practices involving projects conducted for, or with the participation of, representatives of commercial and governmental organizations, other university departments, and similar external participants. At the opposite end are projects based solely on instructional tasks generated by the instructor and implemented entirely within the student group.
Student engagement with external clients and/or beneficiaries. At one end of this continuum are authentic communication practices involving active student interaction with project clients or beneficiaries outside the university. The opposite end represents traditional student work within the university and the development of soft skills through interaction with other students enrolled in the course.
Authenticity of project tasks. This indicator contrasts relevant, authentic projects that “can be included in a portfolio” with traditional instructional projects (game-based or simulated).
Structured student reflection. This indicator assesses methodological competence in incorporating reflection practices into the project implementation plan, in contrast to completing project tasks without specially organized reflection exercises, as may occur, for example, during practical placements.
Degree of procedural guidance. At one end of this continuum are projects that require a separate stage of discussion about implementation because they generally have no standard or algorithmic solution. At the opposite end are projects closely controlled by the instructor and focused on applying a specific set of basic skills.
Orientation toward authentic project outputs. A product outcome refers to the integrated presentation of a project in a form that allows its results to be scaled or makes the project available to other users—for example, by presenting the results as a methodological guide or publishing them on open-access resources. At the opposite end of the continuum are projects that are not used after completion and are not made available for use by others.
Project milestones in medium- and long-term projects. One end of this continuum describes a lack of monitoring; the other describes support for student autonomy while focusing on the metacognitive skills of planning, monitoring, and evaluating students’ own activity.
Dissemination of project outcomes. At one end of this continuum is the traditional presentation of results during a project defense within the student group. At the other end are more intensive practices aimed at developing meta-skills in presenting project outcomes as a news item, a conference presentation, or through other means of communicating project outcomes to an external audience.
3) Developing descriptions of characters with low, moderate, and high levels of each of the eight indicators in instructors’ behavior, based on the methodology of scenario scales (Ludlow et al., 2020). Within each item, the descriptions of all three characters followed a uniform structure; high and low levels were described in affirmative terms rather than by negating the opposite end of the continuum. Within each item, all characters were of the same gender, while male and female character names were distributed equally across items.
4) Conducting cognitive laboratories with five university instructors, two specialists in project-based learning, and three university students. The purpose of the cognitive laboratories was to assess the completeness of construct coverage by the indicators, the clarity of the wording, and the realism of the descriptions. Item wording was revised following the cognitive laboratories.
5) Pilot testing of the scale and psychometric analysis.
An example of an item in the explicit continuum format is presented in Figure 1. The item heading contains the name of the indicator, and the three character descriptions are arranged in ascending order of the characteristics specified in the heading. Respondents may select any of the points beneath the character descriptions or between them; in the latter case, the response indicates a level of the characteristic between the two described stories. Selecting the extreme points indicates that respondents assess the characteristic in themselves as stronger or weaker than in all three stories.
The item instruction was worded as follows: “Please evaluate the extent to which your projects are oriented toward student interaction with the beneficiaries of project work.
Less than Arina
Approximately like Arina
Between Arina and Daria
Approximately like Daria
Between Daria and Tatiana
Approximately like Tatiana
More than Tatiana.”
Responses to each vignette were coded from 0 to 6, from lower to higher levels of the indicator. The instructions for the full questionnaire and the complete text of all questionnaire items are provided in the Appendix.
Sample. The sample comprised 259 instructors from Russian universities who participated in a study conducted by the Ministry of Science and Higher Education of the Russian Federation jointly with the Association of Volunteer Centers and the National Research University Higher School of Economics in the autumn and winter of 2024. Participation was based on self-selection, because only instructors who used the project-based approach in their teaching were invited to complete the questionnaire.
Data collection procedure. The survey was anonymous, untimed, and administered through Yandex Forms. Each item was presented as an image, with the stories also duplicated as text to facilitate reading when completing the questionnaire on smartphones. Previous studies have shown that presenting items as images or in an interactive format using specially programmed items produces invariant item characteristics (Antipkina, Ivanov, Guzhelya, 2025).
Analysis procedures. The questionnaire was analysed in R. Psychometric analysis was conducted using the Partial Credit Model (Masters, 1982) in the TAM package (Robitzsch, Kiefer, Wilhelm, 2023). Statistical analysis was conducted using descriptive statistics and the Mann–Whitney test.
Results
Psychometric evaluation of the instrument
The questionnaire was analysed in R. Psychometric analysis was conducted using the Partial Credit Model (Masters, 1982) in the TAM package (Robitzsch, Kiefer, Wilhelm, 2023).
Dimensionality analysis indicated the essential unidimensionality of the questionnaire: the eigenvalue of the first residual contrast was 1,6, which is below the conventional criterion of 2,0 (Smith, 2007). This indicates that all measured indicators relate to a common construct, supporting the structural validity of the instrument. Thus, the first hypothesis—that project-based teaching practices can be represented as a unidimensional set of continuum-based indicators—was supported.
Weighted Likelihood Estimation (WLE) reliability was 0,86. WLE reliability is recommended for analysing polytomous items; it also accounts for bias in person parameter estimation and does not require a prior-distribution assumption. The conventional internal consistency coefficient, Cronbach’s alpha, was 0,87. McDonald’s omega was 0,84. Together, these indices indicate high reliability of the assessment instrument.
All questionnaire items except one showed excellent fit to the model: weighted and unweighted mean-square statistics (MNSQ) and weighted and unweighted standardized statistics (ZSTD) were below the conventional criteria (1,2 for mean-square statistics and 2,0 for standardized statistics). One item concerning collaboration with external stakeholders had an unweighted statistic above 1,2 but below 1,3, which is also considered acceptable (Boone et al., 2014). These fit statistics indicate unexpected responses to this particular item—for example, respondents with a high overall score selecting low response categories.
The questionnaire scale was well targeted: the mean item difficulty matched the average level of the characteristic among instructors. Thus, the questionnaire demonstrated good psychometric properties, providing evidence of construct validity.
Figure 2 presents the distribution of total scale scores. The minimum score was 0, the maximum was 42, the mean was 22,4, and the median was 23. The distribution of total scores approximated a normal distribution.
The table presents descriptive item characteristics. The most difficult practices for respondents were “Collaboration with external stakeholders”, “Student engagement with external clients or project beneficiaries”, and “Authenticity of project tasks.” The easier items, for which instructors readily selected high levels of the indicators, were “Orientation toward authentic project outputs”, “Project milestones”, and “Dissemination of project outcomes.”
Table
Characteristics of the project-based teaching practices questionnaire items
|
Item |
Difficulty (logits) |
Mean raw score |
SD |
Pearson item-rest correlation (adjusted discrimination) |
|
Collaboration with external stakeholders |
0,21 |
2,73 |
1,78 |
0,53 |
|
Student engagement with external clients |
0,24 |
2,61 |
1,67 |
0,62 |
|
Authenticity of project tasks |
0,15 |
2,80 |
1,64 |
0,68 |
|
Structured student reflection |
0,00 |
2,95 |
1,56 |
0,64 |
|
Degree of procedural guidance |
0,07 |
3,01 |
1,65 |
0,65 |
|
Orientation toward authentic project outputs |
–0,19 |
2,67 |
1,63 |
0,53 |
|
Project milestones |
–0,05 |
3,33 |
1,92 |
0,60 |
|
Dissemination of project outcomes |
–0,07 |
3,12 |
1,65 |
0,64 |
Most inter-item correlations fell within the recommended range of 0,2–0,5 (Stefana et al., 2025), indicating that the items did not duplicate one another. Only the items concerning the use of complex skills and the organization of multiple project milestones were strongly associated (r = 0,6), which is theoretically plausible because more complex projects are difficult to implement without regular monitoring.
Statistical analysis
To test Hypothesis 2, the questionnaire included a question about teaching experience with categorical response options. Teaching experience of “less than 1 year” was reported by 108 respondents; “1 to 3 years” by 20; “4 to 10 years” by 31; “11 to 15 years” by 28; and “16 years or more” by 51. Twenty-one respondents (8%) did not answer the question. We combined these categories to obtain groups that were comparable in size but contrasting: instructors with up to 3 years of experience (128 participants) and instructors with 4 years or more (110 participants). A Mann–Whitney test showed a significant difference in total questionnaire scores between these groups (p = 0,02). The effect size was r = 0,138 (Z = 2,227), corresponding to a small effect.
Figure 3 presents a violin-box plot showing that instructors with moderate and extensive teaching experience had greater response variance and a higher median total score. Overall, responses among highly experienced instructors were more heterogeneous.
To determine what contributed to the difference between the groups, we analysed differences in responses to each scale item among instructors with different levels of experience. Only three items showed significant differences in favour of more experienced instructors: items concerning structured student reflection, project milestones, and public presentation (Mann–Whitney test, p < 0,05).
Discussion
The findings from the questionnaire on instructors’ project-based teaching practices partially corroborate previously reported evidence. For example, the items concerning collaboration with external stakeholders and student involvement in direct communication with them were the most difficult, which is consistent with students’ reports that projects are not brought to practical implementation (Vylegzhanina, 2019).
The unidimensionality of the questionnaire supports the first hypothesis. With respect to the second hypothesis, differences associated with teaching experience concerned only the more traditional pedagogical components of project-based teaching: systematically organizing monitoring of project implementation, presenting outcomes, and, less expectedly, the orientation toward authentic project outputs. No differences by teaching experience were found for the more demanding practices that are specifically characteristic of project-based teaching.
Traditional questionnaires often ask respondents to rate a statement on a Likert scale (Pan, Shankararaman, Koh, Gan, 2021), which does not always fully capture the nuances of implementing a specific practice, whereas qualitative methods are more resource-intensive (Gülbahar, Tinmaz, 2006). The explicit continuum scale format used in this study allows respondents to assess the intensity of their use of a particular practice more deliberately and enables researchers to conduct a more fine-grained analysis of relationships among different aspects of project work.
Conclusion
The empirical validation of the questionnaire on a substantial sample of Russian university instructors supports treating the construct “university instructors’ project-based teaching practices” as a distinct competency. The unidimensionality of the questionnaire data indicates that a common construct underlies the intensity of the individual practices.
The findings obtained with this questionnaire confirm trends noted in other studies concerning the predominantly internal orientation of project work in universities (Vylegzhanina, 2019), while also identifying areas requiring particular attention. For example, limited collaboration with external stakeholders and weak student interaction with real project clients/beneficiaries indicate a need to strengthen the orientation of project work toward the real needs of society and the economy. This underscores the importance of further developing mechanisms for stakeholder involvement in the educational process and promoting a practice-oriented approach to teaching.
Future research: The questionnaire could be improved by refining its indicators. This applies particularly to dividing the item “Encouraging student interaction with external clients or project beneficiaries” into one item on interaction with clients and another on interaction with beneficiaries. It is also debatable whether other university departments should be classified as external stakeholders in the item “Collaboration with external stakeholders”, because this depends substantially on the size and structure of the university and on how “other departments” are interpreted.
Limitations. The limitations of the study include the lack of criterion evidence of validity in the form of a correlation of the scale score with external criteria, as well as a relatively small sample obtained without the use of randomization.