Productive Action In Online Learning Of Data And Media Literacy

The article examines the adolescents' potential for productive action at various stages of the data research cycle. The hypothesis was that the technically intricate phases of data research cycle, which require mathematical and computational skills can be performed by students at a reproductive level following the patterns, whereas the stages requiring data understanding and research design, can be executed creatively and productively. The hypothesis was tested during the online bootcamp aiming to enhance media and data literacy among 8-11 grade students. 53 students aged 14 to 18 from 26 Russian cities took part in the research. Throughout the course students examined textual socio-humanitarian data in geographically distributed teams. Their learning outcomes were compared to those obtained earlier from similar bootcamps on technical and engineering data. Contrary to widespread belief, the main challenge the school students face while learning the basics of data science and machine learning is not the complexities of programming or Math statistics. When dealing with the socio-humanitarian object of research, students successfully coped with computational tasks, but they encountered challenges producing the research design and interpreting results. The study shows that the development of the students' competencies in the basics of scientific research methodology should be considered as a necessary and critical component of educational programs that involve data inquiry. The findings of this study were used for the development of a competency model of data literacy.

Introduction

While the theoretical foundations of media literacy and associated academic and public initiatives have existed for roughly half a century, data literacy is a more recent concept. Since the 1970s, media pedagogy within the enlightenment and critical theory traditions has prioritised cultivating a critically thinking audience resistant to manipulative media influences and misleading representations. However, alongside the critical recipient, the active role of the critical media producer is gaining increasing importance. The digital age has democratised media production, with digital tools empowering anyone for public self-expression [19]. This presents both novel opportunities and challenges for society, politics, and the education sector [20].

Furthermore, the past three decades of information technology development have seen data in various formats become the cornerstone of all digital media. In the context of pervasive datafication [22], the ability to adopt a reflexive and ethical approach towards everyday data circulation is crucial [9]. This fosters the development of individuals capable of making informed decisions, ethical choices, and independently forming opinions [25]. Consequently, data literacy stands as a vital component within the modern competency framework, alongside media, information, and scientific literacy [13]. Data literacy encompasses the computational skills necessary for quantitative data analysis [5], the ability to comprehend and critically evaluate information derived from data [28], and the skill to communicate data analysis results to others [16].

Fotopoulou [15] sees similarities between media and data literacy, arguing that both reading media and data are social and cultural practices influenced by their recipients’ social context. Both media discourse and published data [8] can reflect the agendas of their authors or curators [11], as exemplified by healthcare data or electoral statistics. Critical theory underpins the work of many data literacy researchers [29]. In education, this approach emphasises the importance of ethical data use [6] and the dissemination of "critical data literacy" [23] across diverse social groups. Examples include schoolchildren [7], journalists [18], non-profit organisations, and civil activists [15] who employ "citizen data science" practices. These critical practices are demonstrably gaining increased relevance as the discourse of "data-driven management" [26] becomes pervasive within corporate, educational, and government institutions, often used to legitimise managerial and political decisions.

A rapidly evolving domain at the intersection of media and data is text analysis through natural language processing methods. This was a central theme in the educational bootcamps "Data Campus. Media" conducted by the author between 2021 and 2023.

Engaging students in analytical practices fosters their agency, which occurs when they transition from reproductive to productive activity. This shift is achieved through cultivating an active "producer" role, moving away from traditional educational approaches that focus on training students to reproduce predetermined "correct" results [17]. This stance aligns perfectly with the paradigm of open education [2] with the pivotal task-activity approach as developed in the learning theory of V.V. Davydov [1] and the developmental psychology of D.B. Elkonin [3]. The notion of Productive Action developed by Boris Elkonin in line with the above-mentioned theories and Vygotsky's cultural-historical approach describes the phenomenological and existential structure of the "act of development". A productive action, according to B. Elkonin, is "a coherence of two events: the event of overcoming the inertia of past experience and the event of others' affirmation of a new space of possibilities." Thus, productive action has two stages: first, the creation of a "product"; and second, its inclusion in the life world of others through presentation and generation of a new "semantic field" that enables both author and audience to think and act in new ways. The first stage of productive action involves the event of overcoming the pre-determined (by experience, skills, others) way of thinking and acting and often a negation of something previously accepted by the actor (a stereotypical belief, an algorithm of action, a tradition, etc.). An important characteristic of productive action is that the problem to be solved is formulated by the actor himself and does not presuppose a "ready-made" task with a correct answer. The second stage of productive action is the public presentation of the created product and the trial of its significance and relevance for others. Their recognition of the product (which may or may not happen) is the evidence that the product is capable of generating new meanings and ways of action. The “publication” of the author’s work is a risky existential act, since the author's very Self is at the stake here, Boris Elkonin emphasizes. Yet, "only here does the personality enters the World, attests itself and therefore only here does the Personality becomes a real fact of Being" [4, p. 121].

In theory, in an open learning environment students positioned as "data producers" are able to deal with data without direct instructions on what to do with it, showcasing their analytical and computational competencies. They are able to suggest a hypothesis or a vision of a product, set tasks, craft a data and/or media product and present their outcomes, embodying "productive action” [4]. This approach necessitates developing competencies across data research phases, including goal definition, exploratory analysis, data processing, model evaluation [10] and presenting the results of work to others [21].

A common professional concern regarding Data Science education for K-12 students centres around the perceived necessity of advanced mathematical knowledge (statistics, programming, probability theory, algebra) for truly productive analytical work with data. These topics often fall outside the scope of the school curriculum. However, we argue that such requirements are not absolute, but depend on what we consider to be a productive result.

For K-12 students engaged in a trial of Data Science, we define productivity not in terms of achieving technical perfection in data mining, which necessitates deep and diverse mathematical knowledge. Instead, we align with Boris Elkonin's concept of Productive Action [4]. Here, we consider that utilizing data – a new medium – and its associated analysis tools empowers students to transcend their usual information processing methods and create a novel space of actional possibilities [4, p. 118]. For instance, within this framework, a student could initially grasp the concept of correlation as a practical tool with defined application rules for solving analytical problems. The underlying mathematical aspects can then be addressed at a deeper level in subsequent mathematics education.

The research hypothesis was as follows: (1) for truly productive activity at the stages of data preparation, modelling and evaluation, students need special knowledge of mathematical statistics, probability theory and linear algebra, however these tasks can be solved at the reproductive level (action according to the pattern); (2) the stages of goal understanding, initial data exploration and conceptual development of the research can be fully implemented in a productive way on the basis of existing knowledge and thinking skills.

Methods

The research method is a case study of the online bootcamp "Data Campus. Media", organized in the task-activity approach with the following principles of pedagogical design [2]:

1. the educational event involves problem-based learning;
2. the event is characterized by situational uncertainty, when neither students nor teachers have a standard "correct" solution of the problem;
3. the multiplicity of possible options for students’ self-determination and their educational paths, ensuing from the multiplicity of contexts in which their social life is happening;
4. encouragement of students' autonomous goal-setting and its support including special pedagogical techniques.

The duration of the program was 70 academic hours. Its main topics were "Media Metaphors", "Introduction to Media Studies", "Introduction to Natural Language Processing and Data Research Methodology" and "Python Programming for Natural Language Processing". The learning formats were lectures, master classes, team projects and presentations of the students' analytical developments. When registering for the program, the level of proficiency in Python programming language was tested for subsequent balancing of the project teams according to this parameter. In addition, students filled out an interest survey, the results of which were taken into account when forming the project teams. 53 students of grades 8-11 from 26 Russian settlements aged 14 to 18 years took part in the study. All activities including collaborative code development were organized via cloud services. The participants worked on their projects in geographically distributed teams, communicating via video and text chats using desktop and mobile devices. All datasets, course materials, testing and other materials were available in the cloud learning management system.

Students were offered the following datasets:

1. Metadata of literary texts: 5477 annotations of literary works, including their title, age category, genre, and authors.
2. News feed of an information agency: 360,000 regional news messages dated from 2009 to 2019.
3. Diary entries: several hundred thousand diary texts (predominantly from the 20th century).
4. Film metadata: 250 titles with annotations, ratings, year, country, director, screenwriter, and actors.
5. Song lyrics: 82,452 texts from the Spotify database, along with their artists, genre, release date, and over 20 musical characteristics.

The students had to set a research task for themselves, following the instruction: "Each team must formulate a reasoned hypothesis. This could involve identifying connections in the data, patterns, trends, or anything else. The provided data may be modified or supplemented with any kind of additional data. Validate or refute your hypothesis using text data processing tools and interpret the results of your research." Throughout the project, students were provided with mentorship and consultative support from experts.

To assess team productivity and to monitor the project progress, the following milestones were established:

1. A team was formed, online teamwork tools were set up, and a project microsite was created.
2. Data was tokenized and cleaned.
3. Hypotheses were formulated, and a research plan was established.
4. Lemmatization and modelling were conducted.
5. Analysis and visualization were completed.
6. Results were interpreted.
7. Code cleaned, the results were submitted for expert evaluation.

The assessment of the projects was based on expert evaluations by teachers of Media Studies and Data Science according to the following criteria: 1) clarity of research goals and tasks, understanding of applied concepts, hypothesis formulation and operationalization; 2) quality of interpretation of the results; 3) quality of data preparation and analysis; 4) quality of modelling.

Results

The program resulted in student projects on digital humanities, as shown in Table 1.

Table 1. Student data projects and their evaluation

Note: 1 — team project topic; 2 — number of team members; 3 — team's average knowledge of Python programming language (max. 13); 4 - team performance on milestones; 5 - evaluation of project conceptualisation (understanding and formulation of goals, objectives, concepts, interpretation of results); 6 — evaluation of data work (quality of data processing, analysis, modelling).

Teamwork facilitated students' progression from reproductive to productive action with data within the open educational environment. Supported by teachers and experts, many participants, equipped with a dataset, were able to independently formulate a research question, adopt the position of "data producers," and create an output. However, this study also revealed competence deficiencies in some participants, particularly related to analytical and creative engagement with interdisciplinary issues.

The hypothesis that the technical aspects of the project (data preparation, modelling, and evaluation phases) could be performed at a reproductive level was largely confirmed. While students with insufficient programming knowledge faced challenges in computationally-intensive parts of the research cycle, teamwork mitigated these difficulties through the presence of "programmers" within groups and expert support.

The second hypothesis – that students without specialised training are capable of productive action at the conceptual development stage – received partial confirmation. Team productivity was demonstrably influenced by factors such as dataset size, the ability to construct multi-variable hypotheses, analyse interdisciplinary problems, and activate and apply relevant school curriculum knowledge. In this regard, several teams encountered difficulties, ranging from initial project development – including goal setting, preliminary data analysis, and hypothesis operationalisation – to later stages of interpretation and presentation. Nonetheless, with appropriate expert intervention, most teams successfully navigated these challenges. Notably, two projects (those utilising news and literary texts) achieved a level of performance exceeding expectations for K-12 students.

Consider, for example, a project investigating the representation of Russian regions within the federal news agenda. The dataset comprised approximately 300,000 texts from a federal news agency, spanning the period 2009-2019. The research team consisted of seven students from various Russian cities, with only four actively contributing to the project. The team leader, a 14-year-old student from Surgut, guided the project's direction. The team defined its goals, selected an appropriate topic modelling algorithm, and obtained a set number of interpretable themes (ranging from official activities to protest rallies). They subsequently analysed the temporal dynamics of these themes and their geographical distribution. Additionally, regional topics were plotted on an interactive map, highlighting the most prominent themes for each region, effectively creating a geoinformatic product.

The interpretation of these qualitative results prompted the students to pose critical questions regarding the representation of their own regions in the news. Framed within Boris Elkonin's concept of productive action [4], the product generated by the team and presented at a later stage created a novel "semantic field" for both the researchers and their audience. This new perspective challenged previously held assumptions and facilitated critical questioning of the media's portrayal of their regions. The students began asking themselves questions about the predominance of state officials and law enforcement agencies in the news and the underrepresentation of cultural and public organizations; about why so disproportionate attention is paid to emergencies and criminal incidents in the provinces? Does the participants’ subjective perception of their territory coincide with its data-based media model, and what is the reason for possible discrepancies – the quality of data processing or the media bias? Thus, the result of the second stage of the productive action became the subjects’ change of their interpretive framework, ultimately their local ontological model that answers the question  “How the world really works?”

Discussion

Our research suggests that the most significant challenge students face when engaging with data research practices lies not primarily in the technical aspects, but rather in the conceptual dimension. In light of these findings, it seems crucial to shift the emphasis within educational discourse on data literacy and data science education. The current focus on including various components in curricula should be reframed to acknowledge the critical role of students' scientific competencies. While some advocate for increased programming and IT integration into statistics and data handling instruction [24], others emphasise the need for curricula to focus on civic responsibility and data ethics [27]. While both these components are undoubtedly important, our observations in educational practice highlight the fundamental significance of scientific competence. Essential for meaningful data research, this competence encompasses skills such as differentiating between the known and the unknown, identifying and comprehending research-relevant concepts, applying logical reasoning, analysing and contrasting observations, integrating knowledge from diverse disciplines, formulating and testing valid hypotheses. This cluster of competencies is indispensable for "understanding data." We concur with researchers such as W. Finzer [14], A. Cuoco [12] and others, who insist that students need to be accustomed to viewing the world through the lens of data. From our perspective, the ability to question and solve problems using data is unattainable without a foundational understanding of the scientific method and scientific thinking. It is these competencies, rather than technical data handling skills, that we believe hold paramount importance.

Turning to technical skills, our study found that activities related to applying data processing methods (coding variables, data type correction, lemmatization etc.) presented fewer difficulties, particularly for teams with members possessing strong programming skills. This can likely be attributed to the largely reproductive nature of the technical work. Code snippets and examples readily available online and in supplied study materials facilitated software code implementation.

For example, configuring and tuning a machine learning model, even for students unfamiliar with its underlying mathematical principles, can be considered a reproductive skill. As observed, a proficient programmer can readily copy and minimally modify code snippets for their specific task, a common practice among contemporary programmers. Subsequently, they can achieve acceptable results through trial-and-error parameter adjustments on the model, without necessarily delving into the mathematical underpinnings. This clearly demonstrates good reproductive skills, rather than true productive action.

Our experience in implementing similar educational programs in both STEM and humanities suggests that students encounter less difficulty with productive action when tackling engineering or technical problems (e.g. binary image classification). Conversely, greater challenges emerge when modelling sociocultural phenomena (e.g., "human capital" or "poverty") or analysing textual data. In the former case, defining project goals and tasks, comprehending the data's categorical structure, and identifying additional data needs are less problematic for students. However, the latter scenario may present difficulties as early as exploratory analysis. Students might struggle to "grasp" the object, formulate a conceptual definition, identify its essential characteristics and their interdisciplinary correlations, develop hypotheses, and select appropriate data.

At the same time, research focused on socio-humanitarian and socio-economic objects aligns well with the leading activity and social situation of development in adolescents and young people (namely, active social participation, self-determination regarding values, interests and careers, ethics and citizenship) [3]. By engaging students in data research on such topics, we can foster a deeper understanding of the relationships between their constructed models of social reality, themselves, and society as a whole. Additionally, the inherent interdisciplinary nature of socio-humanitarian themes encourages the development of students' complex knowledge and skillsets applicable across various spheres of their future academic and professional endeavours.

Comparing student attitudes towards socio-humanitarian and engineering-technical tasks and datasets, we argue that data literacy education should prioritize datasets that enable modelling of sociocultural objects embodying meanings, values, and practices shaped by social interactions and cultural and economic contexts. Within an open educational environment, interacting with such data necessitates reflection on both the socio-political implications of knowledge production and dissemination through data, and on students' own relationship to these processes. This fosters the ability to critically evaluate these processes, formulate personally significant data research topics and projects, and present them to a public audience.

The organisation of open media and data literacy education should consider the social aspects of scientific practice. This allows for a more complex modelling of the educational program aimed at the concurrent development of technical, disciplinary, and conceptual skills. These three components mirror the research practice model proposed by A. Pickering [30]. Our study suggests that the most deficient aspect of scientific practice and knowledge production amongst K-12 students lies not in the "disciplinary" (mastering data analysis methods) or "technical" aspects (overcoming the challenges of unfamiliar data and learning programming tools), but in the "conceptual" aspect. This aspect concerns the scientific concepts, ideas, theories, and models that researchers use creatively to interpret data. It aligns with the student's subjectivity ("human agency" in Pickering's terms), which refers to the ability to make independent decisions about the design and course of their research. This deficiency is not solely attributable to a student's previous educational background, but is demonstrably influenced by contextual and sociocultural factors. Consequently, this area presents a critical point of application for pedagogical efforts and techniques aimed at fostering student agency and developing agency, enabling them to be proactive and independent in setting and achieving goals.

Conclusion

Our hypothesis that students could perform the technical aspects of the project at a reproductive level without in-depth mathematical and statistical knowledge was confirmed. However, the hypothesis that students without specialised training could achieve productive action at the conceptual development stage received only partial support. While two projects unexpectedly exhibited high levels of productive action across both stages, most participants encountered difficulties in identifying and comprehending research-relevant concepts, distinguishing the known from the unknown, applying logical reasoning, and formulating hypotheses.

These observations lead us to conclude that the primary obstacle to truly productive analytical work with data for K-12 students is not a lack of specific mathematical statistics and programming knowledge, but rather a deficiency in core scientific research methodology competencies. This includes the ability to utilise theoretical concepts as instruments for epistemic practice. This predicament is also linked to student subjectivity, defined here as the ability to make independent decisions about the content of their research practice. We posit that this aspect should be the primary focus of both the pedagogical design of educational programs and their associated psychological and pedagogical support.

Furthermore, our findings suggest that socio-humanitarian content within analytical tasks appears to be more universally applicable for early adolescence compared to engineering and technical content. This alignment stems from both the socio-cognitive characteristics of this age group and the potential to establish a valuable interdisciplinary foundation for future professionals across a broad range of in-demand careers.

1. Davydov V.V. Teoriya razvivayushchego obucheniya [Theory of developmental learning]. Moscow: OMC INTOR, 1996. 544 p. (In Russ.).
2. Popov A.A., Ermakov S.V. Didaktika otkrytogo obrazovaniya [Didactics of open education] (3rd ed.). Moscow: NKC, 2020. 352 p. (In Russ.).
3. Elkonin D.B. Psikhicheskoe razvitie v detskikh vozrastakh. Izbrannye Trudy [Mental development in childhood. Selected works] (3rd ed.). Moscow: MPSI, 2001. 416 p. (In Russ.).
4. Elkonin B.D. Productive Action. Кul'turno-istoricheskaya psikhologiya = Cultural-historical psychology, 2019. Vol. 15, no. 1, pp. 116—122. DOI: 10.17759/chp.2019150112 (In Russ.).
5. Bargagliotti A. et al. Pre-K-12 guidelines for assessment and instruction in statistics education II (GAISE II) (2nd ed.). American Statistical Association: 2020.
6. Baumer B.S., Garcia R.L., Kim A.Y. et al. Integrating data science ethics into an undergraduate major: A case study. Journal of Statistics and Data Science Education, 2022. Vol. 30, no. 1, pp. 15-28. DOI: 10.48550/arXiv.2001.07649
7. Bryant C. et al. A middle-school camp emphasizing data science and computing for social good. SIGCSE 2019 Proceedings of the 50th ACM Technical Symposium on Computer Science Education. New York, NY: Association for Computing Machinery, pp. 358—364. DOI: 10.1145/3287324.3287510
8. Carmi E., Yates S.J., Lockley E., Pawluczuk, A. Data citizenship: rethinking data literacy in the age of disinformation, misinformation, and malinformation. Internet Policy Review, 2020. Vol. 9, no. 2. DOI: 10.14763/2020.2.1481
9. Cope B., Kalantzis M., Searsmith D. Artificial intelligence for education: Knowledge and its assessment in AI-enabled learning ecologies. Educational Philosophy and Theory, 2020. Vol. 53, no. 12, pp. 1229-1245. DOI: 10.1080/00131857.2020.1728732
10. Cross-industry standard process for data mining. Wikipedia. Available at: https://en.wikipedia.org/wiki/Cross-industry_standard_process_for_data_mining (Accessed 01.08.2023).
11. Crawford K. The hidden biases in big data. Harvard Business Review Blog Network, 2013, April 1. Available at: http://blogs.hbr.org/2013/04/the-hidden-biases-in-big-data/ (Accessed 10.09.2023).
12. Cuoco A., Goldenberg E. P., Mark J. Habits of Mind: an organizing principle for mathematics curriculum. Journal of Mathematical Behavior, 1997. Vol. 15, no. 4, pp. 375-402.
13. Data-Pop Alliance and Internews. Beyond Data Literacy: Reinventing Community Engagement and Empowerment in the Age of Data. Data-Pop Alliance White Paper Series. Available at: https://datapopalliance.org/wp-content/uploads/2015/11/Beyond-Data-Literacy-2015.pdf (Accessed 23.09.2023).
14. Finzer W. The data science education dilemma. Technology Innovations in Statistics Education, 2013. Vol. 7, no. 2. DOI: 10.5070/T572013891
15. Fotopoulou A. Conceptualising critical data literacies for civil society organisations: agency, care, and social responsibility dilemma. Information Communication and Society, 2021. Vol. 24, no.11, pp. 1640-1657. DOI:10.1080/1369118X.2020.1716041
16. Gibson J. P., Mourad T. The growing importance of data literacy in life science education. American Journal of Botany, 2018. Vol. 105, no. 12, pp. 1—4.
17. Hardy L., Dixon C., Hsi S. From Data Collectors to Data Producers: Shifting Students’ Relationship to Data. Journal of the Learning Sciences, 2020. Vol. 29, no. 1, pp. 104—126.
18. Hewett J. Learning to teach data journalism: Innovation, influence and constraints. Journalism, 2016. Vol. 17, no. 1, pp. 119—137.
19. Knaus T. Technology criticism and data literacy: The case for an augmented understanding of media literacy. Journal of Media Literacy Education, 2020. Vol. 12, no. 3, pp. 6-16. DOI: 10.23860/JMLE-2020-12-3-2
20. Knaus T. Making in media education: An activity-oriented approach to digital literacy. Journal of Media Literacy Education, 2022. Vol. 14, no. 3, pp. 53-65. DOI: 10.23860/JMLE-2022-14-3-5
21. Lee H.S, Mojica G.F., Thrasher M., Baumgartner P. Investigating data like a data scientist: key practices and processes. Statistics Education Research Journal, 2022. Vol. 21, no. 2, Article 3. DOI:10.52041/serj.v21i2.41
22. Mejias U.A., Couldry N. Datafication. Internet Policy Review, 2019. Vol. 8, no. 4. DOI: 10.14763/2019.4.1428
23. National Academies of Sciences, Engineering, and Medicine. Data science for undergraduates: Opportunities and options. National Academies Press: 2018. Available at: http://doi.org/ 10.17226/25104 (Accessed 23.09.2023).
24. Nolan D., Temple Lang D. Computing in the statistics curricula. The American Statistician, 2010. Vol. 64, no. 2, pp. 97—107. DOI: 10.1198/tast.2010.09132
25. Pangrazio L., Selwyn N. 'Personal data literacies': A critical literacies approach to enhancing understandings of personal digital data. New Media and Society, 2019. Vol. 21, no. 2, pp. 419—437.
26. Rieder G., Simon J. Datatrust: Or, the political quest for numerical evidence and the epistemologies of Big Data. Big Data and Society, 2016. Vol. 3, no. 1. DOI:10.1177/ 2053951716649398
27. Schanzer E. et al. Integrated Data Science for Secondary Schools: Design and Assessment of a Curriculum. Proceedings of the 53rd ACM Technical Symposium on Computer Science Education V. 1 (SIGCSE 2022). New York, NY: 2022. Pp. 22-28. DOI:10.1145/3478431.3499311
28. Schield M. Information literacy, statistical literacy and data literacy. IASSIST Quarterly, 2018. Vol. 28, no. 2, pp. 6—11.
29. Wise A. F. Educating Data Scientists and Data Literate Citizens for a New Generation of Data. Journal of the Learning Sciences, 2020. Vol. 29, no. 1, pp. 165—181.
30. Pickering A. The Mangle of Practice. Time, Agency and Science. Chicago: Univ. of Chicago Press, 1995.