Datu zinātība skolai

Data-driven decision making is a complex process in which data is used and interpreted within a context to transform it into information. It is this context that enables users to utilize the data and transform it into the information necessary for development of solutions (Mandinach, 2012). Ellen Mandinach points out that data-driven decision making involves systematic data acquisition, analysis, evaluation, and interpretation within the educational context. Paul Bambrick-Santoyo explains that the data-driven decision making model must provide answers to two questions: How do we know what students have learned? What will we do if the teaching process has not been sufficiently effective? Data-driven decision making is crucial not only at the level of the entire school but also at the classroom level. At the micro-level (data literacy in the classroom), more familiar terms include evidence (data)-based assessment and assessment literacy.

In Latvia, elements of data-driven decision making (DDDM) are gradually entering the education system for the development of solutions, which respond to these key questions. For example, a quality monitoring system for education^[1], which is more focused on the needs of the education system, is being introduced in the country. This edited collection of papers predominantly explores working with data at classroom, school, and municipality levels. Similarly, in other parts of the world, the data literacy of school management teams and teachers is not always sufficient. Such insufficiency can hinder broader and faster implementation of DDDM approach – primarily because the use of DDDM for the development of solutions is a complex process. Research indicates that DDDM approach can be implemented in two ways. Firstly, by adapting one of the many DDDM models for school management and creating a data utilization protocol within the school, as well as implementing a monitoring system for the introduced solutions. Secondly, school management can foster a culture of data use, and develop the data literacy skills of the personnel themselves, learning – collaboratively with teachers – to solve current issues in the teaching process.

To obtain answers to the questions formulated above, researchers propose a complex data analysis model for the development of solutions in schools, which analyses the quality of education in terms of its effectiveness.

The complex data analysis model for school development solutions is based on the systems theory model “Input-Process-Output”. Within the framework of this model, students’ performance is compared to the national average based on four criteria (average students’ performance, three-year trend in average students’ performance, average students’ performance in categories of curriculum, and average students’ performance at various cognitive levels at both classroom and school levels).

School development solutions within the framework of complex data analysis are elaborated by analysing and identifying significant correlations between students’ performance, teacher performance in the classroom, and actions of the school management. When planning the implementation of school development solutions, the school context is also considered, including the average socioeconomic status of the student population.

While constructing a model for complex data analysis, consideration was given to the analysis of factors that affect a student^,s performance. Approximately 40% (according to the data of McKinsey & Company) of student learning outcomes and performance are explained by factors related to teachers and school, indicating that the school as an organization is directly responsible for continuous development and improvement. Explaining both teacher and student factors, it is crucial to consider the dynamics of the education system and the hierarchical influence, where the classroom exists within the context of the school, and the school within the context of the local municipality and the country.

Considering the complexity of the education system, systemic resilience, and hierarchical self-organization, linear and simplified school development solutions have limited impact. In creation of comprehensive solutions for school development, both controllable and uncontrollable factors that have a significant influence on the education system are considered. Researchers have been challenged for years to identify additional factors at the school level, aside from socioeconomic status, that could explain variations in students’ performance. “Academic optimism” (Hoy et al., 2006a) is one of the constructs that help to explain differences in students’ performance within seemingly similar schools. Academic optimism consists of three categories: academic achievements as a value at school, collective effectiveness, and trust. Researchers explain that academic optimism encompasses both cognitive and affective dimensions. They believe that there is a mutual interaction among these three dimensions.

The emphasis on learning as a value (on academic performance) reflects a specific behavioural model in a school that promotes the improvement of educational achievements; this is evident in actions. All parties – students and teachers – highly appreciate the persistence and high performance (Hoy et al., 2006a).

The sources of collective effectiveness lie in recognized mastery experience, sharing best practices, and believing in achievements that can be attained by teachers as a group in their work with students By observing, modeling, and imitating the behaviour of others, information is gained about the achievements that the teachers can attain themselves, thereby influencing self-efficacy. Efficacy and trust are parts of academic optimism – collective teachers’ efficacy largely reflects the group’s beliefs, and trust includes an affective dimension, gives teachers confidence that they are effective in working with students, regardless of difficulties and challenges. This motivates teachers to reach challenging goals and persistently move forward. Collective efficacy influences academic optimism and teachers’ beliefs in their efficacy and the school’s ability as a community to affect student learning.

Trust in teachers and students allows experimentation and the introduction of innovations even in situations where success is not achieved on the first attempt, fostering collaboration between parents and teachers.

In response to the question of “How do we know what students have learned?”, researchers from the University of Latvia, Interdisciplinary Centre for Educational Innovation (UL ICEI) have developed an innovative three-dimensional model for analysing students’ performance at the school and classroom levels, based on the application and implementation of the DDDM approach through examination of global experiences. This model offers the school management team and teachers the ability to analyse the data on students’ performance across three dimensions of teaching and learning: the content of the subject, application of knowledge and skills, and cognitive depth. According to the model, a methodology has been devised to analyse students’ performance in national assessments in lower secondary education – specifically, in the following areas: Latvian language, mathematics, and natural sciences. This methodology operationalizes the DDDM process sequentially in alignment with criteria, enabling the identification of issues in the teaching process at both school and classroom levels, as well as their potential causes. The proposed methodology allows for the identification of gaps in teaching and learning content, indicating the level at which specific categories of knowledge and skills have been acquired within a class, a class group, or the entire school in comparison with the national level. Likewise, it enables the observation of whether a productive learning process dominates the classroom (for instance, students have an opportunity to analyse information, solve problems, and develop metacognitive skills), which is significant for fostering deeper learning and conceptual understanding among students, – one of the objectives of the education reform in Latvia.

The changes^[2] made in the normative documents, which come into effect beginning with the academic year 2023/24, are grounded in the theory of evidence-based assessment and introduce significant changes to the assessment practices at the classroom and school levels. Similar to international comparative studies, assessment communicates the level of evidence-based performance, i.e., a score of 9, 10 in secondary education or D (mastered in depth) in primary school (in the author’s country) indicates that the student is capable of demonstrating a complex action in a new situation, performing at a high cognitive level, or displaying high achievement.

The performance of students in a specific subject (such as mathematics) can be analysed comparatively by using the DDDM approach, not only at the classroom, school, or municipal level but also within the education system or at an international level. One of the indicators enabling comparison in this manner is the number of students with high and low performance.

Utilizing the data on student high and low performance in a specific academic subject, it is possible to identify trends at the municipal, school, and classroom levels. This can be achieved by employing additional necessary information to facilitate further planning of the educational process and provide essential support to both teachers and students, to increase the number of students with high performance and/or reduce the count of students with low performance.

International research data indicates that the basic mathematical skills of Latvian students are relatively higher in comparison with those in Europe and worldwide. However, students’ performance is weak in tasks involving high-level cognitive operations, i.e., there is a relatively small number of students with high performance. Increasing the proportion of students with high performance is one of the goals of the education system.

The UL ICEI adapted methodology involves determining the number of students with high and low performance in schools using national-level examination results. Examination results and expert analysis allow for more precise identification of task’s cognitive levels, thus defining the thresholds for high and low students’ performance in specific tasks. The data on high and low students’ performance provide a more accurate assessment of the situation in particular schools and class groups than average student evaluations.

A more precise assessment of students’ performance could be inferred, if there were a balance in the cognitive levels of tasks in the national-level examinations. A small number of tasks with high cognitive levels hinders the precise determination of students’ high-level performance in tasks, consequently affecting the count of students with high performance. By increasing the number of tasks with a high cognitive demand in national examinations, these data could be utilized to compare the dynamics of students with high performance over time. In addition to acquiring supplementary information, it is recommended to analyse students’ work to infer the strategies they employ, and their approaches to the task, among other factors.

To answer the initial question – “What will we do if the teaching process has not been sufficiently effective?”, first of all, the tools are required for data acquisition about the learning process. The methodology and tools developed and validated by UL ICEI are designed to analyse the impact of teaching quality on students’ performance, enabling teachers, school management, and the education system as a whole to obtain evidence (data) about changes in the course of the teaching process.

The established methodology and set of tools for analysing the teaching practices employed by teachers provide an opportunity to operationalize the educational goals and quality criteria outlined in educational policy documents down to the classroom level. This allows for the collection of evidence-based data on teachers’ teaching practices and their impact on students’ performance. Such data can be utilized for data-driven decision making to enhance the quality of education at the level of the education system, school, or individual teacher.

The foundation of the analysis of teachers’ teaching practices is a theoretical framework based on research-based pedagogical principles, aimed at providing students with deep learning, which is necessary for the acquisition of 21st-century skills. The teaching analysis is conducted by a previously trained expert, observing the teacher's instructional performance in practice, and comparing it to previously validated performance level descriptions.

Using the established tools, it has been determined that the enhanced instructional approach (deep learning) is integrated into classroom sessions; however, it is insufficient to fully equip students with the skills needed for the 21st-century competence. Analysing the course of teaching, qualitatively recorded differences in teaching that exist within the school framework, between schools and academic disciplines, researchers have acquired the grounds to conclude that students do not have access to uniformly high-quality teaching (Greitāns & Namsone, 2023).

The differences in teachers’ instructional practices are identified by creating Teacher Teaching Profiles (TTPs) according to a specific set of crucial criteria for achieving particular goals (such as the opportunities for students to develop understanding, learn to think, use information technologies, etc.). In the process of forming TTPs, experts apply criteria and determine the desired level of performance for each of them. By collecting data on observed teaching practices, actual performance data is included in the TTPs. The disparity between the desired and actual performance identifies the needs for professional growth, namely, the lack of knowledge and skills that need to be prioritized in order to provide students with quality education and to achieve the set objectives.

Analysing the process of teaching, qualitatively recorded differences in teaching have been identified, providing a basis for a conclusion that students do not have equal access to quality education (Greitāns & Namsone, 2023). Differences have been identified among subject areas and educational stages, within a single school, and amongst different schools. In the study aimed at characterizing the variations in the quality of learning, 6 distinct teaching profiles have been identified.

Based on the data obtained from classroom observations, typical teaching profiles (TPs) have been formed, which characterize the current teaching practices (Greitāns & Namsone, 2023). For each TP, potential causes for the lack of knowledge and skills have been identified, and corresponding priority solutions for professional development have been suggested for each TP (for example, possibilities for students to develop understanding, cultivate critical thinking skills, employ information technology, etc.).

Once the data on the teaching process have been collected, developmental solutions (interventions) are planned, a significant component of which involves enhancing the professional development of teachers.

To determine the necessary support for a teacher, firstly, according to the school’s objective, appropriate criteria are set forth to assess the teacher’s competence and formulate the description of necessary competencies (aligned with the school’s objective). Then, utilizing various data sources, the teacher’s actual competence is determined. The teacher’s actual competence can be evaluated using data from classroom observations, knowledge tests, surveys, interviews, and daily practice artifacts. By comparing the teacher’s actual and necessary competencies, one can infer the “competence gap”.

For each individual teacher, the required support and professional development solutions can be planned using information about the “competence gap”. The studies conducted by UL ICEI (Dudareva et al., 2019) identify gaps between expert-observed classroom performance and teachers’ knowledge (knowing but not applying what has been learned), as well as teachers’ self-assessment of their performance (believing they do or handle something differently, or providing socially desirable responses that correspond to findings identified in other studies (Kapuza & Tyumeneva, 2018)). Therefore, overcoming the identified gap in teaching performance requires more than self-assessment. According to UL ICEI (2022d), professional feedback is also necessary, which entails data-driven support provided by a professional support person (learning consultant, mentor, etc.).

First of all, the most significant gaps should be identified (the criteria, where the actual and necessary competence differ the most), and then the competence in the remaining areas should be improved in a prioritized sequence, according to the criteria aligned with the goals of the school. The larger the “competence gap”, the more personalized professional development is required. The teachers with substantial  “competence gaps” need individualized methodological support (one-on-one work with a mentor or learning consultant).

Data-driven decisions can also be applied to enhance the effectiveness of professional development.

Effective professional development of teachers brings about changes in teaching and student outcomes. For professional development to be effective, it must adhere to four principles – expanding the teacher’s vision of how to teach, motivating the teacher to learn, developing the teacher’s skills, and implementing the acquired knowledge in the classroom. The implementation of these principles of effective teacher professional development is facilitated by incorporating the mechanisms of effective teacher professional development into the process.

The effectiveness of professional development is linked to its measurable impact.

The model for effective professional development of teachers elaborated by UL ICEI is designed to align with the context of Latvian education. This model follows the four principles of effective professional development of teachers and engages mechanisms to ensure its effectiveness. The UL ICEI model for effective professional development of teachers can be adapted to various school and teacher preparation contexts.

The effectiveness of school-based professional development of teachers significantly depends on the quality of expert (consultant) work within the school and the involvement of the school management team. The expert (consultant) is responsible for implementing the mechanisms for effective professional development of teachers, while the school management team is responsible for creating an appropriate school climate, fostering mutual trust, and ensuring resources.

When introducing innovations into school practices, it is necessary not only to plan their implementation purposefully but also to measure the quality of their implementation. The implementation of a high-quality innovation can be characterized by four sequential phases – situation exploration, preparation, actual innovation implementation, and sustainability.

Introducing various innovations into school practices does not always lead to the desired results. This can be explained by the fact that the implementation of innovation needs to occur qualitatively, i.e., it should align with good implementation practices that meet the criteria such as acceptability, cost, adaptation, fidelity, appropriateness, feasibility, and penetration (Proctor, et al.).

To effectively implement innovations in school, the first step is to conduct an analysis of the current situation. The school management team gathers data about the issue to be addressed and the stakeholders involved. They collaborate with those stakeholders (teachers, staff, experts, etc.) to establish the objectives to be achieved (criteria for successful implementation – acceptability, appropriateness, penetration). Subsequently, the preparation phase takes place, where the school management team ensures necessary resources, plans and implements teacher competency development aligned with the objectives (criterion – cost). When initiating the practical implementation of the innovation, the school management team monitors the implementation process (criteria – feasibility, fidelity) and provides an appropriate support to teachers as needed, involving professional support personnel.

A complex data analysis model for developing solutions can also be utilized by analysing data at the municipal level.

The indicators characterizing students’ relative performance (referred to as “outcomes”) describe students’ performance in municipal schools. They enable identification of the necessary improvements for further development of each specific school or groups of schools with similar performance. These indicators include a relative assessment of students’ average performance in high, medium, and low-performance groups in municipal schools; a comprehensive relative assessment of students’ average performance in municipal schools for each educational stage, considering the students’ socioeconomic status index; a relative assessment of the proportion of students with low and high performance; the trends in students’ performance in specific subjects and grade groups over multiple years.

To analyse students’ performance in relation to teaching at the municipal level, it is recommended to determine the average observed performance during the hours of instruction according to several criteria groups. Attention should be given to the significance of differences in teacher performance within a single school and among schools. The differences in the average observed performance within a single school should be analysed when students in any of the municipal schools experience difficulties in mastering specific subjects over a period of several years, as well as in the cases when there is a need to identify the risk of unequal educational opportunities (“weaker” students with “weaker” teachers) and to intervene to mitigate it. It is a priority to ascertain the extent of performance differences that are detected within a single school according to the category of basics of teaching.

To determine the extent to which the actions taken by school management lead to a successful implementation of teaching and learning (as the core processes) in terms of achievement (for example, whether the students feel physically and emotionally safe in schools), the typical practice in a specific school is compared with the recognized best practice. This is done by providing an average comparative assessment and its description, as well as identifying differences between schools.

The indicators of the school growth (development) process (referred to as “Processes II”) help to establish whether growth (development) in school is a value, namely, whether development is data-driven and purposefully guided towards the realization of a clear vision. Different developmental phases are discernible in the management of growth within the school. Various growth management phases require different developmental solutions for the school.

Other research data and accumulated empirical experience demonstrate significant differences in students’ performance, overall staff quality, teaching and learning quality, and growth management both within individual schools and among schools. This necessitates the creation of distinct, differentiated development solutions closely tailored to the specific needs of each individual school, rejecting the traditional practice of imposing a single methodological theme upon all schools, and so on.

To create such differentiated and contextually relevant development solutions, municipalities must begin with an exploration of the current situation of each school. For this exploration to be of high quality and provide substantial value, a generalized and superficial evaluation is inadequate. Instead, an in-depth and comprehensive analysis of students’, teachers’, and school management’s performance is necessary, enabling a clear identification of the pertinent challenges for each municipal school. Such analysis can be conducted by an external expert or internal municipal-level experts with appropriate competencies.

It is essential that school development is planned in relation to its existing situation or starting position. Different schools have different starting positions, and therefore municipal-level development solutions cannot be universally applicable; they must be tailored to each specific school.

^[1] Ministry of Education and Science, Republic of Latvia. Education quality monitoring system. Available: https://www.izm.gov.lv/lv/izglitibas-kvalitates-monitoringa-sistema 

^[2] Cabinet of Ministers “Regulations Regarding the State Basic Education Standard and Model Basic Education Programmes”. Available: https://likumi.lv/ta/en/en/id/303768