During the COVID-19 pandemic, EdTech tools and resources flooded into classrooms nationwide. As the unprecedented, pandemic-driven levels of school funding recede and budgets tighten, school systems are deciding which resources are used, which are effective, and which they will discontinue using. At Discovery Education, we are calling this movement The Great Rationalization and Consolidation.
As a former K-12 school administrator who has participated in the process of vetting materials for both purchase and renewal, I can say unequivocally that the leading determinant as to which edtech resources stay and which resources go is data showing evidence of impact. As school leaders undertake their own district’s rationalization and consolidation, it is important administrators understand what types of data to look at, and how to understand the story it tells.
There are three main sets of data that edtech companies can offer school leaders to show evidence of “impact” or change and growth over time. Those data sets are:
- Comprehensive usage reporting
- Commitment to ongoing research and development
- Alignment to outcomes and evidence of learning
Let’s take a deeper look at each.
Comprehensive Usage Reporting
Comprehensive usage data are detailed analyses of how teachers and students engage with digital tools and content within a K-12 learning environment. Usage data provide valuable insights into the engagement, utility, and effectiveness of the resources, and can be served up in dashboards or reports that provide useful insights for decision-making about purchases and renewals. Usage data sets may include trends over time, user feedback, progress toward standards mastery and understanding, and disaggregated subsets for use in remediation and placement.
When examining usage data, administrators should pay attention to patterns in teacher and student interactions with specific content and features, identify successful implementation strategies, and pinpoint features, services, and products that are not optimal for the district’s teaching and learning needs. To get the most out of the data, begin by formulating the questions needed to inform decisions. Then, home in on finding the answers by analyzing data responsive to those questions. Evidence of student engagement with the materials, understanding, and application is a critical part of the usage analyses. A successful edtech product should demonstrate meaningful engagement and positive outcomes, justifying its continued use or renewal. If, on the other hand, the data reveals limited engagement or effectiveness, it may be a signal for administrators to consider alternative solutions.
Commitment to Ongoing Research and Development
A commitment to ongoing internal and third-party research and development for edtech curricular resources signifies a dedication from the product provider to continuously enhance and refine products. For K-12 administrators, it is crucial to assess the depth and consistency of this commitment when considering adoption or renewal. When reviewing such data, administrators should consider the rigor of the study, the credentials of the researchers, and the actions that the company took in response to the findings.
An important framework for assessing the quality of the research is articulated in the Every Student Succeeds Act (ESSA). ESSA defines three levels of evidence-based practices to guide educators and policymakers in making informed choices. Tier 1 requires strong evidence supported by experimental studies; Tier 2 requires moderate evidence from quasi-experimental studies; Tier 3 requires promising evidence supported by correlational research or expert opinion; and Tier 4 requires the product show a demonstration of a rationale via a logic model based on rigorous research. An additional consideration for administrators is evaluating the alignment between the research evidence, program rationale and your district’s objectives.
The most important takeaway for K-12 administrators should be confidence in the vendor’s commitment to developing evidence-based solutions for school systems, and a culture of continuous improvement. Investment in ongoing internal and third-party research and development for rigorous studies is an indication that the vendor values high-quality and effective products that meet the needs of districts.
Alignment to Outcomes and Evidence of Learning
When assessing EdTech products, a pivotal consideration lies in their capacity to facilitate student growth and achievement of content aligned to state standards. Administrators must not only evaluate whether a product fulfills its intended purpose but also scrutinize how well it aligns with their curriculum and instructional objectives. The effectiveness of an edtech resource should be measured not just by its complementarity to learning goals but also by its ability to yield tangible evidence of student progress. Administrators, while analyzing alignment data, should seek discernible correlations between the use of the edtech resource and favorable learning outcomes, encompassing engagement levels, the attainment of learning objectives, and students' demonstration of knowledge through both formative and summative assessments.
The key takeaway for K-12 administrators lies in cultivating a comprehensive understanding of how the resource substantiates measurable student success specifically aligned with standards-based objectives. A compelling alignment to outcomes and robust evidence of learning should drive a positive decision to continue investing in or renewing the product. Conversely, if the data indicates a misalignment with educational goals, administrators should be prepared to make a no-go decision in favor of more effective alternatives that better support their desired learning outcomes.
Conclusion
In the era of the Great Rationalization and Consolidation, the role of data in shaping decisions on edtech resources cannot be overstated. As school districts navigate the aftermath of the COVID-19 pandemic and scrutinize their budgets, the movement towards streamlining and optimizing edtech resources is essential. The three key indicators of a vendor’s commitment to evidence-based products —comprehensive usage reporting, commitment to ongoing research and development, and alignment to outcomes and evidence of learning—serve as invaluable tools for administrators. Comprehensive usage data can be used to cull engagement patterns, successful implementation strategies, and product effectiveness. A commitment to rigorous research and development fosters confidence in a vendor's dedication to evidence-based solutions and continuous improvement. Aligning edtech resources with learning objectives and securing evidence of student progress offers administrators a nuanced understanding of the resource's impact on learning. The effective use of data in the decision-making process empowers K-12 administrators to navigate the complexities of edtech adoption and renewal, ensuring that resources align with educational goals and contribute meaningfully to student success.
About the author
Lisa Katz is the Vice President of Research and Analytics at Discovery Education. Prior to joining Discovery Education, she served in various administrative roles in Maryland’s public education system.
