The Importance of Setting-Level Data in an Early Warning System

Guiding school improvement requires early warning indicators (EWI) at the setting level—school and classroom—too, which can be used to evaluate school practices, improve school climate, and enhance teacher effectiveness. The ultimate goal of setting-level indicators, according to the University of Chicago report, is to make “changes in adult behaviors and school systems so that they are more effective in supporting students.”

Aggregating Student Data

Setting-level indicators can come from a variety of sources. One valuable source is aggregated student-level data. We’ve previously discussed individual student data in relationship to the ABCs (attendance, behavior, and course performance)—three reliable indicators of student success. We learned how a weekly review of ninth-graders’ attendance could identify individual students at risk.

But we also learned how the analysis of aggregated attendance data over time—by period, subject, class- room, or student subgroup—could reveal insights about a school that may suggest broader pathways to closing the achievement gap.

“Knowing a student’s race, gender, or disability status may not significantly in¬crease the prediction of whether they will graduate,” say researchers at the University of Chicago. “But those pieces of information aggregated at the setting level may guide decisions about potential interventions.” The Chicago report goes on to cite various demographic lenses, such as gender, through which to track course performance (the C in ABC) at the setting level:

For example, a school that has considerably higher graduation rates among their girls than their boys might monitor gender differences in ninth-grade course failure rates—an early indicator of high school graduation—to test strategies intended to reduce the gender gap in graduation.

Schools may then zoom in on gender differences in failure rates classroom by classroom: Schools may … compare failure rates across different classes and teachers. If course failure is clustered in a few classes, interventions might be more appropriately aimed at teachers or departments, rather than individual students. On the other hand, if students are failing class- es without a strong difference across subjects, the level of intervention may be more appropriately targeted at school culture and school structures.

The example above is just one way that schools can use patterns in aggregated student data to understand inequities in educational outcomes for traditionally underserved subgroups.

Focusing on SEL

Social-emotional learning (SEL) data is another source of setting-level indicators that can bolster your district’s EWS. Looking at aggregated SEL data may lead to the creation of safe and supportive learning environments.

SEL data is collected around five competencies—self-awareness, self-management, responsible decision-making, relationship skills, and social awareness. Schools that have implemented SEL programs have seen significant improvements in attendance rates, dramatic reductions in disciplinary referrals, and marked increases in graduation rates.

Measuring Teacher & Student Perceptions

Another integral way of understanding school climate and identify areas for growth is through the use of perception data. This information can make its way to educational leaders through surveys of students, parents, and teachers. In Chicago, annual surveys probe five separate areas that map to Bryk’s Five Essential Supports: leadership, instructional guidance, professional capacity systems, student-centered learning climate, and parent/school/community ties.

Looking at Formative & Summative Data

As with student-level indicators, setting-level data can be formative as well as summative. Formative indicators can be made available in real time and facilitate more rapid feedback. Schools, for example, could initiate a campaign to increase attendance, and check results on a biweekly or monthly basis.

Summative data, on the other hand, is generally collect- ed at the end of a given year and can be used to make programmatic changes for the following year. If you have the ability to look at longitudinal data, you’ll be even more likely to notice institutionalized patterns that may require attention.

Thinking Strategically

Determining the right setting-level indicators to monitor involves thinking strategically about the priorities of your district. The report from the University of Chicago explains how setting-level data changed the way schools in the Windy City supported individual students:

When the district integrated on-track rates into the accountability system for high schools, it provided a signal that high schools should pay more attention to students’ performance in the ninth-grade year. Before that, ninth grade was often seen as a year when students could make mistakes and still recover. Eventually, individual schools developed very different practices around dropout prevention than in the past, interventions that focused on preventing failures in the ninth grade. These changes have been credited with dramatic improvements in graduation rates in the district.

When I was Chief of Schools at the San Francisco Unified School District, we leveraged a strategic plan that detailed organizational aspirations. It included a graduate profile that became our North Star, guiding action at the district, site, and classroom levels to help all students find their inner spark.

With such a North Star in place, districts can begin to formulate an EWS that measures the right data, leverages the right resources, and provides the right central office supports to help every student reach their highest potential.

Setting-level indicators can provide visibility into a school’s progress in its effort to support all students. They can provide a fuller picture of the educational experience, so that any flaws in the system can be addressed. It is our moral imperative to serve all students, especially those who need a little extra assistance in order to excel.

We’ll talk about the people behind the data, those who drive action on behalf of every student in the next post.

*****

Illuminate Education is a provider of educational technology and services offering innovative data, assessment and student information solutions. Serving K-12 schools, our cloud-based software and services currently assist more than 1,600 school districts in promoting student achievement and success.

Ready to discover your one-stop shop for your district’s educational needs? Let’s talk.