Student-Level Data in an Early Warning System: Less is More

In the last post, we discussed the importance of using just a few proven early warning indicators (EWI) to avoid overwhelming teachers, administrators, parents, and students with unnecessary data points. We learned that attendance, behavior, and course performance are far and away the best indicators of student outcomes, and that these “ABCs” should form the core of any effective EWS.

This post will examine how the San Francisco Unified School District (SFUSD) has been using GPA and attendance data from eighth grade students to improve high school graduation rates.

SFUSD staff and researchers from the John W. Gardner Center for Youth and Their Communities (JGC) at Stanford University came together, under the auspices of San Francisco’s Bridge to Success (BtS) initiative, to discuss ways to identify incoming ninth graders who need extra supports.

The JGC determined that the presence of two data points, measured at the end of eighth grade, reliably predicted an on-time graduation rate below 25%:

•  GPA below 2.0
• Attendance rate below 87.5%

The initiative analyzed two cohorts of students (one group entering ninth grade during the fall of 2005-06 and the other in 2006-07) to determine the ability of the data points to predict graduation risk. Averaging the numbers from the two years gave researchers a single cohort of 3,382 students, with a four-year high school graduation rate of 72.9%.

Incoming ninth graders who entered high school with one of the risk factors were half as likely to graduate (43%) as students with no risk factors (84%). The graduation rate of students with two risk factors dropped all the way down to 15%. (See Figure 1)

A total of 749 students (22%) entered ninth grade with one or both risk factors. Of the students with only one risk factor, 274 (58%) had a GPA below 2.0 and 202 (42%) had an attendance rate below 87.5%.

Of the two risk factors, GPA was the stronger predictor of failure to graduate on time: Students with a low GPA were less likely to graduate (39%) than students with low attendance (48%).

Using EWI to Provide Student Supports

By analyzing eighth-grade student data through the lens of GPA and attendance, district officials can flag incoming ninth graders who might benefit from additional supports.

Summer Programs for At-Risk Students

One promising intervention is a summer program designed to help at-risk students make a smoother transition to high school and arrest any learning loss that could take place during the long break. Because students would have to be identified earlier for recruitment in such a program, JGC researchers identified similar predictive indicators—a GPA below 2.0 and an attendance rate below 85%— measured in the middle of the eighth-grade year, rather than waiting until the end of the second semester. This approach identified 633 students with one or both risk factors.

Scaffolding Academic Success in Ninth Grade

Support is also crucial throughout the first year of high school to ensure that all students stay on track. Figure 2 shows that, regardless of a student’s risk factors, academic success in the first semester of ninth grade has a significant effect on long-term success in high school.

Students who passed all core courses in the first semester of ninth grade showed significantly higher graduation rates than students who failed even one core course, regardless of their pre-existing risk factors. Of the students with zero risk factors, 87% passed all their core courses in the first semester, but those who failed even one course had a four-year graduation rate that was 33 percentage points lower.

For those students with one risk factor, failing one or more core courses led to a 23-percentage-point drop in the likelihood of graduating. Again, course performance proves the stronger indicator of academic success: Ninth graders with a low GPA were less likely to pass all their core courses than students with low attendance.

For students with both risk factors, passing all core courses more than doubled the percentage of students who graduated, but only 26% of these students were able to pass all their core courses.

This analysis points to the importance of identifying EWIs at various points in eighth and ninth grade. While eighth-grade data can capture a large proportion of likely dropouts, some students without obvious warning signs at that time will still not graduate, and the failure of a core course in the first semester of ninth grade is clearly a red flag.

Optimizing Your Early Warning System

It’s important to keep testing theories of action and look for other valuable data to enhance your EWS. If you can look at historical data for every school in your district, for example, you’re likely to identify data points that may be locally relevant, such as family involvement, school climate, instructional quality, etc.

Developing an EWS requires a substantial investment of time and resources, and requires a consistent commitment to understand what it takes to identify at-risk students and help them succeed. What’s more, because the number of identified students is unlikely to be divided equally among schools, strategies to improve student performance may need to be substantially different at each location.

Systemwide transformation can occur when top-level administrators understand the power of data. The 2018 report, High School Graduation and College Readiness Indicator Systems: What We Know, What We Need to Know, from the University of Chicago Consortium on School Research, clarifies how states and districts can use EWIs to drive student success:

• EWIs communicate district priorities for supporting at-risk students to each school.
• Education agencies can use EWIs to communicate priorities and expectations about the work in which schools should be engaged.
• Districts can use EWI data to provide milestones to monitor and support at-risk students.

In San Francisco, the use of an early warning system is working, as SFUSD continues to improve and exceed statewide graduation and A–G course completion rates. The district’s 2014-15 Cohort Outcome Data shows an 84.9 percent graduation rate, compared to the state rate of 82.3 percent. The data represents a 2.8 percent increase for SFUSD since 2011-12 and a 7.7 percent increase since 2009-10. These are remarkable advances.

Interestingly, the cohort of students graduating in 2015 was the first to benefit from the district’s EWS and the resulting supports. Most impressive from an equity standpoint, the graduation rate increased year over year for most traditionally underserved subgroups, including Latinos and African Americans, as well as English Learners, Special Education students, and socio-economically disadvantaged students.

Next, we’ll learn about the importance of setting-level data in effecting positive, systemic change.

Would you like to learn more about using early warning systems to provide students appropriate supports? Read our latest eBook:

*****

Illuminate Education is a provider of educational technology and services offering innovative data, assessment and student information solutions. Serving K-12 schools, our cloud-based software and services currently assist more than 1,600 school districts in promoting student achievement and success.

Ready to discover your one-stop shop for your district’s educational needs? Let’s talk.