Developing a Data Strategy for Early Warning Indicators

In my previous post, I discussed the ways in which early warning indicators (EWI) and early warning systems (EWS) are designed to advance an equity agenda. By identifying at-risk students early and providing them with additional supports to help them reach their highest potential, these tools enable more students to experience the rewards of steady achievement and personal development throughout their K-12 education.

In this post, I hope to show you how you can use data to identify at-risk students, determine appropriate interventions, and measure the interventions' effectiveness.

Which Early Warning Indicators Are the Most Effective?

The first step in creating an EWS is to determine which data points you should use as early warning indicators. Too much data can be overwhelming, so it’s best to keep the core of your EWS as simple as possible. Choose a limited number of indicators proven to predict student outcomes.

Fortunately, research provides many answers. In the early 2000s, investigators from the Consortium on Chicago School Research, the Center for Social Organization of Schools at Johns Hopkins University, and the Philadelphia Education Fund determined that attendance, behavior, and course performance—collectively known as the ABCs—were the most powerful indicators of student outcomes—more powerful than demographics or test scores.

These findings have been validated by numerous other studies, which have come up with specific thresholds for each indicator in most circumstances. According to On Track for Success, the landmark 2011 report by Civic Enterprises and the Everyone Graduates Center at Johns Hopkins University, these optimal thresholds are:

• Attendance: Missing 20 days or being absent 10 percent of school days
• Behavior: Two or more mild or more serious behavior infractions
• Course Performance: An inability to read at grade level by the end of third grade; failure in English or math in sixth through ninth grade; a GPA of less than 2.0; two or more failures in ninth grade courses; and failure to earn on-time promotion to the tenth grade.

Obviously, interventions should occur long before children meet these thresholds, but knowing the thresholds provides a pathway for monitoring each data point.

What Other Indicators Should I Consider?

Having core indicators doesn’t mean you can’t test others along the way, but sticking with the basics, especially in the beginning, will prevent your stakeholders from being deluged with data. In addition to the ABCs, most experts agree that the addition of locally relevant indicators may improve the effectiveness of any given EWS.

Status indicators, for example, may be helpful in identifying, and ultimately reversing, patterns of institutional inequity. These include data about race and socio-economic status, as well as other factors such as having a learning disability, parent education levels, homelessness, foster care, and interactions with the justice system.

Similar to status indicators, causal indicators include mental illness in students or parents, substance abuse, a history of bullying or being bullied, low self-esteem, poor social skills, and poor time management skills.

While it’s necessary to understand the key indicators that put students at risk, it can also be useful to understand indicators that lead to student success. Known as protective factors, these indicators can include things like strong attendance, engaged parents, and a high degree of persistence in managing tasks.

One more issue to consider is the use of historical (or longitudinal) data to get a fuller picture of risk factors over time, especially at key transition points such as those between Pre-K and kindergarten, elementary school and middle school, and middle school and high school. Historical data can give you a more comprehensive view of any particular student, and can also help you identify institutionalized patterns in schools and classrooms throughout your district.

Student-Level Versus Setting-Level Data

Most of the data you’ll use in your EWS can, and should, be looked at through both an individual student perspective and a school climate perspective. Let’s take attendance data, for example. To identify at-risk students, you may conduct a weekly review of ninth graders, flagging un-excused absences to see if interventions are required. At the same time, for the sake of school improvement and accountability, you can review attendance data at the setting level—by period, subject, or classroom—to identify the need for systemic changes.

We’ll take a closer look at these types of data in subsequent posts, but it should be noted here that setting-level data is critical to improving instructional efforts and other student services. When a school climate is healthy, it sets the stage for optimal student learning.

Data Analysis & Training

The use of accurate data to create an effective EWS requires analysis and input from teachers, administrators, parents, community partners, and students. Without people in place to collect, analyze, and distribute data in a timely manner; to determine appropriate interventions for specific students; and to evaluate the effectiveness of interventions, the data itself doesn’t mean much. Think of your EWS as a continuous inquiry cycle that requires ongoing monitoring and continuous improvement to get better results over time.

We’ll go into more detail in later posts, but best practices suggest that each school should create a leadership team to champion, implement, and manage your district’s EWS. Trained staff should not only collect, analyze, and distribute the relevant data, but they should also ensure that their audiences—teachers, parents, and students—understand and know how to act on the reports.

Teachers and site leaders occupy a critical position in the middle of the data pipeline. They receive data from the system, and they also need to be out on the front lines, discussing the relevant information with parents and students to drive action. Your leadership teams should provide ongoing support to help educators make meaningful use of the data that flows across their desks.

Advances in technology are making it easier than ever to compile data into digestible, intuitive reports that are easy to act on, both online in the form of data dashboards or as printable emails or PDFs. Illuminate Education, for example, in partnership with eduCLIMBER, offers one of the most easy-to-use data visualization tools available.

Implementing Intervention Programs

After determining your data indicators and leadership teams, it’s time to get to the heart of the matter: implementing intervention programs.

As districts determine their capacity to help students flagged for intervention, resources may need to be more equitably allocated based on student needs. This may require getting more funding for specific schools, seeking new community partnerships, or hiring specialized personnel.

The bottom line is that finding the resources for a full-blown EWS will likely involve a paradigm shift, some political maneuvering, and an equity-based leadership agenda that emphasizes inclusivity.

Experience has shown that it can take two to three years to fully implement an EWS in a given district. Because most successful EWS efforts are field-tested in a pilot program before district-wide rollout occurs, it’s important not to go too fast.

Although early warning systems are still in developmental stages, their execution has improved considerably during recent years, mainly because of greater data availability, enhanced analytics, and educational leaders committed to data-driven decision-making.

In the next post, I'll dive deeper into the use of student-level data to identify at-risk students.

Would you like to learn more about using early warning systems to provide students appropriate supports? Read our latest eBook:

*****

Illuminate Education is a provider of educational technology and services offering innovative data, assessment and student information solutions. Serving K-12 schools, our cloud-based software and services currently assist more than 1,600 school districts in promoting student achievement and success.

Ready to discover your one-stop shop for your district’s educational needs? Let’s talk.