Taking Stock of Your Assessment Program

Will Your District Provide Quality Data to All Stakeholders This School Year?

As school districts prepare for the start of a new school year, it's very likely that district administrators have not completed a detailed inventory and gap analysis of their district assessment program.

Informal surveys of districts completed over the last 10 years indicate that only about 1 in 5 school districts have embarked on a systematic process of designing a comprehensive balanced assessment system. In the remaining 80% of districts, they typically have an assessment system that is lacking key components and also have legacy assessments that aren't providing key stakeholders the type of data that guides critical decision-making.

Let’s start from the beginning and define a comprehensive balanced assessment system:

A comprehensive balanced assessment system is a coherent system of assessment tools, assessment methodologies, data systems and data protocols that provide data to key decision-makers to improve educational outcomes for students at each stage of the learning process.

With the passage of ESSA, schools are even more in need of close examination of their system of assessments as ESSA requires districts to create “a comprehensive continuum of evidence-based, systemic practices to support a rapid response to students’ needs, with regular observation to facilitate data-based instructional decision-making.”

Clearly, ESSA has a specific focus on the tenets of multi-tiered systems of support (MTSS), and it can be argued that an effective MTSS will have a comprehensive balanced assessment as a necessary component. The typical shortcomings of district assessment programs are key academic measures across the grade span, missing or inadequate assessment of social, emotional and behavioral (SEB) competencies and mental health screening, inappropriate applications of assessments, and difficulty collecting and reporting a variety of qualitative data.

Key Factors to Consider Around MTSS

When we think about assessment systems through MTSS frameworks and models, there are a variety of key perspectives that must be addressed:

1. Quality assessment programs must provide any array of assessment methods delivered throughout the school year. In a general sense, districts can break down their assessment program with the following schema outlined below. Assessments can be viewed as summative, formative, or interim, but also, they can be either normative or criterion-referenced. In reality, however, any assessment can have formative properties if the data is used for inform current, ongoing instruction.

The array of assessment methods must include universal screening assessments in literacy and math administered at key intervals such as fall, winter and spring. Progress monitoring assessments must be installed to provide educators data on the efficacy of tiered interventions for students, and interim benchmark assessments need to be administered at least quarterly so teachers can monitor the mastery of key learning progressions and standards. Certainly, teachers should also employ a variety of formative assessment techniques on a daily basis to inform their instructional decision-making. These methods can be as simple as student observations or as structured as using “exit tickets.”

Finally, districts need to consider assessments related social, emotional and behavioral (SEB) competencies and mental health screening. The fact is that only a small percentage of districts are conducting universal screening of these domains and screening systems can provide valuable early identification of students in need of behavioral and mental health services. The data is unequivocal—social, emotional and behavioral status is foundational to academic learning and districts leaders are now beginning to see the “tier 1” importance of SEB assessment and programming.

2. Students in different tier levels have different assessment needs. Beyond the universal screening of all students within an assessment program, it's critical for districts to understand that students identified with academic needs are to be administered assessments designed to diagnose the underlying deficits and provide valid, reliable, and efficient assessments to measure progress. Students in tier 1 will require a more general assessment program while students well- below or above grade level will require different assessments.

3. Students in different tier levels follow their unique assessment path. Related to the second point, students within the three-tiered model will be administered an individualized set of assessment tools and methods and evolves as teachers examine the data and outcomes. For example, a student who is placed in a tier 2 reading intervention that is progressing in reading automaticity (rate) but not showing progress in accuracy (errors) will require different progress monitoring assessments than a student with the reverse pattern of performance. The assessment path must be flexible to change based on the ongoing stream of student performance data.

4. Districts need to collect “effect” and “causal” data. Effect data represents the typical data that schools review and analyze. Data points such as high stakes assessment results, computer-adaptive academic assessments, attendance, behavioral referrals, district common assessments and classroom formative assessments. We call these effect data because student performance on these measures is predicated on other internal and external student causal variables and events.  These causal variables can be conceptualized by the following model outlined below. It's quite easy to imagine, for example, that student performance can be deleteriously impacted by such things as teaching technique or a neurobiological disorder. Thus, we must have “data” on these causal variables to inform our analyses of student needs.

5. Deploying a data system to support decision-making. Your balanced assessment system will not deliver on its potential if the data is housed in disparate and disconnected locations. Your assessment system demands a robust platform to store and report all student data in effective ways; this includes effect and causal data. Having all the data in one integrated platform will support the work of PLC’s and other stakeholder groups. Because teachers are not well-versed in the use of student data, your system should provide data visualizations that quickly elucidate trends and patterns in the data and allow users to “drill into” the data to answer their essential questions as shown below:

Finally, your data system must allow users to bring together multiple measures of effect and causal data and allow for subgroup disaggregation so staff can truly understand causality to drive effective intervention design. The diagram below shows a simple example of this concept:

Final Thoughts

In closing, as districts resolve to embark on the journey of improving their assessment system during this upcoming 2018-19 school year, some leaders may have difficulty visualizing how all of these various assessment components need to be assembled to deliver quality data to their stakeholders, particularly with regards to how an assessment system should be configured through the lens of MTSS.

Therefore, below is a very detailed infographic designed to represent the unique assessments required for each tier level to be delivered throughout the school year. This particular infographic is focused on academic assessments but one could also approach social, emotional and behavioral assessments using a similar approach.

The center of the concentric circles is Tier 1 in green and the outer ring in blue represents very high performing students. At the 1 o’clock position, you'll notice a black band cutting across the tiers representing universal screening in the fall of the year for all students. Often, a computer-adaptive measure is the preferred tool here since it will adjust individually to each student to hone in on their zone of proximal development.

Further screenings occur in the winter and spring to track growth and progress. District interim assessment are represented in the lighter gray bands and can be administered to all students as appropriate. Within the colored tiers, students will follow their unique assessment path, with your tier 3 students receiving much intensive and diagnostic assessments to inform instruction.

You can download the entire diagram featuring our comprehensive balanced assessment system and how our various services/platforms fit within the system:

*****

Illuminate Education is a provider of educational technology and services offering innovative data, assessment and student information solutions. Serving K-12 schools, our cloud-based software and services currently assist more than 1,600 school districts in promoting student achievement and success.

Ready to discover your one-stop shop for your district’s educational needs? Let’s talk.