How to Use Quality Assessment Content to Build Assessment and Data Literacy

If you’re like most educators, your credentialing program focused on two main things: teaching strategies and classroom management. Information about how to assess and understand data is lacking in many of these programs today.

To fill that gap, districts are now working to build assessment and data literacy through professional development (PD). But many districts are strapped for PD resources. Year after year, assessment and data literacy sessions can unintentionally get surpassed in favor of other workshops.

One solution to this problem lies in using your high-quality assessment content, or items. Item banks do a lot more than just populate assessments! Your assessment items can be used in tons of cost-effective, high-impact exercises to use in professional learning communities (PLCs) to help grow your assessment and data literacy and increase your capacity to support student learning.

Here are some examples of how you can repurpose the high-quality content from your assessments to address some of the biggest areas of PD need that I see in schools.

Depth of Knowledge (DOK)

Understanding DOK alignment between assessment and instruction can be difficult. In my experience, it’s one of the most commonly misunderstood assessment concepts. It doesn’t have to be!

Good news: if your assessment content has been analyzed and tagged to a DOK level by professionals, you have PD at your fingertips.

Try this. In your PLC or group, start by studying the definition and levels of DOK. Then, to better understand the concept, pick out a few assessment items. As a group, discuss what makes each item a DOK Level 1, 2, 3, or 4. Then, discuss as a group how to teach that concept or skill to that same DOK (a conversation we often leave out in PLCs).

With this knowledge in hand, your group can also work together to check and revise your existing instructional maps to ensure students arrive at the right DOK by the time of assessment.

This simple exercise can be so powerful. Not only does it grow our understanding of DOK’s connection to assessment and instruction, but teachers are also working directly with the content they’re teaching in the classroom.

Start your conversation by defining DOK and the DOK levels.

Distractor Rationales

A quality item should do two things: it should tell us what students do and don’t understand, and it should guide our next instructional steps.

Quality content helps do this by providing distractor rationales, or reasons why a districator (or incorrect answer option) may have been selected. Quality distractor rationales are typically psychometrician-provided and pinpoint student misconceptions so teachers know exactly where students are struggling.

When analyzing distractor rationales post-assessment, we can use them to plan differentiated instruction for the entire class, small groups, or individual students. But there’s a big professional development opportunity in analyzing them before the assessment, too.

In your PLC or group, select a few items, and work together to discuss the distractor rationales and common misconceptions they represent. How will you address those misconceptions as you introduce the concept in your classroom?

Bonus: distractor rationales make for great classroom discussion, too. Have students analyze the most missed items in your classroom. Ask them to identify the most commonly selected distractor, pinpoint the misconception, and how to correct it for next time.

Use the distractor rationales in your content to pinpoint misconceptions and inform next steps.

Item-Specific Rubrics & Student Work Samples

When it comes to scoring student work based on a rubric, inter-rater reliability can be a big challenge. Ideally, we want to ensure scoring for constructed responses is consistent and fair, where student work would receive the same score in any classroom.

High-quality constructed response items come with item-specific rubrics and sample student responses. Those sample responses are a great professional development opportunity that also increases consistency and fairness in how constructed responses are graded.

As a PLC or group, come together around those student response samples and item bank rubrics to determine what the student evidence shows. Discuss where the work falls on the rubric and why. Discuss how it compares to student work with a different rubric score. (You can also do this with exercise with anonymous student work samples.)

Use the response examples to help increase inter-rater reliability, consistency, and assessment literacy around constructed responses

Focusing on Site-Specific Assessment and Data Needs

Was the assessment I administered reliable and valid? Did it meet the targets I intended it to meet? How did my test questions perform? What was the range of difficulty of my items?  Did my items discriminate between high and low learners?  

Are these questions you hear in your PLC meetings? If not, they should be. When we’re all able to participate in those types of conversations, we create better assessments and take better next steps.

Quality item content provide tons of illustrative examples to help your team understand assessment literacy concepts and key terms (like reliability, validity, P-value, Point Biserial, etc.) and how they fit together. You’re able to hand-pick the terms or concepts that the group is collectively least comfortable with and customize your study accordingly.

Try viewing item statistics in your assessment platform. What do they mean? What do these stats tell you? Which stats are we confused about in terms of meaning or importance? Talking through some of these questions takes your PLC to the next level.

Test statistics provide a checklist of terms and concepts to know—and unlock more feedback from your assessment results.

Illuminate Education provides high-quality assessment content, professional development, and pre-built assessments to districts across the nation. Reach out today to learn more.

*****

Illuminate Education partners with K-12 educators to equip them with data to serve the whole child and reach new levels of student performance. Our solution brings together holistic data and collaborative tools and puts them in the hands of educators. Illuminate supports over 17 million students and 5200 districts/schools.

Ready to discover your one-stop shop for your district’s educational needs? Let’s talk.