BI Tool: Multiple Rounds of Feedback

When I reflect on my work with hundreds of school districts & LEAs across the United States, I realize that the true power of Illuminate’s BI Tool is not just in creating powerful reports / data visualizations—the true power lies in its ability to transform the data culture in school districts. Without Illuminate’s BI Tool (or a tool of comparable power and versatility), a data leader has little choice but to accept the canned reports provided by the system and essentially force these reports on their team. No matter how well-designed a system’s canned reports are, this “use what we have” approach often fosters resistance and an unhealthy data culture. Stop me if you’ve never encountered resistance to analyzing and using data inside your organization… With the BI Tool, however, comes the ability to create completely custom data visualizations that are sharable and scalable within the Illuminate application.  Used correctly, this power can help transform the data culture in your organization from a top-down, “use what we have” approach to a collaborative, “what do we need” approach.  With the BI Tool, instead of bringing canned reports to their team, a data leader can ask their team, “What reports do we need?”

All too often, I see data leaders gather the available data and ask, “How can we use this data?”  Or, “What does this data tell us?” As far as I’m concerned, this approach is backwards—putting the cart before the horse, so to speak. This is akin to having an answer and then trying to figure out what question is being answered. In contrast, we could start by asking questions, and then gather the data and create the reports that answer those essential questions that people need answered.

Anyone who has ever created a custom data visualization with Illuminate’s BI Tool knows that the process is very iterative. You create a report, and you may think it’s perfect, but as soon as the report is used in the real world, you’re going to get a lot of feedback & suggestions (whether you want it or not!) about how to improve the report. And so you listen to that feedback, and tweak the reporting accordingly. This feedback loop goes on almost indefinitely, which is precisely the point. With this feedback loop, you are now one step closer to a healthy, collaborative data culture.

In our previous post, Jacob Walker from Sacramento City Unified School District describes how the “ongoing quality control with subject matter experts with our Student Services team was invaluable.” And there you have it.  The “ongoing quality control” is the feedback loop I’m talking about. (If you missed it, read all about here!)

In the same vein, for our February blog post, Alliance College-Ready Public Schools, the largest charter school network in Los Angeles, highlights one of their favorite BI Tool reports and repeatedly describes the “multiple rounds of feedback” that led to their custom “Data Summary” reports being “by far our most used reports.”  I would argue, the very reason their custom BI Tool report is used more than any other is the collaborative, iterative process that they’ve implemented with the help of the BI Tool. Read all about the incredible work Alliance College-Ready Public Schools has done with Illuminate’s BI Tool below.

Please join Illuminate’s BI Tool User Community to collaborate with Illuminate BI Tool users throughout the country.  Simply follow this link and click the “Follow” button at the top.  (To participate in the forum, login to Illuminate, click your “Control Panel,” select “Illuminate Help Center,” and click on “Community.”)

If you are interested in contributing to the BI Tool Blog, please email me at: freyherme@illuminateed.net

Sincerely,

Franck Reyherme
Director of Business Intelligence Services
Illuminate Education

*****

Multiple Rounds of Feedback

By: Erin Lowther, Alliance College-Ready Public Schools

Alliance College-Ready Public Schools is the largest charter school network in LA. We serve about 13,000 scholars in grades 6-12, with 8 middle schools, 15 high schools, and 3 combined high school/middle school complexes. Alliance was founded with a mission of operating high-performing high schools and middle schools in low-income communities in California with historically underperforming schools, that will annually demonstrate student academic achievement growth, and graduate students ready for success in college. I joined Alliance’s Academic team a little over a year ago as the Data and Assessment Coordinator. I work with an Academic Data team within our larger team, with Lacey Westphal, our Data Analyst. I manage most of the testing administration and user management in Illuminate, and both Lacey and I create reports using the BI tool.

The report below is called “2018-19 IA2 CPM Math Data Summary: Grade 6.” It shows overall percent correct by subgroup, overall percent correct by question group, and comparison of overall percent correct from previous interim assessments, for the 6th grade CPM math interim assessments. Teachers use this report during Data Day and planning meetings after our district-wide interim assessments in math. This report, and it’s science and ELA counterparts, are by far our most used reports.

Alliance’s interim assessments are aligned to end of year SBAC assessments. Knowing how students are performing on each standard and question group allows teachers to look for opportunities to reteach. This report also shows growth from the beginning of year assessment to the midyear and final interim assessments. Educators are able to see how students are, on average, performing on questions aligned to standards or question groups. We also have a DDI (or data driven instruction) tool that teachers are able to fill out during school-wide Data Days that inform them about which standards students need the most support on.

When designing reports, we try to keep it simple and break up charts with other ways of showing data, but bar charts are always the easiest to understand. The design of this report was chosen through multiple rounds of feedback from those who were leading and facilitating data days. It also corresponds to the DDI (data driven instruction) tool. We use color to differentiate charts from each other (different content = different color scheme) and also to differentiate between subgroups and interim assessment rounds. To be sure that these reports will be usable for teachers, we engage multiple rounds of feedback. Also, our district office hosts a preparation meeting for facilitators before Data Day so that people who are leading sessions are aware of how to use the reports and what the data means so that they can lead accurate and positive discussion.

This report provides all the information a teacher needs to quickly assess how their students performed on the interim assessment as well as subgroups of interest. They can then take that context into the response frequency reports available in Illuminate to dig deeper.

Our biggest challenge is always knowing what information teachers need. Another challenge for us is to make sure any duplicate students aren’t counted, and that all the question groups in the assessment fit on one page! For other BI Tool users who may create a similar report, try to figure out a way ahead of time to reduce the number of reports you need to update. Currently, after the end of each round of interim testing, we need to update the SQL code to include the assessments that were taken. We are currently getting ready to publish a new version that will allow users to filter to an assessment rather than creating a separate report for each one.

*****

Illuminate Education is a provider of educational technology and services offering innovative data, assessment and student information solutions. Serving K-12 schools, our cloud-based software and services currently assist more than 1,600 school districts in promoting student achievement and success.

Ready to discover your one-stop shop for your district’s educational needs? Let’s talk.