BI Tool: Distance Learning Engagement Report

For so many districts, the ongoing school disruptions have upended the processes of monitoring attendance, participation, engagement, and connection. Traditionally, those metrics revolve around students being physically present in brick-and-mortar environments: teachers can scan the classroom to see who is absent and who is engaged in learning; educators can meet with students 1:1 by simply pulling them aside before or after class or asking the student to drop by the counselor’s office. With remote instruction, those built-in processes went away, and monitoring student engagement has become challenging, given the multitude of synchronous and asynchronous ways students can participate in remote instruction. And at the same time, districts saw an increased need to track and quantify those metrics.

When I reached out to Dr. Dan Sosa, Director of Research, Assessment & Evaluation/Technology Services at Riverside Unified School District, about creating a BI Tool Report around remote learning data, his team was already grappling with this work. We worked collaboratively to determine which questions their team wanted to be able to answer—and what actions they wanted to be able to drive—based on remote learning data. From there, we created visualizations that would make it easy and efficient to answer those questions. The outcome was the Distance Learning Engagement Report—a completely new report designed to highlight participation, engagement, and communication patterns at the district, school, grade, and subgroup level.

As you delve into Dan’s post, it’s important to highlight two behind-the-scenes processes that his team practiced and will impact other teams’ ability to turn these data into useful reports:

1. Start by identifying the questions you want to answer with your remote learning data and mold your data collection around those questions
2. Create a consistent methodology and terminology around how those data are collected, so the data can be meaningfully aggregated (Dan has said that this is his biggest advice for educators embarking on this work)

The first step of any BI Tool Report is the data—and while many districts are collecting some kind of data around remote learning, many others are grappling with how to start. To assist with this, our team provides a pre-populated data entry table—in addition to the actual BI Tool Reporting—to DnA clients who choose to implement this report in their district. The data entry table is customizable to include your own communication, engagement, and participation data metrics, such as how students are participating. A tile can be pushed out to all teachers so that data entry is a click away for either daily or weekly data collection. If you’d like to learn more, please reach out to me.

Please join Illuminate’s BI Tool User Community to collaborate with Illuminate BI Tool users throughout the country. Simply follow this link and click the “Follow” button at the top. To participate in the forum, login to Illuminate, click your “Control Panel,” select “Illuminate Help Center,” and click on “Community.” If you’re interested in contributing to the BI Tool Blog to share your reporting experiences, please email me at: freyherme@illuminateed.net

Sincerely,

Franck Reyherme
Director of Data Visualization Services
Illuminate Education

***

Distance Learning Engagement Report

By: Daniel Sosa, Ed.D., Director of Research, Assessment & Evaluation/Technology Services, Riverside Unified School District, CA

Like so many other districts across the country, the spring 2020 school closures forced our staff and students to move into a 100% remote instruction environment almost overnight. With no notice, our team developed a plan to deliver instruction to our 40,000+ students and put the necessary systems in place. I’ve started calling this phase “crisis learning”—a much different scenario than the planned “distance learning” experiences our teams are preparing for the fall. We were recreating systems on the fly and grappling with a number of competing issues. 

As we worked to continue to provide instruction and support for our students, we were also anticipating that our Board would have a number of questions around student engagement, participation, and communication throughout the months of remote/crisis learning. Our team began to plan how we would eventually communicate about those metrics and therefore started to collect the data we’d need to answer those types of questions. Because we had to move so quickly, this data collection process turned into an immense Google Sheet with 50 schools-worth of data and 2,000 teachers updating the sheet at the same time—a far cry from what we’d have chosen if we’d had time to plan and prepare. It was quickly apparent that we needed both a uniform way to collect those data as well as visualizations to help us make sense of that massive data set.

At the same time that we were grappling with how to approach that work, Franck reached out about the possibility of creating a BI Tool Report to collect and analyze Distance Learning data. Franck helped our team identify the data that would be needed to populate a report that could answer our key questions—and in doing so, we found that we were already collecting much of the necessary data in our Google Sheet. My colleagues and I gathered the data sets and sent them over to Franck. From there, we worked collaboratively on the report and transformed that enormous spreadsheet into useful, interactive data visualizations with analytics and filtering.

The Distance Learning BI Tool Report has helped us answer the questions that we knew the Board was going to ask: Which students were participating or engaged? Which students weren’t, and what were likely some of the factors for the students who weren’t engaging? Beyond that, we’re able to filter down to see vulnerable student groups and school or grade-level groups to identify the students who hadn’t participated in X amount of weeks. So, instead of having to go through that immense Google Sheet, our team can simply filter on various schools, grades, ethnic groups, and language groups within the report to see who didn’t participate and immediately contact those kids and families to see how we can offer support. This has been extremely powerful for us: it helped us provide the right supports during the spring, and has also helped to inform our summer offerings and identify the students we needed to target for support during the summer.

The Distance Learning Report also enabled us to provide key talking points to our superintendent. We were able to say: We had 505 English learners who never engaged one time, or We had 1,276 African-American students who never engaged at all. What are we going to do for those students? It gave our decision-makers the right data points they needed so they could push on the system in a positive way and ask what support would and should look like.

The report also provided important information around communication methods and patterns. RUSD is a big organization—the 16th largest school district in California—and for that reason we’re constantly asking ourselves whether we’re communicating efficiently and effectively. As part of our Distance Learning Engagement Report, we tracked our various “communication pathways,” or the various methods we used to communicate with students and families throughout crisis/remote instruction. Then, as part of our distance learning planning this summer, we administered surveys to parents and families with a number of questions, one of which was, “Which of these communication pathways worked best for you to receive information and updates during Distance Learning?” We included all the communication pathways we had used during the spring, all of which we tracked on the report. It was surprising and interesting to find that text and email—the methods used most during the spring—came up in the survey results as either the first- or second-best positive communication method of choice for parents, students, and families. We had actually been expecting a mismatch and anticipated that we might have missed the mark in terms of communication. By using the Distance Learning Engagement Report, we found that we actually hit it.

This was our first time exploring the BI Reporting Tools within Illuminate DnA. Knowing what I know now, I wish I would have explored these much sooner. The Distance Learning Engagement Report made all the disparate data elements we were collecting so easy to manage, visualize and question, helping our teams provide significant support to our students and families during a time of crisis.

*****

Illuminate Education partners with K-12 educators to equip them with data to serve the whole child and reach new levels of student performance. Our solution brings together holistic data and collaborative tools and puts them in the hands of educators. Illuminate supports over 17 million students and 5200 districts/schools.

Ready to discover your one-stop shop for your district’s educational needs? Let’s talk.