How do each of the data tools actually work?
There are a number of imports that the Illuminate system supports. Some of them are very straight forward, others are a bit more complex. Below is a list of the various import datasets and some information on how they work, what you should consider, and any other relevant information about the dataset. You can download full data specs from our downloads page. As always, if you are working through any data imports and have questions DEFINITELY contact us at 949-242-0343 or firstname.lastname@example.org.
Illuminate Core Data Specification
The Core Data Spec is really the heart of the system. Without the data from this core data spec the system will not function. Additionally, it is very important that data this data is loaded first and is the highest priority in your data implementation. There is no need to worry about or even think about data in the Extra Data Spec until you have your Core Data in a good place. Here is a list of the datasets that are part of the Core Data Spec
Sites is a one-time import. Your site file will only be imported at the very beginning of your implementation. After the initial data load all site management is handled through the UI.
Terms is no longer supported. This dataset has been replaced by the term manager in the UI.
The courses file should represent your entire course catalog. There are many fields available to import in this dataset, but few are required. Course IDs must be unique across the entire installation. For example, Course ID 1234 cannot be "Algebra" at one school and "Keyboarding" at another.
This file should represent ALL of your staff. Anyone that will be accessing Illuminate should be in this file.
The studemo file is the "students" file. It will contain all of the basic demographic information for each of your students. While much of the data in this dataset are not required, without some of the data reports will not run correctly. For example, without EL data or special ed information you will not be able to run certain subgroup reports. One important note about the studemo file: it does not need to be complete on day 1! You can simply send what you can initially and then append more data to the file as you work through your implementation.
When did students enter and exit school and what grade level are they in? The enter and exit dates are critical in this file for longitudinal data analysis. One of the more common issues that is seen in enrollment data is that dates overlap. A student cannot have enrollment dates OUTSIDE of term dates. Students cannot have multiple enrollments at the same site with overlapping dates.
Course ID + User ID + Period + Site ID = Section ID. The master schedule file should represent all of the uniuqe course/teacher/period/site combinations of course offerings.
The roster import will place students into sections. It is completely dependent on successful enrollment and master schedule imports. Like enrollment, enter and exit dates for students are critical!
Illuminate Extra Data Specification
The Extra Data Spec is just that ... extra data. Some of it may matter to you, some of it may not. Some data you may want to load immediately, other data you may want to load at a later date. Typically, once the Core Data is fully loaded, then folks begin to look at prioritizing key data sets from the Extra Data Spec.
The roles file is available to import role data for all non-teacher users. Teacher accounts are automatically created in the system through the master schedule and roster imports. Never send teachers in the roles.txt file. The roles.txt file should be used to create Principals, District Admins, Clerks, etc. The import will create roles based on the text vaules that are provided in the file. User IDs are then affiliated to the roles for each academic year. If you have a Principal who needs three years of affiliations at her school, she should have three rows of data in the file.
The programs.txt has one main requirement: the program record MUST contain either a program start date or a program eligibility start date. Without at least one date to indicate when the student participation in the program began, the program record will be rejected.
Contacts is one of the more complicated datasets. Because of the nature of contacts and the fact that contact data is typically stored differently across various source systems, contact data is not updatable via imports. This means that contact data will perform a student based wipe and reload. For any student that is in the file, all of their contact data will be deleted and then the data in the file will be imported. There is an exception to this behavior. At import time you can chose to "append" contact data. This will simply import data from the file and is potentially dangerous since it may create a lot of data duplication.
One of the key components of the contact data set once you're using Illuminate is the concept of "primary". In order to know who a student's primary contact is, the student needs a primary household. This information is deduced during import. A contact who is a legal guardian AND is listed as a primary contact will be determined to be the student's main residence (primary household)
This dataset should represent final grades. Traditionally, this dataset is only used for secondary schools. The transcript.txt import REQUIRES that ALL transcript records are included for any of the students in the file. The reason for this is that the transcript import replaces all data for incoming students. The process is simple and effective. All of the transcript records are first loaded into a staging table. From there, any of the data that exists in the live database is DELETED for any of the students that exist in the staging table. After that, the data from the staging table is migrated to the live tables.
This process ensures that your transcript data is always up to date. Any changes that you make in the SIS, any student grade changes, any grade deletions, any new records - they're all accounted for with the student based data replace. For this reason, it is imperative that your transcript file is always complete for any of the students that are contained in the file. The recommendation is that your file should contain all transcript records for currently active students. A sample pseudo query would look like: SELECT * FROM transcripts WHERE student_id IN (SELECT DISTINCT(student_id) FROM current_enrollment);
Use the lanaguage file to provide supplemental language data. There is some minor overlap with the studemo file in the language file.
Use the specialed file to provide supplemental special education data. There is some minor overlap with the studemo file in the specialed file.
Provide attendance data for students. Do not sent present marks. Send only non-present attendance records.
Discipline is no longer supported. This dataset has been replaced by the behavior import.
The behavior import is "incident-centric". This means that the key element of the import is the behavior incident, not the student. Each record in the import file should represent a behavior incident. One incident record will define the incident (columns 1-31), the participants in the incident (columns 32-42), and the details of the incident for the student participant (columns 43-138).
Each row in the file can represent one student participant (columns 32-33), one employee participant (columns 34-35), and/or one non-student/employee participant (columns 36-42). It is not required that an incident contain all three types of participants. All three types of participants can, however, exist on one row of data. It is required that an incident record contain at least one participant. An incident cannot exist without a student participant.
In the event that there are more than one student participants for an incident, the file should contain as many rows of data for the incident as there are student participants. It is expected that columns 1-31 of these records will be repeated for each incident record, columns 32-33 will differ as they will identify the various student participants, and columns 43-138 will differ as they will identify the various student participant specifc elements of the incident as it relates to the participant.
Traditionally used to import progress grades.
This file will create transfer records for students.
The student portal import is an INSERT ONLY import. This import is used to create accounts. There is no updating being performed. This tool cannot be used to change student usernames or passwords. In the event that you need to update the student data for portal access, you'll need to contact Illuminate support and ask to clear out ALL students and then reimport the file. Otherwise, manage the portal access via the UI.
Like the student portal import, this file is INSERT ONLY as well. This is only used for account creation, not maintenance.