HEI Data Submissions: Correcting Errors in Data Files

Revised June 20, 2003

There are two types of corrections that data reporters may need to make after locating errors in their files: the first occurs to data files prior to loading onto the database; the second applies to changes after the data files have been loaded onto the database. 

Corrections Prior to Loading onto the Database 

Data reporters usually learn about errors in data files when they view the results of the edit program. The edit program shows which records contain errors and which fields within the record are in error, and it describes how the data in the field do not meet requirements. 

To correct errors revealed by the edit program, the data reporter uses a "local" copy of the whole file, corrects the errors, and then transmits and edits the corrected file. It is important to recognize that corrections at this point in the process are not actually made to files stored on the HEI database. This approach implies the need for data reporters to maintain a copy of the data file transmitted to HEI. 

No Processing of Partial Files: Note also that there is no provision for processing partial files. Even in the case of a 1,000-record file that contains only one error, the correction must be submitted as part of a complete file. Experience has shown that the benefit of simple processing is greater than the cost of re-transmission. In considering this arrangement, keep in mind that at this point in the process, no data have actually been loaded onto the database. 

Edit Results: Errors and Warnings: HEI produces two distinct types of edit results: errors and warnings. Errors are invalid entries, and a data file containing errors cannot be loaded onto the database. Warnings are entries that are valid, but unexpected. Data files containing warnings can be loaded onto the database. 

Corrections After a File is Loaded onto the Database

The Delete Switch: The delete switch is used in the relatively rare case of a record that needs to be corrected after it has been loaded onto the database but before the edit window has closed. The interdependent nature of HEI data files makes such corrections inevitable. This situation occurs when the processing of a subsequent file reveals the need to change previously loaded data. 

Data reporters will learn that files have been loaded onto the database via e-mail messages from Regents staff. To delete already loaded data, submit a new file with the record or records that need to be deleted and set the delete switch to "Y" (for Yes). To replace a record, first delete the incorrect record and then enter the correct record with the delete switch set to "N" (for No). Please note that both the deletion and the replacement records can be submitted in the same file, but the deletion record must appear before the replacement record. The deletions and replacements will be made when the file is loaded onto the HEI database.

Please remember that this type of correction is used only after data have actually been loaded onto the database. The more usual error correction method is the reprocessing of entire files of data that have not yet been loaded onto the database. Note also that partial files are to be used when correcting data already loaded onto the database. Data reporters must submit only deletion and replacement records when making such corrections.

File Update: The update feature allows campus data reporters to update information in a file that has been loaded onto the database without deleting dependent files and without deleting the original record. If a data reporter needs to update an aspect of a file, he or she should request a load of the new file submission that matches the key fields for each record in the submission file, contains all fields referenced in the data submissions document for that file including the corrected information, and has the Delete Switch set to "N." The HEI load programs will overlay the older data with the newer data.

File Revisions After the Edit Window Has Closed: In instances where a mistake is found in the data after the edit window has closed, contact the Regents staff person responsible for that file.