Loads an HMIS CSV export into a reporting database, runs 14 data quality checks, tracks agency follow-up and tests every backup by restoring it.
| File | What it shows |
|---|---|
| Data quality report (PDF) | Summary, findings by check, error rate by project, rules, example records and the database administration checks |
| flagged_records.csv | One row per flagged record, ready to split by agency |
| dq_summary.csv | Flagged count, records checked and rate for each check and project |
| README | How to run it, how it is built and what each check finds |
| Source code and synthetic export (ZIP) | Python 3 standard library, SQLite, SQL rules in one file |
Reporting period October 1, 2025 to September 30, 2026: 913 active enrollments for 592 clients across six demo projects.
| Check | Finds | Flagged | Checked |
|---|---|---|---|
| DQ01 | Name missing, unknown or refused | 17 | 913 |
| DQ02 | SSN missing or failing format rules | 68 | 913 |
| DQ03 | Date of birth missing or impossible | 14 | 913 |
| DQ04 | Veteran status missing for adults | 13 | 709 |
| DQ05 | Disabling condition missing | 21 | 913 |
| DQ06 | Household without exactly one head of household | 13 | 726 |
| DQ07 | Prior living situation missing | 25 | 730 |
| DQ08 | Exit destination missing (8, 9, 99, 30 or blank) | 65 | 651 |
| DQ09 | Exit date before entry date | 2 | 651 |
| DQ10 | Entry recorded more than 4 business days late | 94 | 789 |
| DQ11 | Exit recorded more than 4 business days late | 57 | 651 |
| DQ12 | Overlapping shelter or transitional housing stays | 10 | 382 |
| DQ13 | Possible duplicate client records | 12 | 712 |
| DQ14 | Shelter or outreach enrollment open over 90 days | 31 | 86 |
The generator planted 12 duplicate client pairs, 10 overlapping stays and 2 exits keyed before entry. The run flagged all of them.
The HMIS stays the system of record. The reporting database is rebuilt from each export. A trigger makes loaded export tables read-only, so corrections are made in the HMIS and arrive with the next export. Each load stores its reporting period and the SHA-256 hash of every source file.
Findings are tracked through to correction. Each flagged record carries a status (open, sent to agency, corrected in HMIS, accepted as is), and a second trigger logs every status change with a timestamp.
Indexes follow the checks. The overlap and duplicate checks are self-joins on person, date and identifier indexes; the report prints their query plans.
Backups are tested. Each run copies the database with SQLite's online backup API, runs an integrity check on the copy and compares every table's row count and content hash with the live database.