Data quality
Nobody is required to make these numbers agree. These checks show how often they do not.
Why this matters
A center's report has several tables that describe the same people: consumers served, consumers by age, by race, by disability, records opened and closed. When they disagree, at least one is wrong, and a reader has no way to know which. The checks below are arithmetic only. They cannot tell whether a count is true, only whether the report is consistent with itself.
The flag count is published for every center and every state, beside every other measure, so the other measures are read with the right amount of caution. A state whose centers' tables add up has a better claim to its numbers than one whose tables do not.
The eight checks
- Consumer records do not add up. Records carried over plus new records opened should equal consumers served.
- More received a service than requested it. For at least one service type, the count receiving exceeds the count requesting.
- Goals achieved exceed goals set. For at least one goal category, more goals were achieved than were set.
- Age table does not match total served. The age bands do not add up to consumers served.
- Race table does not match total served. The race and ethnicity rows do not add up to consumers served.
- Disability table does not match total served. The disability-type rows do not add up to consumers served.
- Closure reasons do not match total closed. The closure reasons do not add up to the total number of records closed.
- Goal achievement at or above 98%. With 20 or more goals set, nearly every goal was achieved, which can mean goals are recorded as set and achieved at the same time.
How often each check fails, nationally
| Report year | Reports | Any | Consumer records do not add up | More received a service than requested it | Goals achieved exceed goals set | Age table does not match total served | Race table does not match total served | Disability table does not match total served | Closure reasons do not match total closed | Goal achievement at or above 98% |
|---|---|---|---|---|---|---|---|---|---|---|
| 2016 | 348 | 22.4% | 0.0% | 0.0% | 1.7% | 0.0% | 2.0% | 20.1% | 0.0% | 0.9% |
| 2017 | 351 | 18.5% | 0.0% | 0.0% | 2.0% | 0.0% | 1.4% | 15.1% | 0.0% | 1.4% |
| 2018 | 349 | 20.6% | 0.0% | 0.0% | 1.7% | 0.0% | 1.4% | 17.5% | 0.0% | 1.4% |
| 2019 | 348 | 7.2% | 0.0% | 2.6% | 2.0% | 0.3% | 0.3% | 0.9% | 0.0% | 1.7% |
| 2020 | 349 | 5.2% | 0.0% | 2.6% | 1.7% | 0.0% | 0.0% | 0.0% | 0.0% | 1.4% |
| 2021 | 351 | 8.5% | 0.0% | 3.7% | 2.8% | 0.0% | 0.0% | 0.0% | 0.0% | 2.6% |
| 2022 | 351 | 4.8% | 0.0% | 2.0% | 1.7% | 0.0% | 0.0% | 0.0% | 0.0% | 1.1% |
| 2023 | 349 | 8.3% | 0.0% | 3.7% | 2.9% | 0.0% | 0.3% | 0.3% | 0.0% | 2.0% |
Flags by state
| State | Centers | Centers with a flag | Total flags | Flags per center |
|---|---|---|---|---|
| Alabama | 3 | 0 | 0 | 0.00 |
| Alaska | 2 | 0 | 0 | 0.00 |
| American Samoa | 1 | 1 | 1 | 1.00 |
| Arizona | 5 | 1 | 1 | 0.20 |
| Arkansas | 4 | 1 | 1 | 0.25 |
| California | 49 | 8 | 9 | 0.18 |
| Colorado | 6 | 0 | 0 | 0.00 |
| Connecticut | 5 | 0 | 0 | 0.00 |
| Delaware | 3 | 0 | 0 | 0.00 |
| District of Columbia | 2 | 0 | 0 | 0.00 |
| Florida | 17 | 0 | 0 | 0.00 |
| Georgia | 9 | 0 | 0 | 0.00 |
| Hawaii | 2 | 0 | 0 | 0.00 |
| Idaho | 3 | 0 | 0 | 0.00 |
| Illinois | 24 | 2 | 3 | 0.12 |
| Indiana | 6 | 0 | 0 | 0.00 |
| Iowa | 6 | 0 | 0 | 0.00 |
| Kansas | 5 | 0 | 0 | 0.00 |
| Kentucky | 2 | 0 | 0 | 0.00 |
| Louisiana | 4 | 0 | 0 | 0.00 |
| Maine | 1 | 1 | 1 | 1.00 |
| Maryland | 8 | 0 | 0 | 0.00 |
| Michigan | 15 | 1 | 1 | 0.07 |
| Mississippi | 1 | 1 | 1 | 1.00 |
| Missouri | 4 | 0 | 0 | 0.00 |
| Montana | 4 | 0 | 0 | 0.00 |
| Nebraska | 2 | 0 | 0 | 0.00 |
| Nevada | 4 | 0 | 0 | 0.00 |
| New Hampshire | 4 | 0 | 0 | 0.00 |
| New Jersey | 8 | 2 | 2 | 0.25 |
| New Mexico | 4 | 1 | 1 | 0.25 |
| New York | 23 | 3 | 4 | 0.17 |
| North Carolina | 7 | 1 | 1 | 0.14 |
| North Dakota | 4 | 0 | 0 | 0.00 |
| Northern Mariana Islands | 1 | 0 | 0 | 0.00 |
| Ohio | 10 | 0 | 0 | 0.00 |
| Oklahoma | 4 | 1 | 1 | 0.25 |
| Oregon | 5 | 0 | 0 | 0.00 |
| Pennsylvania | 10 | 0 | 0 | 0.00 |
| Puerto Rico | 3 | 1 | 1 | 0.33 |
| Rhode Island | 1 | 0 | 0 | 0.00 |
| South Carolina | 3 | 1 | 1 | 0.33 |
| South Dakota | 4 | 0 | 0 | 0.00 |
| Tennessee | 6 | 0 | 0 | 0.00 |
| Texas | 21 | 1 | 1 | 0.05 |
| U.S. Virgin Islands | 1 | 0 | 0 | 0.00 |
| Utah | 6 | 0 | 0 | 0.00 |
| Vermont | 2 | 0 | 0 | 0.00 |
| Virginia | 6 | 0 | 0 | 0.00 |
| Washington | 6 | 1 | 1 | 0.17 |
| West Virginia | 3 | 0 | 0 | 0.00 |
| Wisconsin | 8 | 0 | 0 | 0.00 |
| Wyoming | 2 | 1 | 1 | 0.50 |
How a council can use this
- Ask a center about a specific flag. Most are explainable (a consumer who declined to state a race, a goal recorded after it was met) and the explanation is itself useful.
- Exclude heavily flagged centers from peer comparisons, or say so when they are included.
- Write a reconciliation step into the next subaward: the center's report, the state's report and the audited financial statements should agree, after the year offset is corrected.