NOTES:

Sun, Sep 13 EDITORIAL Check out Pascal's notes on Security for a template of good note-taking.
Sun, Sep 13: Please be patient while Justin enters his notes ...
Mon, Sep 14: Update: It took a while to get these notes compiled and uploaded ... apologies. The "raw" notes are currently uploaded. Need to clean up the notes a bit.
--Jmiranda 03:18, 14 September 2009 (EDT)

Data Quality Reports

(consensus is that these should not be called reports since they usually require some interactivity)

Revised Notes

Questions to answer

For what purpose are we/you creating data quality reports?
- To improve clinical care
- To facilitate research studies
- To improve data quality accuracy
- For aggregate reporting
- To improve availability/timeliness of data entered into system (is data ready for next visit?)

more to come soon ...

Raw Notes

For what purpose?
- clinical care
- research
- data entry errors
- timeliness of data (consult date vs reporting date) -> useful for find things like missed clinical appt
standards for data quality
- double entry module
how can we check data input
tools used for data quality
- data exports on cohort (very manual process)
- BIRT reports using SQL to implement data quality rules
  - e.g. RETURN VISIT DATE, CURRENT DRUG REGIMEN IF ON ARV, IDENTIFIER FORMAT MATCHES PROGRAM, MISSING PATIENT VISIT
catch errors when they occur (form entry)
data quality rules will be implemented in next version of reporting tool (v1.5)

Implementation specific examples
- AMPATH (ada)
  - uses SQL export + SAS to run data quality checks
  - more validation added to form schema
  - there are always exceptions in form validation
  - infopath - no way to handle complex validation like "weight gain"
- MVP (andy) Who gets/uses the report? What's the workflow around data quality reports?
  - Providers? Data Managers?
- Rwanda (cheryl)
  - currently using hard-coded module to perform data quality checks
    - found 2000 historical errors, only 50 left after about a year
  - adding an indicator requires programming (e.g. it's difficult)
  - need to configure these rather than program them
  - needs: Data manager should be able to click a button to view report
  - automated report sent to each site every week
  - need to be able to distinguish between errors and abnormal conditions
- Malawi (Evan)
  - point-of-care data entry (e.g. touchscreen) helps improve quality
  - in order to enter an "invalid" weight the user must authorize the change
- AMPATH (Ada)
  - Warnings/alerts are ignored if they continuously popup.
- Lesotho (Jeremy)
  - Re-evaluation of paper forms to find errors
- Mike
  - brought up Google Summer of Code project data integrity module as a potential solution for some
- Ada
  - module needs to be tested, workflow needs to be
  - data quality checks MUST happen on form entry!!!
  - OR at worst, same day validation
  - e.g. of an alert trigger "patient grows 30 centimeters in 3 weeks"
  - clinical reminders triggered on inaccurate data entry
- Christina
  - clinical (difficult) vs technical (easy) validation
- Evan
  - another challenge: once we find the errors, how do we resolve them
  - suggestion: batch entry form
  - cross patient off the validation fail list
- Andy
  - what about metadata quality reports
    - # of days to get forms completed
    - provider is no longer seen as completing reports ... what happened?
- Cheryl
  - uses data entry stats and data quality modules
  - has queries that support # fo forms completed by day and location
  - how long does it take for forms to be completed
  - what was behind a correction
  - important to know who made the error
  - is someone making an error a lot
  - trends are more important than finger pointing
    - fix training, workflow
- Darius
  - can we get a concrete set of requirements for data quality rules engine
    - patient list / cohort builder query that shows all patients that have failed a particular check. could lead to a batch entry form to fix errors.
    - dashboard to show trends over time
- Evan
  - define error -> cohort
  - alert on patient dashboard
  - track that error happened (don't imply)
  - error should not go away, we should comment that "error was fixed by ____"
- Andy
  - A better term for data quality errors is "exception"
- Evan
  - Add alert/exception to patient
- Types of exceptions
  - universally impossible (absolutes) rules that are applied to all patients (birthdate cannot be in future)
  - errors that occur when accidentally "merging" patients (entering form for the wrong patient)
    - e.g. two encounters on the same date
    - always use identifier to search for patient (never allow search by name when entering a form)
  - suggestions ("did you really mean to do this ...?")
  - data quality vs clinical problem
  - list of patients who should have come in
  - tracking encounters over steps (registration > intake > ?)
  - rapid SMS -> alert form needs to be filled in
  - form design implications
    - validation as post processing (rather than form validation) will allow you to find issue with the form design/workflow
  - system allows user to "force" enter through authorization (like in Malawi)
  - not a report, but an interactive reporting/analysis tool (should not be a printed PDF)
  - what about when it looks like an error but isn't
    - e.g. height cannot increase a certain amount of time
    - "stop showing" rule (alerts create data): ignore forever | ignore until next visit
  - want to know that exceptions existed even if someone ignores the alert
  - e.g. "# of pills given" does not match calculation of "# of pills needed until next visit date"
    - associate alerts with single data points
  - are clinical issues different from data quality checks
    - workflow issue (who uses system), escalate quality error to clinicians (clinical issue)
    - always start as a data quality issue, move to a clinical issue
  - let's keep the collaboration alive: what's the best means for that?

Resources

2009 Implementers Group Meeting Program Data Quality Reports

Data Quality Reports

Revised Notes

Questions to answer

Raw Notes