Wiki Spaces

Documentation
Projects
Resources

Get Help from Others

Q&A: Ask OpenMRS
Discussion: OpenMRS Talk
Real-Time: IRC Chat | Slack

Resources

Page tree
Skip to end of metadata
Go to start of metadata

Work in progress!  Please commend and add ideas. Some questions we have:

  • How would these data be visualized or put into a dashboard?
  • What do we expect to change (statistically speaking to show a true difference)?
  • Let's refine these KPIs (what are numerators and denominators)? How do the KPIs match with the what do we want to know? How can we make these practical and in-line with what's possible to measure?
  • How can we partner with the Analytics squad as part of the implementation? 

What do we want to know?

KPIs & analytics

What can we do with this information?

Data Collection Requirements

End-user focus


Do we see a stable regular pattern of use by expected end users?

  • Week on week average logins per user per week by role

  • Histogram of average logins per week per user by role for the last month, 6 months

  • Logins by hour and week day

With local user knowledge, we can look for concering patterns of discontinued use, or of use only by a fraction of expected end users, or patterns of use that are not consistent with point of care and timely data entry. We can investigate and resolve training, local prioritization, IT infrastructure etc. barriers to adoption.

  • For each request, the activity logs store the timestamp, session id, and user id, so we should be able to calculate user logins by this

  • Could cross-reference user again role or provider_role table to group by role

What is the most common use of the system?

Top 3/5/10 next “clicks” or actions, disaggregated by user role.

Determine what most people are actually doing in the system - searching for a patient? Adding a form?

Has to be from logs.

Is there frequent access to view patient dashboards during clinic hours by clinicians?

What do clinicians look at (click on) on patient dashboards?

  • Average unique patient dashboard views per user per week by role, also by patient clincal program

  • What are the “next click” when somene is on a dashboard, disaggregated by role.

Understand if the system is being used to inform clinicians about patient status, possibly at point of care

  • Activity logs track each load of patient dashboard (“/mirebalais/coreapps/clinicianfacing/patient.page”) and included query parameters could be used to determine the patient (patientId) or whether this is the generic dashboard or a program-specific dashboard (dashboard parameter)

Is there regular data entry by data clerks?

Encounter table

  • Patient ID

  • Encounter type

  • Encounter location

  • Provider

  • User who entered the data

  • Submit timestamp

  • Encounter date or timestamp

  • Number of non-blank obs

  • Incremented count up per patient per encounter (1 = first encounter)

  • Incremented count down per patient per encounter (1 is most recent encounter)

  • Week on week average or median gap between encouter date/time and encouter submission.

Identify data timeliness issues that may indicate barriers to the data entry workflow and work with local colleagues to resolve them.

  • Would think this would be done not by looking at the logs, but by looking at the encounter table and comparing encounter_datetime to date_created

  • Could group by encounter type and/or creator

Usage by role and provider typeNumber of logins per role and provider type

Registrations


Is there a regular pattern of patient registration and visits that indicates regular use as the system of record for patient registration.

Patient table with

  • Patient ID

  • Registration date

  • Registration last modified date

  • Death date

  • Weekly new patients registered

  • Weekly % of patient registration records modified

  • Weekly patients marked as dead

Confirm that activity matches information we have about the current catchment area and clinical capacity.

  • Weekly new patients registered could be calculated by looking at registration encounters

  • Would need to research further how/if we could track patient registration records modified

  • We don’t directly track when a patient is marked as dead, so we might need to go to the logs for this

Is use as the system of record for patients sustained?

Covered with patient and encounter tables

  • Average per week of average time live pateint has been in the system (distribution?)

  • Proportion of pateints registered with no further activity (“never active”), patients with activity in the last 10, 5, 1 year and 6 months, 3 months 1 month.

An increase in the time that live patients are in the system, along with other indicators, shows that system use is expanding, and being used for long term tracking of patients. Other patterns may indicate a lot of patient duplication, or registration of only a subset of patients.

  • We’d need to determine what we consider “active” but could likely do this by looking at patient encounters in the DB

Clinical workflow focus

Is the check in feature being used to initiate the patient visit workflow?

  • Check ins - average week on week

  • Check ins with no encounters (or no obs?)

Inconsistent use indicates that there may be other systems (paper, Excel) used to manage patient visits

  • Done by looking at check-in encounters in the database

  • Encounters are group by visits, so a “check-in with no encounters” could be determine by a visit that has only one encounter, a check-in encounter

Is patient data being entered regularly and completely?

Encounter table

  • Weekly encounters, weekly encounters per patient, by type

  • Average obs per encounter that are not blank? by encounter type?

We can see if certain forms or form sections seem to be underused.

  • Weekly encounters can be determined by looking at the encounter table

  • FWIW, obs should never to “blank”, they are only “not there”, so we’d to manually categorize all the potential obs in an encounter (might be tedious, but shouldn’t be difficult)

Is patient data important for continuing care and clinical program being added regularly

Dispensing table

Diagnoses table


Statistics TBD on

  • Prescriptions, dispensing

  • Diagnoses

  • Dispositions

  • Labs

Likely averages per week or per visit, per patient

Determine if these features are being used to the level consistent with other system use, if not, this indicates gaps in implementation where other systems may be used (such as registers) as the system of record for these key aspects of the patient medical record


Are clinical programs being used to track patient status

Program table

  • New program registrations per week, average, week on week

  • Program status updates and program outcome updates, also per week, average, week on week

Determine if clinical program features are being used consistently and matching the scale expected for each program. Inconsistent or low patterns of use indicate that data entry is delayed, likely from a separate system of record.

  • New program registrations can be determined just by looking at the patient_program table and the date_created

  • Status updates would be harder to track via the DB because it only the most-recent changed date is recorded

  • Activity log tracks when a POST is made to a particular program enrollment, but does not track what, if anything, is changed

Data completeness

Is the data entered as completely as expected to:

  • Indicate full adoption as the system of record for patient data
  • Support needed data use
This is very specific to individual forms in the EMR.

Examples:

Determine if forms are entered in duplicate because all data is not available for one data entry session or via a single physical encounter with the patient

Determine if data is complete enough so that indicators are an accurate representation of patients

This comes directly from encounter/observation data from the forms being monitored for completeness.

Data use focus

Note, this excludes use of data from the reporting DB such as for HIV and MCH.


Are people using the system data exports for reporting?

  • Data exports per week or month, by user, user role, and export

Monthly (or less frequent) access to priority exports indicates possible missed potential for timely and internal use for regular monitoring and decision making.

  • We might need to add more logging to properly track this…we track GET requests to the reports page, but it doesn’t look like we capture the date ranges (which are in the POST request)

Are people directly querying the db?

  • Are SQL queries logged?

  • I believe SQL queries are not logged by default, though we could turn this on, but I believe it might be difficult to distinguish direct SQL queries from those the applicant is providing

Are people accessing system data via API?

  • Requests per week or month by user and request

  • What do we mean by “via the API”? The client app accesses data via the API, so might be hard to distinguish from individual people accessing; though if there’s a particular request we are looking for we could likely track that

Data Quality

Is data in plausible range?
  • Are patients in a normal age range? (ie. not 200 years old)
  • Are there pregnant males? 






  • No labels