Child pages
  • Data Integrity Workflow Module
Skip to end of metadata
Go to start of metadata

Primary mentor

Jeremy Keiper

Backup mentor

Ada Yeung

Assigned to

Harsha Kumara

Abstract

The Data Integrity Module allows users to run integrity checks against the database and identify problems for review. Each integrity check has several records with a status of New, Ignored or Voided. Users browse through the records, look up paper forms and verify the data. The only way to remove a record from the listing is to mark it as Ignored or re-run the integrity check after fixing the problem (at which point the record is voided because it is no longer an error). It's very hard to manage and fix large number of integrity checks with having thousands of record for each check.Data Integrity Workflow Module allow users to efficiently and collaboratively manage data integrity checks.This module enable manager to assign records in a particular integrity check to users(can be a analytic people) for analyse the issue with the record and fix it.

Objectives

  • Allow a manager to assign individual results or entire groups of results to another user for follow-up
    One of the major feature of this module is to assign  single or group of records of each integrity check to users(can be analytic people) for follow up and fix the errors in the records.Data Integrity Module not having a option to assign  records.Because of this only few privilege people can be access and fix the errors in each a integrity check by going through all the records.This is not feasible approach for OpenMRS instances which having large data capacities.There may be thousands of records results in a integrity check which having numerous errors.One of main objective of this module is to collaboratively solve the errors in a database by dividing task to responsible people.

  • Allow a user to log in and see just his or her assigned results
    After manager assign records to a users,it's required users to view their assigned checks for further investigation for fix errors.User will receive a table of assigned records.Through this web interface user will select preferred reports to work on.

  • Allow a record to move through workflow stages, such as In Progress, Waiting on Information, Verified, etc. (this workflow can be provided by implementer community members)
    Managers need to keep track of the stages of the records which are assigned to follow up users. User can be change the status of a record to a another stage to indicate the he/she progressing on that record.
  • Allow managers and follow up users to comment on records
    Sometimes managers or the follow up users required to comment the details about a particular record. Follow up users will have lot's of issues in the process of solving integrity problems in a database which are can be express through the comments.May be managers also want to highlight some description about a record through a comment.

  • Provide summaries for seeing workflow stages and progress over time.
    It's very useful to have summaries and reports about a record. These summaries and reports will include assigner changes,stage changes and other required details. 

Module functionality

Data integrity workflow module enable very flexible environment for manage the integrity records within openmrs databases. When someone use data integrity module which is the parent module of this project, he may experience with large number of integrity checks followed by number of integrity check records per each check. Within very few administrative people, it's very hard to manage all the integrity workflow records spread within all the integrity records of the system. So this module enable managers or administrative people to assign people for each records to follow on and fix particular records. Record assign capability and privilege will only be granted for managerial persons in the system. Followers can change the working status and comment on particular records for get further clarifications on the process of fixing the particular integrity record.

      System Privileges

    • Manage Record Assignees //for managers privilege
    • View Record Assignments //for followers privilegSystem functions

    System Functions

    • Managerial level user can view integrity records and assign results of records to low level users to follow up and fix integrity violations associated with each records.
    • Managerial level user can remove assginees from assigned records
    • Managerial level user can comment on particular record and can change the current working stage of a particular record
    • Managerial level user can view record wise summary and also whole check wise summary
    • Managerial level user can add and edit workflow stages and keys associated with integrity checks
    • Follow up user can view set of assigned records to him
    • Follow up user can comment on a particular record and change the current workflow stage of a record

System Views

Module is consist of several views to achieve the required functions mention in above, In following sections will contains the set of functions associate with each view and how it can be use for achieve the required functionalists.

Administrative links for manager


  • In this view managerial level person views set of links to goto different views.In Manage Records link, it allow managerial users to see the available integrity checks in the system

  • Change Check Key link will direct to page which shows the available checks with keys to change keys of selected checks.
  • View records link allows to view the assigned checks for logged user
  • Add Workflow Stage link allows to users to add new workflow stages

Low level user links

  • Low level user will only see the View Records view which allow them to see the assigned records for them

Integrity Check View

  • Managerial person only can view this page by clicking Manage Record link
  • This window shows available integrity checks in the system with associate key for each key which will be default key at initially
  • By clicking one of check,user can view the associate checks for a particular check

View records for a check


  • In this view managerial user can view the available records for a selected check
  • This view allows managerial user to assign set of records for a user by simply tick the check boxes and then clicking assign button in the page.It will popup a dialog which users can enter the name assignee to assign the selected or all the records within a check
  • As same way managerial person can remove users from record by tick the check boxes and hit unassigned button in the page
  • By clicking a the username link,managerial person can view the set of records assign for user for a currently viewed check
  • history link allows users to see the set of actions perform within a record from the beginning

Assign User View


  • After managerial level person hit the assign button, pop up dialog will appear to enter the username of a user need to associted with selected records
  • Another option is in the popup dialog which allows to managerail person to choose assign selected or all records for user

Check Summary View

  • This is the second tab in the integrity check records view which enables several functions for managerial level person
  • In this view managerial level person can view summaries of a check by assignee, stage or record status. 
  • Managerial level person can identify which records assign to users,how many records in each workflow stage and status of the records.
  • Managerial level person can simply click links and navigate to pages which  shows the records associated with each category in summary view
  • In later custom query view enable managerial level person to filter out set of a records within user specified constraints.For eg: if managerial level person want to list the set of fixed records within specific time duration by a particular user, it can be done through this query view.

Record History View


  • This window is the page which user navigate after clicking a particular record's history link
  • Managerial level person can view set of operations perform upon a integrity record from the beginning

Integrity Record View


  • Simply clicking the record id of integrity record will be naviagated to this view
  • This view is enable managerial user to normal low level user to discuss and change the status and stages of records
  • For managerial level person, module will be given privileges to reassign user to particular check.This function is can be achieved by change button next to the username in the view.But low level user will NOT have this button enabled for change the assignee of a record.
  • Low level user can comment and change the status and working stage of the record
  • Managerial level person also can be done above mentioned operations and addition operation which allows to change the user.

Change Check Key  View


  • Managerial level person can simply click upon the key particular record and change the check key in the navigated page.

Add Workflow Stage

  • This view allows managerial level person to add a custom workflow stage to the system

Low Level User View 


  • For normal integrity record follow up user will be see this view, after follow up user click the View Records link,user will be directed to this view which shows the all assigned records for the user
  • Follow up user can click the record id to enter into full integrity record view

These are the set of views in the module which provide the required functionality specified in the module.

Final evaluation demonstration

               

Requirements

Initial version of the module will be compatible with OpenMRS 1.8.x

Use Cases

UI Mockups

  • Manager Views
  • Analytic user's view

E-R Diagram


Storyboard


workflow

Project Plan

Project Timeline

May 27-Jun17

  • Discuss about the final project deliverables with the mentors and update the project requirements documents according to it.
  • Improve the user interfaces for manager and user while researching about possible approaches.
  • Getting more familiar with hibernate and Spring framework.

Jun 17-Jun 25

  • Finalize the user interfaces in the project
  • Designing and creating necessary table schemas
  • Explore more about writing module in openMRS

Jun 25-Jul 29 

Implement the first phase of the project

Goal-Finishing manager view implementations

  • Creating the table structures through liquibase 
  • Creating backend services with use of the dependent data integrity module
  • Implement the manager view
  • Creating pages for view report summaries
  • Finalise the manager view implementations
  • Get a initial start of implementing user view
  • Partially implementing the user views

Jul 29 -Aug 2 

  • Mid evaluations 

Aug 2- Sep 9

Implement the second phase of the project

Goal-Finishing user view implementations and finalise the project

  • Polishing first phase implementations according to feedbacks
  • Implement the user view front end pages
  • Perform necessary changes at the backend
  • Finishing user view implementations
  • Writing test cases and test the project for verify its’ functionality
  • Creating wiki pages for the module

Sep9-Sep16

  • Extra week for finishing and wrap up project work

Sep16-Sep23 

  • Final evaluation

Extra Credit

Resources

https://github.com/openmrs/openmrs-module-dataintegrity

https://modules.openmrs.org/modules/view.jsp?module=dataintegrity

Data Integrity Module 

44 Comments

  1. Harsha Kumara, so excited to have you to work on DINT.  (smile)

    One of the suggestions from AMPATH Data Quality Team is to include UI options of specifying date range and clinic encounter locations without updating such in the sql queries.  

     

    1. Thanks for the comment Ada Yeung. If you have any suggestions on what kind of details need to be in a report of a record please let me know.So I can structured the database according to your suggestions. (smile) .

    2. That is a request that affects the actual integrity checks, but this project is intended for enabling workflow ... assigning individual results to users for follow-up, and managing the entire lifecycle of those assignments.  I recommend filing a ticket for your request.

      Harsha Kumara is asking for your opinions on what information would be most useful to a manager like Lelei or Sheila at AMPATH on a dashboard from this module.

  2. For "Provide summaries for seeing workflow stages and progress over time", is there a mock-up?

    AMPATH would like to be able to generate different summaries, such as

    • by type (reviewed and resolved/corrected, reviewed but no need to take actions (since the data is correct after being investigated manually), haven't been reviewed yet, etc.
    • by user
    • by defined time period based on reviewed date
    • any of the above combinations
  3. Thanks a lot for providing what need to be in summaries. I'll create a mock up according to these requirements.Then we can discuss further what need to be improved.

    Jeremy Keiper Ada Yeung I would like to know the feedback on views to a manager and a analytic person.

    Manager Views

    1) Manager have the view of listing all the integrity checks and their details such as failing or not.

    2) Then Manager can select a integrity check and proceed to view the records of it.In records view, manager perform tasks such as assign user,remove user and from this view the current status of a records along with current assignee stage

    3) Each record have option to generate summary

    4) Manager can select a record and proceed to the detailed  view of a record where manager can perform tasks such as change a stage of analytic person and comment

     Analytic Views

    1) Has a view of listing all assigned integrity checks 

    2) Analytic person can select a record and proceeding to the detailed view of the record where user can perform tasks such as change working stage,comment 

    UI mock ups for above views(instead of summary mock up) are listing in the page and if I can have a feedback on them,it will be really helpful me to finalize  views of the project.  

  4. Thanks, Harsha Kumara.  It would be helpful to first define the difference between Manager role and Analyst role before we spec out what options/features should be included for each of them.

  5. Users of manager role have the permissions to assign a integrity check results for people from analyst role to work on and fix the errors.

    Users of analytic role only have the permission to view the assigned records and work on fixing the errors in each record.

    Hope now clear about these roles. (smile) 

  6. Very clear.  (smile)  

    For manager role, is it possible to include summary view of what I posted in previous comments (i.e. to know how many have been resolved, ignored, and hasn't started/reviewed yet)?

     

  7. Yes, Manager role will have those privileges.(smile). Actually Analytic people just work and comment on their assigned records.

  8. Perfect!  Let us know if you need to chat on reviewing the mockup.  (smile)

  9. Thanks a lot Ada Yeung. Catch you through IRC (smile).I think it's better to arrange a discussion for finalize the mock ups and other functions. Jeremy Keiper WDYT?

    1. I think we have enough new features defined, just need to specify which ones to implement and clarify the mockups.  It would make sense to review them with Ada soon.  Perhaps we can develop a storyboard explaining each user's specific screens and how they are used?

  10. Let me know when you guys would like to get together to review.

  11. Hi Jeremy Keiper  ,I have added the storyboard.It's my first storyboard (big grin).If something to need to be change there,i'm happy to know it and correct it. I need to finalize the UIs. I already have started the implementation. Let's arrange a discussion and confirm the UIs.(smile)

    Thanks,

    Harsha

  12. My schedule of these two weeks is very busy.  Here are my availabilities of this week:

    • June 18th (10am to 11am EDT)
    • June 19th (10am to 11am EDT)

    If those timeslots don't work, let me know and I will send my availabilities for next week.

    Cheers!

  13. Thanks Ada Yeung. Tomorrow i'm free at that time.Usually i'm mostly anytime available online after 7 PM GMT(9.30 AM EDT right?). Jeremy Keiper are you available tomorrow at that time? 

    1. Great.  Hope that timeslot will work for Jer.  (smile)

    2. Oh wow, I totally missed this ... and Ada even mentioned it this morning.  So sorry!  I had a meeting from 10am-11am today so I would not have been able to make it.

      We talked about having a late-Thursday or early-Friday meeting.  To have Ada attend, I think it would be wise for it to be early Friday (EDT).  Harsha Kumara, will that work for you?  Ada Yeung?

    3. ... actually, we had not talked about that.  It was with the project I am a backup mentor for.  Again, sorry for the confusion I am causing.  Ada Yeung and Harsha Kumara, please agree on another time and I will chime in with my availability.

  14. Hi, Jer, it was supposed to be yesterday 10am EDT (not this morning).  (smile)

    My schedule for the rest of this week is pretty bad.  I don't have any timeslot opened this Friday.  The only timeslot I have is 1 to 2pm EDT tomorrow (June 20, 2013) if that works for you both.

    1. That time slot works for me.

      1. I'm okey with that time ...(smile). Let's have the meeting tomorrow.How can I arrange the meeting? Is it like a hangout or skype call? 

        1. We can use https://connect.iu.edu/ada so that we can see your screen.  (smile)

        2. Let's try to use connect.  We can either use Ada's dedicated conference room or one from OpenMRS (like omrsdf).  Make sure you are on IRC at that time and we will share the link.

          1. Thanks a lot Ada Yeung and Jeremy Keiper. I'll be online at IRC on that time for sure. 

  15. Hi Jeremy,

    I have bit of confusion in the workflow diagram.We have seperate stages for a record and per assignee right? By default record stages are New,Void,Ignore. And on the otherhand record assignee has stages Assigned,Waiting for information, Verified,On Progress. This workflow diagram shows transitions from these two set.Am I correct?

    Since I depend on the integrity module,I just having a reference to the integrity check,and records from that module.

    Question 1 

    By default record status are change from new to void or void to new when someone run the integrity check. I may need to introduce new status such as resolved. But since these status are not yet available in the integrity module,what will be the best approach?

    Question 2

    User need to change the record status from my module right? For example user may need to mark record as resolved after some workaround.But I think this will be effect from the point where integrity check runs.Not sure?

    Question 3

    When assignee fixing the record he/she need to go to the database level and fix it.And he may mark as record is verified but he might not know whether issue is fixed or not right? (It will know after someone runs the check)

    Thanks,

    Harsha


     

    1. The workflow of a record in Data Integrity Module is very straightforward, just like you stated.  You should consider this to be an additional workflow that is informed by the original one.  I recommend maintaining workflow for a DINT record within the new module and somehow hook into DINT to receive event notifications upon a change to a record's state.  You might be able to use the Event Module for this without changing DINT, but do not be afraid to add code to DINT to support your needs.  Just be sure to create a ticket before you begin work on DINT.

      Your assumptions in questions 2 and 3 are correct.

  16. Thanks Jeremy, I have done a search of workflow engines.but hardly find any which suits to this.Most of giving a BPM based workflow engines as suggestions.If you have come across a engine please let me know?I'm now going with my alternative approach.

    I'm planning to separate the workflow module completely with the data integrity module. I'll add necessary changes to data integrity module. Now my major concern is updating the record status of a integrity record. I'll keep a status with my workflow record.But when integrity check runs in the data integrity module.It need to be updated. As you suggested I'll look at the of using Event Module for this project.

    If event module suits,then most of my concerns will over.

    Thanks,

    Harsha

  17. Hi Jeremy Keiper and Ada Yeung,

    I have attach a manager view UI diagram. Any comments about it?

    Jeremy Keiper I'm bit concern about using the record id in the manager page. For a integrity result, it assign with a UUID and a table id.I have use table id here.But I think neither of them is good to use as displayed record id. So for a viewing purposes only,I can do some trick to uniform the record ids per check. Do you have any thoughts?

    For a status changing of a record, I think it's good to introduce some few more status to the data integrity module. If I can have a way to listen for the "integrity check run" method execution,then I can act according to it. I have look at the Event Module Usage,but it bit of lacking of resources.I will dig more deeper and raise a question in dev list if I couldn't find solution through event module.  I would like to know the opinions of you about it? 

    Thanks,

    Harsha

  18. Hi Jeremy KeiperAda Yeung,

    I have progressing on with the module development.Shall we arrange a quick meeting for discussion on the implemented front end UIs of my module.

    Jeremy seems like I can use event module for listen the changes in the integrity check. If that succeed,I can do record status changes according to it. (smile) . I have tried with the event module,but some exceptions throws out.I'll dig more deeper and see how it can be solved.

    Since I'm having reading week at my university,I'm available throughout the weekdays.Any time both of you free,We can have the meeting.

    Thanks for the help of both of you.

    Harsha, 

    1. Hi, Harsha Kumara, I am available today at 1:30pm if it works for you and Jeremy.  Thanks!

      1. Jeremy Keiper will you available at the time that Ada mentioned? (smile)

        1. I don't think Jeremy got a chance to read his email.  We can try to see if he can meet tomorrow at 2pm EDT.  Thanks!

  19. Hi Ada YeungJeremy Keiper,

    I have implement the most the parts of summary views. Shall we arrange a quick meeting for review the things I have done. I'm mostly available these days. Let me know the free time slots of both of you.So we can arrange the meeting. (smile).

    Thanks,

    Harsha

    1. Thanks for sharing update, Harsha!

      I am opened on the following timeslots:

      • July 31 (Wednesday):  11am to 1pm EDT
      • August 1 (Thursday): 1:30pm to 3:30pm EDT
      • August 2 (Friday):  1:30pm to 3:30pm EDT

      Hope one of those would work for you (Harsha Kumara) and Jeremy Keiper.  Thanks!

      1. Thursday and Friday are good for me, but Wednesday is not.

        1. Cool.  Harsha, you pick the actual time.  (smile)

        2. I'm okey with Thursday. Let's make it on that day (smile).

          1. 1:30pm EDT Thursday then.  Thanks!

  20. Hi Jeremy Keiper and Ada Yeung,

    Since I have mostly done with further improvements.Can we have a quick discussion on Thursday or Friday? I should be available anytime both of you prefer.

    Thanks

    Harsha

    1. Harsha,

      Here are my availabilities for Thursday and Friday:

      Thursday: 2pm to 3pm EDT

      Friday: 2pm to 3pm EDT

      Thanks!

      -ada

       

      1. Hi Ada Yeung,

        Jeremy also available in the Friday: 2pm to 3pm EDT timeslot.So we can have the discussion on that time.

        Thanks,

        Harsha

        1. Cool.  Will chat at 2pm EDT.  Thanks!