Metadata Server Management

Why care about managing concepts?

Someday you might deploy more than one implementation, multiple servers, or need a collection of concepts which are different from the CIEL concept dictionary.  There are trade-offs for each of these scenarios:  standalone, master/slave, and central curation.


ProCon
Standalone
  • No dependancy on others, so timing depends on you
  • No need to migrate
  • You are the expert
  • Transparency to you
  • No internet required
  • No benefit to the community
  • No advantages from community knowledge, clinical expertise, and medical terminology
  • You are the expert
  • You do all the maintanance
  • Limits for multiple servers and implementation
Master/Slave
  • Gain expertise from others
  • Create a collection from the best and multiple sources
  • Flexible for multiple servers and implementation
  • Cloud-based
  • Many different mechanisms
  • Time consuming concept management process

CIEL with subscription
  • The community and experts are constantly improving and expanding concepts
  • Cloud-based

  • CIEL dictionary has too much (ie. clinician searches for "malignant" and finds more than 1000 concepts)
  • Requires internet for updates
Central curation
  • The community and experts are constantly improving and expanding concepts
  • Most flexible option for multiple servers and implementations
  • Cloud-based
  • OCL is not ready for collections



Standalone:  This is could be quickly implemented when there's a single OpenMRS server and implementation, but difficult if it grows to multiple servers and implementations.  An example of this is the implementation at Partners In Health/Malawi (APZU).  The concept dictionary is based on a fork of the concept dictionary in 2008 (AMPATH/OpenMRS concept dictionary and later Partners In Health concept dictionary).  It is difficult to benefit from any maintanance and improvements that are done in the CIEL or "golden" PIH dictionary (ie.  ICD10 and SNOMED mappings, duplicate concept names in pre OpenMRS 1.7, etc.)  The one positive aspect is that concepts can be created quickly and without discussion or reliance.


Master/Slave:  There are 3 examples of this:

  • CIEL dictionary is the "master" and your implementation subscribes to modifications
  • Multiple servers are connected via sync module.  Concepts are only updated on the sync parent and changes are replayed on the child servers.   This is used at Partners In Health Rwanda (Inshuti Mu Buzima).
  • A "golden" metadata server has curated concepts for various implementation.  Concepts have been managed over time, additional concepts are added from CIEL and locally.  All concepts are managed here and subsets are packaged here using mds or groovy and exported to individual implementation. 

Partners In Health implementations (Haiti, Liberia, and Lesotho) use centralized concept management and the "golden" server.   The "golden" concept server is maintained in the cloud and used for PIH concept management with a curated set of concepts.  These steps outline the process for creating mds packages for implementations along with the "PIH Concept Management workflow" figure:

  1. If a concept is needed, look if it already exists in that local OpenMRS implementation.
  2. If it doesn't exist in local implementation, check in the "golden" PIH concept dictionary.  The "golden" PIH concept dictionary is deployed on an OpenMRS 2.1.3 server in the cloud.  
  3. If the concept exists on the "golden" concept server but is missing concept terminology (ICD10, SNOMED, etc), check the CIEL concept dictionary.   These mappings can be added directly on the "golden" server OR by creating an metadata sharing (MDS) package from the CIEL dictionary and imported into the "golden" server.  MDS allows that concept data (sources, classes, datatype, mappings, names, etc) can be access and ignored/merged.
  4. If the concept doesn't exist in the PIH "golden" dictionary, use mds to get the concept from CIEL.
  5. If the concept doesn't exist in the CIEL dictionary, propose the concept to CIEL management.
  6. If the concept doesn't have the appropriate locale (ie. French, Haitian Kreyol, Spanish, etc), add to the PIH "golden" server OR propose to CIEL. 
  7. If the concept is specific for PIH, create the concept directly on the "golden" concept server.  For example, there might be a list of Haitian insurance companies.  This is not proposed to the CIEL dictionary.


Central Curation:  see OpenConcept Lab (OCL) and CIEL


Guidelines for concept management

  • Use one central OpenMRS server for "golden" metadata (concepts, etc)
  • When possible, use the CIEL dictionary for new concepts instead of recreating a new one.
  • Use the Open Concept Lab (OCL) or Maternal Concept Lab (MCL) for viewing the CIEL, PIH, AMPATH dictionaries.  
  • Use Metadata sharing (MDS) for getting existing concepts.  
    • Use MCL to generate an export package with concepts for import into your local "golden" server  OR
    • Use the CIEL dropbox to get the latest mysqldump of the CIEL concept* tables.  Install this on a local OpenMRS server along with the MDS module.  Export packages from this server for your local "golden" server.
  • Create a concept source for this server (ie.  Rwanda, PIH) where the concept_ids will be used for mappings (ie.  PIH: 12)
  • Add unique mappings terms for use with forms and reports (ie. "PIH: 12", "PIH: Anemia", "CIEL:123456", "Rwanda: 388").  The names must be unique. See https://openmrs.atlassian.net/wiki/display/docs/Concept+Terms+and+Mappings
  • These mappings should have a concept reference map type of "SAME-AS".
  • Sentence case (ie. "Easter egg hunt") is preferred for concept names. 
  • Each concepts on a Metadata Server should have one or 2 maps (ie.  "RWANDA: 12" and/or "RWANDA: Anemia")
  • Add terminology mappings where useful (ie. SNOMED, ICD10, RxNORM, ICPC2, LOINC). These are helpful for standardizing diagnoses, procedures, and billing.  These are already populated in CIEL and PIH.
  • Locale - Concepts should be in appropriate language:  English, French, Spanish, Haitian Kreyol, Kinyarwanda, etc.
  • Validate the following with the validation module:
    • concepts
    • concept names
    • concept description
  • Confirm that all user fields in concept* tables have existing users.  A variety of fields (creator, voided_by, changed_by, and retired_by) are on the concept* tables and correspond to the user_id in the users table.  If concept* tables have been directly updated via mysql (mysqldump and source), then users might not exist for concept* tables.
  • Cleanup and possibly delete unused concepts
  • Delete or retire duplicate concepts.  If a concept is retired, it can still be used in the system for past or future obs; however, it will not appear by default in concept dictionary searches or htmlform dropdowns.  Retiring a concept doesn't cause any problems.  Deleting a concept when it's used is a problem.  If a form uses the concept, it will break the form.  If an obs used the concept, that will also produce an error.
  • Check for appropriate classes and datatypes 
    • diagnosis, symptom, finding, program, workflow, state or misc should have a datatype of N/A
    • LabSet, MedSet, and ConvSet should have set members
    • questions should not have a datatype of N/A

Guidelines for htmlforms

  • Maintain the latest forms on the metadata server
  • Use mds to create htmlform packages which include all the concepts, program, drugs, locations, etc.
  • Forms and reports should use mappings to a concept dictionary (ie. CIEL or Rwanda) and not uuids or concept_ids
  • <htmlform>
        <macros>
                    <!-- Hemoglobin 3ccc7158-26fe-102b-80cb-0017a47871b2 21 -->
            <at:var at:name="HEMOGLOBIN" />=RWANDA:21
        </macros>
     
       <p>Hb: <obs conceptId="$<at:var at:name="HEMOGLOBIN" />" /></p>
    
    

    Add concept mappings within macro in the htmlform

Tools

  • Groovy module - Can be used with sync or non-sync environments to make changes to the OpenMRS database. The groovy modules allows for groovy scripts to be saved and managed within OpenMRS.
  • Metadata Sharing (mds) - Used to copy all metadata (concept and forms) between systems. Works well with sync and helps to propagate changes to all servers.
  • Validation module - Checks the concept dictionary for duplicate concept names, concept descriptions with null descriptions, Locations, etc.
  • Github repo for MySQL and groovy scripts  
    • Concept cleanup
    • Check for duplicate mappings (when a reference term from PIH, CIEL, or Rwanda should not be on multiple concepts)
    • Deleting concept terms, maps, sources (ie. local, AMPATH, etc)
    • Adding new concept terms, maps, source (ie. Rwanda)
    • Renaming a concept source (ie. MVP/CIEL)