Have you implemented OpenMRS? Please participate in the Implementation Site Survey. If you already have, thank you!
Child pages
  • Sync 2.0 Architecture Overview

Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

Sync 2.0 aims to use FHIR to achieve synchronization between a network of nodes. It assumes one master Parent node and multiple slave Child nodes connected to the master. Parent. Each child can also act as a a Parent, which allows a multi-level Sync network. In this topology, the synchronization can be done in two directions: master Parent to slave Child and slave Child to masterParent. Sync 2.0 will support both independently, as one-way sync is being used in the field currently by implementations adopting Sync 1.0. In both models the methods of communication will be generally the same - an atom feed will be read, then data will be retrieved and pushed, preferably using FHIR. If FHIR representations are not available for resources being transmitted, we will fall back to OpenMRS REST representations.

All synchronization will be initiated by slavesthe Childre, independent from the direction that data is transmitted.

...

The diagram above gives an overview of a typical Sync network composed of a master server Parent server and multiple child Child nodes synchronizing with it. 

Pull from

...

Parent

In order to keep their data in Sync, Slaves Children should read data from the mastertheir Parent. This will be a three step process:

1) The slave Child reads through the atom feed exposed by the master and its Parent and filters out events it is not interested in

2) For events that it is interested in, the data is pulled from masterParent

3) The data is persisted by the slaveChild

Slaves Children will be reading the feed exposed by mastertheir Parent, event by event. The master Parent will expose a complete feed of events - slaves will Children will be in charge of filtering and pulling only the resources they are interested in. When they come across an event that they interpret as requiring synchronization, they will trigger retrieval of the object using the client facade, which will deal with retrieving the object using the correct method of transmission based on the data type - either by delegating to a FHIR client, a REST client or a different client injected by implementation.

...

Metadata retrieved through the REST API will have to be inserted using conventional methods - best if it directly interfaces with OpenMRS repository classes to insert the metadata into the db.

The slave Child will keep track of what was synced based on its marker in the masters feedParents feed, i.e. the marker should not be moved when a connection error happens. Retries will be controlled by configuration, it should be also possible to make the slave proceed a Child proceed through the feed even if a record fails to synchronize. Since the audit UI will allow manual retry of failed transactions, administrators will be able to resolve conflicts and retry transactions manually.

Pulling data from master can from a Parent can be enabled/disabled independently from pushing data to master Parent in order to allow implementations the possibility to configure one way synchronization.

The graphic below shows the steps required for the pull model:


Push from

...

Child

The push model will be used for pushing data from slaves Children to the master Parent node. Push is also done in three steps:

1) Read own data feed, filtering out events that don't need to be synchronized with masterParent

2) For events that need synchronization, read them from local endpoints either using FHIR or REST services

3) Push the representation to masterto Parent

Slaves Children will publish their own atom feeds, similar to the masterParent. Each slave Child will read it's own feed and based on what it reads from it, the slave Child will push the objects to master Parent for synchronization.

After reading the feed, Sync will have to decide whether this is data that should be synchronized with masterParent. If an object in the feed is supposed to get synchronized, the slave Child will first fetch it from its own FHIR server (or REST if it’s metadata) through localhost  and forward it to masterthe Parent. This will allow simplicity with Sync 2.0 code - the mechanism for reading the slaves feed Children feed will be similar to the master Parent one. This might also allow multi leveled Sync - since slaves Children publish their own feeds, they can also act as masters Parents for lower level slavesChildren.

Error handling is similar to the one described above for push.

Pushing data to master Parent can be enabled/disabled independently from pulling the data from master Parent in order to allow implementations the possibility to configure one way synchronization.

...

The diagram below illustrates a general overview of the Sync module architecture. Note that components might be simplified and in the real implementation split into multiple pieces. It is shown from the more interesting perspective of a slaveChild. Note that each slave Child can also act as a Sync masterParent.


Gliffy Diagram
nameSync 2.0 Arch

Things to note:

  • Each slave Child can also act as a masterParent
  • A sync module has two feed reader instances - the local feed reader and the master Parent feed reader. The local feed reader is reading the local feed and triggers a  push to masterParent. The master Parent feed reader reads the master Parent feed and triggers a pull from masterParent.
  • SyncPushService and SyncPullService put together the business logic for pushing and pulling
  • The SyncAuditLogger is responsible for maintaining an audit of Sync operations in the database
  • The ClientFacade delegates to either the FHIR client or a REST client, based on the data type and whether a FHIR method is possible. The underlying transfer operation should be transparent to clients.
  • SyncPullService uses either FHIR services from the FHIR module or database persistence to save the data locally

...