Wiki Spaces

Documentation
Projects
Resources

Get Help from Others

Q&A: Ask OpenMRS
Discussion: OpenMRS Talk
Real-Time: IRC Chat | Slack

Documentation

Page tree
Skip to end of metadata
Go to start of metadata

From this mailing list thread...

On our last few design calls we've been working through the generic AttributeType mechanism that we're introducing in 1.9. (

Error rendering macro 'jira'

Unable to locate Jira server for this macro. It may be due to Application Link configuration.

)
We wanted to summarize the current state of things. Some of this is in trunk, but some is refactoring Darius is doing. We're especially interested in thoughts about how these classes should be linked together with parameterized types, since we're not convinced we've gotten that right:

/**
 * Capable of converting T to/from a String that can be persisted in the varchar column of a database. 
 * E.g. a date would be stored as yyyy-mm-dd and an image might be stored as a uri pointing to a 
 * PACS system. Also capable of validating T, e.g. so we can use a plain java.util.Date to represent 
 * "date-in-past" but limit its possible values.
 * Defines a String "datatypeHandled", e.g. "date", "regex-validated-string")
 */
interface CustomDatatypeHandler<T> {
  T fromPersistedString(String);
  String toPersistedString(T);
  void validate(T);
  String render(String persistedValue, String view);
  // can be configured with setHandlerConfiguration(String);
}

/**
 * holds the definition of a custom datatype (which is handled by a handler of the above interface). 
 * For example VisitAttributeType and GlobalProperty (we're able to do typed GPs trivially now)
 */
interface CustomDatatyped {
  String getDatatype(); // required
  /**
   * if specified, will be handled by this specific CustomDatatypeHandler, otherwise it will be handled 
   * by the default handler for this thing's datatype
   */
  String getPreferredHandlerClassname(); // optional
  String getHandlerConfig(); // optional
}

/**
 * for user-defined extensions to core domain objects, which would be handled by adding custom 
 * database columns in a less generic system. E.g. VisitAttributeType implements AttributeType<Visit>
 * datatype/handler specified via CustomDatatyped superinterface
 */
interface AttributeType<OwningType extends Customizable> extends CustomDatatyped, OpenmrsMetadata {
  Integer getMinOccurs();
  Integer getMaxOccurs();
}

/**
 * trivial implementations of AttributeType (via BaseAttributeType)
 */
class VisitAttributeType
class LocationAttributeType
class ProviderAttributeType

/**
 * Implemented by domain classes that may be customized by the user via custom attributes. 
 * E.g. Visit implements Customizable<VisitAttribute>, Location implements Customizable<LocationAttribute>
 * Has convenience methods for dealing with a collection of attributes, of different types:
 */
interface Customizable<AttrClass extends Attribute> {
  Collection<AttrClass> getAttributes();  //includes voided
  Collection<AttrClass> getActiveAttributes();  //non-voided
  void addAttribute(AttrClass);
  void setAttribute(AttrClass);  // voids other attributes of the given type
}

/**
* holds a value managed by a CustomDatatypeHandler. E.g. VisitAttribute, GlobalProperty. 
* Any implementation of this has a corresponding CustomDatatyped implementation. "persistedValue"
* is a String suitable for persisting in a db varchar column; "objectValue" is what you'd want 
* to work with in the API.
*/
interface CustomValue {
  String getPersistedValue();
  void setPersistedValue();
  Object getObjectValue();  // don't remember why this isn't <T>
  void setObjectValue(Object);
}

/**
* value corresponding to an AttributeType (which defines its datatype, whether it's 
* required, whether it can repeat, etc), e.g. VisitAttribute corresponds to VisitAttributeType
*/
interface Attribute<OwningType extends Customizable> implements CustomValue, OpenmrsData {
  OwningType getOwner();
  void setOwner(OwningType);
  AttributeType<OwningType> getAttributeType();
}

/**
* trivial implementation of Attribute (via BaseAttribute)
*/
class VisitAttribute
class LocationAttribute
class ProviderAttribute

/**
* this is interesting in that it both defines a custom datatype and stores its value
*/
class GlobalProperty implements CustomDatatyped, CustomValue

Comments welcome.

Roger's notional x-ray object

Radiology Document

Roger's proposal

These use cases all rely on the custom data service object and handler object described below. These three use cases show the way the three different uses of custom datatypes interface with the cdso and handler.

Use case 1 Use case 2 Use case 3

Roger's notes on the custom data service object

These are work notes, see below for the final answer

Interface with the custom datatype consumer (settings (formerly Global Properties from platform 1.8 downwards), object attribute, complex obs)

  • one object per consumer – in attribute context, this means one per property type; in the settings (formerly Global Properties from platform 1.8 downwards) context, this means one for each property that is being displayed
  • constructor() – default handler, handler config="", minOccurs=0, maxOccurs=1
  • constructor(String handler, String handlerConfig, int minOccurs, int maxOccurs)
  • the consumer loads the custom datatype with all its values and receives them back via save
    • standard interface
      • data is exchanged through a List<CustomDataRow>; deleted objects have newValue=null, new objects have oldValue=null

        public CustomDataRow {
          public String key;  // optionally set by/returned to consumer
          public String oldValue;  // set by/returned to consumer
          public String newValue;  // returned to consumer
          public String voidReason;  // returned to consumer
          public Object data;  // for internal use by custom data service object
        }
        

        The list is copied to a local list on load and then to a temporary list before returning from save; changes to the list by the consumer have no effect and the data object is never returnd to the customer

      • load(List<CustomDataRow>) (empty indicates no values exist)
      • save() : List<CustomDataRow> throws DataError Exception (empty indicates no values exists and none were added; underlying data is saved)
      • customerIterator() : ListIterator<Integer>
    • alternate interface where maxOccurs=1 (throws UnsupportedOperation Exception if maxOccurs != 1)
      • load(String) (null indicates no value exists)
      • save() : String throws DataError Exception (null indicates not set or voided; underlying data is saved)
      • getVoidReason() : String (returns null if not present)
    • both interfaces
      • cancel() (any changes not previously saved are abandoned and the custom data service object is cleared)
      • isDirty() : Boolean (whether any changed have been made to the list)
      • validate() : List<String> (each element is validated and the multiplicity is validated; any errors found are returned as messages; if no errors are found, returns and empty list and dirty is cleared; calling validate before saving guarantees no error will be thrown)
      • cancel() (changes are discarded and loaded data is cleared)
  • the custom data service object delegates handler calls to the handler

The Custom Data Service object as a data service

  • differs from other datatype services
    • may delegate to other datatype services
    • may have load-on-demand for large objects
    • may delegate to external data sources
      • may need multiple strategies in get/set methods if allocation of data elements between external and internal services changes depending on the external service
  • serves the consumer as a data service for the custom datatype
    • uses integer interface to select an element (=0 after load)
      • size() : int
      • index() : int
      • index(int) : int throws DataError Exception (sets and returns index, throwing error if > size()-1
      • getTitle() : String (exposes the short text representation of the indexed)
    • all elements may be get/set, but UnsupportedOperation Exception may be thrown by non-public or read-only elements
      • implemented by calls to getX(null) or setX(null, parameters), see below
  • serves the handler as a data service for the custom datatype
    • exposes the list iterator to the handler
    • all elements may be get/set, but UnsupportedOperation Exception may be thrown by read-only elements
      • implemented by calls to getX(this) or setX(this, parameters)
  • serves external data sources as a data service for the custom datatype
    • all elements may be get/set, but UnsupportedOperation Exception may be thrown by read-only elements
      • implemented by calls to getX(this) or setX(this, parameters)

Interface with the handler

  • The data service object initializes the handler object with its config
  • The data service object maintains the iterator and calls validate() for each element as part of its own validate method
  • The handler delegates to the data service object calls to the consumer service or other object services to get necessary information
    • Example: get all possible answer concepts to a question concept extracted by the handler from handler config
    • Example: get all values of this element currently existing in the DB
    • Example: get the birthdate of the patient to whom this attribute belongs

Interface for external data sources

  • Any connection setup such as logging into a remote service should be done in the constructor; it should be ready to go when the data service object references it

Use case 3 seq diagram

Roger's final design

Custom Datatype Object Model

Attribute/Method

Description

CustomDataService Interface

* handlerName : String

Handler to use to render

* handlerConfig : String

Handler config to use to render

* minOccurs : Integer

Minimum number of values (>0)

* maxOccurs : Integer

Maximum number of values (0=infinity)

- handler : CustomDataHandler

Handler instance based on Handler

- data : List<CustomDataRow>

Holds a set of values for a single datatype

+ loadAll(data : List<CustomDataRow>)

Loads a set of CustomDataObjects

+ saveAll() : List<CustomDataRow>

Saves changed data and returns newValues to persist or voidReasons, throws data error if invalid

+ load(data : String)

Loads a single oldValue, throws UnsupportedOperation exception if maxOccurs != 1

+ add(data : String, key : String)

Adds an oldValue and key; clear must be called first if not new

+ save() : String

Saves changed data and returns newValue to persist, throws UnsupportedOperation exception if maxOccurs != 1, throws data error if invalid

+ getVoidReason() : String

Gets voidReason for singleValue, null if not void, throws UnsupportedOperation exception if maxOccurs != 1

+ clear()

Resets all data objects discarding any changes

+ isDirty() : Boolean

Returns true if there are changed objects

+ validate() : List<String>

Validates each data item and checks multiplicity, returns messages if errors are found

+ render(style : String) : Object

Delegates to handler

+ render(style : String, index : Integer) : Object

Delegates to handler

+ getScript(): String

Delegates to handler

+ getHTML(name : String)

Delegates to handler

+ getProperties() : Properties

Delegates to handler

+ setProperties(p : Properties)

Delegates to handler

+ size() : Integer

Returns number of values

+ getElement(name: String, index : Integer) : Object

Returns named data element of indexed value => getElement(null,name,index)

+ setElement(name: String, index : Integer, o : Object)

Sets named data element of indexed value => setElement(null,name,index,o)

+ getElement(name : String) : Object

Returns named data element of value0 => getElement(null,name,0)

+ setElement(name : String, o : Object)

Sets named data element of value0 => setElement(null,name,0,o)

+ getValue() : Datatype

Returns typed value, throws UnsupportedOperation exception if this is not a simple type

+ setValue(d: Datatype)

Returns typed value, throws UnsupportedOperation exception if this is not a simple type

+ getTitle() : String

Returns the short representation of value0

+ getTtile(index : Integer)

Returns the short representation of valueindex

CustomDataHandlerSupport Interface

+ getElement(me : Object,name: String, index : Integer) : Object

Get named, indexed data element, exposed/unexposed depending on me

+ setElement(me : Object, name: String, index : Integer, o : Object)

Set named, indexed data element, exposed/unexposed depending on me

+ size() : Integer

Returns number of values

CustomDataHandler Interface

* handlerConfig : String

Handler's copy of its config

+ render(style : String) : Object

Renders multiple objects in style

+ render(style : String, index : int) : Object

Renders single object in style

+ getScript(): String

Reference to an include file to go in header portion of page

+ getHTML(name : String)

HTML to insert in text for getting multiple objects, name property is prefixed with name

+ getProperties() : Properties

Returns a properties object containing properties to be returned on submit

+ setProperties(p : Properties)

Receives a properties object after submit to update value list

CustomDataRow Class

+ key : String

Optionally set by/returned to consumer

+ oldValue : String

Set by/returned to consumer

+ newValue : String

Returned to consumer

+ voidReason : String

Returned to consumer, null if not voided

- data : Object

For internal use by custom data service object

  • No labels

5 Comments

    • I'm starting to agree with Roger about the benefits of having a separate class for the datatype; however, I haven't reconciled how we'd avoid having to fully realize the values just to get their title (e.g., I want the title for a thousand radiology results handled as custom datatypes; however, I don't want to have to load all the images and I don't want a bunch of values that are only half-realized).
    • Why CustomDatatyped?  Why the "d" at the end?  I would expect a custom datatype to have the interface CustomDatatype.
    • How does this translate to complex obs?  Not a fan of the name "persistedString", but could live with it... but definitely wouldn't want it as a method on Obs.
  1. Unknown User (r.friedman)

    This comment moved from Improved Person Attribute Types.

    I'm afraid in our discussion Sep 21 we lost some of the points we made the previous week;  (1) We should have the ability to introspect complex objects/datatypes.  (2) All handlers of the same datatype should persist equal objects in the same way.  Instead, we reverted to the Burkean view that only the handler knows how to persist the object and that the datatype name is only a name, it need not represent an actual class so long as the handler can handle it.

    Let me introduce some terminology for this discussion using the notional Radiology Document in the text.  "Datatype" means a class name; this is stored as  string in an attribute, observation or global property type, e.g. Radiology Document.  "Value" means an instance of a datatype represented as a string in an attribute, observation or global property value.  "Complex object" means an object to be persisted, including at least a value, possibly some "intermediate objects" (Radiology Report and Radiology Film) and possibly some "payloads" (the externally stored images).

    One of Burke's main points was that handlers could not be constrained in the way they persisted their objects.  I don't see how this is consistent with the two points of the previous week.  It means that nobody could write a handler without knowing the internals of the initial handler.  For example, I could write an integer handler whose value was the text representation of the integer unless the integer was 7734, in which case the value would be "7734 (turn me upside down)".  Nobody wanting to write an integer handler could ever anticipate this.

    Another consequence of the only the handler knows how to persist philosophy is that when a complex object references an existing object (such as Concept in the diagram), that fact is lost.  There is no longer any referential integrity.  It makes complex objects unexchangeable via data/metadata exchange because of their hidden references.  I have called this the boxing problem in previous posts.

    I propose that datatypes be persisted in one of 3 ways: (1) java primitives can be stored in canonical form; (2) where there is a table whose data object class is the datatype, the value can be either the id or uuid; (3) the datatype can be "flattened".  This flattening should be by only one method, for which I see 3 possibilities: (a) the serialized version of the object, using some known serializer; (b) xml; (c) json.  I include the flattening option for simple complex objects such as blood pressure.

    Note how option (2) deals with the unloaded module issue.  While the datatype itself no longer exists, the referential integrity constraints it persisted in the database continue to be maintained.  Also it is possible to have a default display (RadiologyDocument-2345).  I wish I could think of a way to persist the hibernate.cfg.xml as a way of creating the datatype class that wouldn't interfere with routine updates of the definition by the module.

    Option (3) is not as automatic in handling referential integrity.  At least xml and json would allow references to be identified, although it would be quite expensive and maybe reserved for preventing purging of referenced objects. At least xml could have a default display attribute.

    Let's suppose that Burke's idea of a radiology datatype is more like the image per se. Then the datatype would be like RadiologyFilm with just the imageReference field and can be stored in option 3 form. But as RadiologyFilm shows, you would often want to have some data elements as well as the payload key and if these data elements were simple enough one might even use option 3. But hopefully the objects we are talking about are common enough and rich enough that it makes sense to put them in tables. The performance differences in filtering and sorting between SQL and code are substantial.

    On another point, I think we need to have HL7 as a standard view type.

  2. Unknown User (darius)

    Roger,

    Can you give a pseudocode snippet of how people would interact with your custom data service object? (Assume the datatypes are already defined.)

  3. Unknown User (darius)

    Roger,

    I've thought about this a lot, hopefully enough that I can describe those thoughts in relatively few words. (See Mark Twain + more time.)

    What I like about your proposal is introducing a strongly-typed Datatype interface (instead of a weak String), and being more clear about splitting the stuff that's currently in my CustomDatatypeHandler and in the to-be-renamed AttributeService.

    I don't like that it introduces a per-item-with-custom-values service. This has to be either stateful or slow. And it requires us to deal with custom values as groups in the service layer (e.g. all "Awards" for a provider, rather than treating those as multiple individual attributes). It doesn't really follow the pattern we use everywhere else in the application of dumb DTO + service, and I worry about its scalability.

    Given that, and given the code we already have written, I think this approach should be feasible to code quickly, and take the best elements of your proposal:

    1. Introduce a CustomDatatype interface
      1. Handles serialization to/from a varchar, and validation.
      2. Probably also has a render method which is only used when there's no Handler available.
      3. This guarantees that multiple handler implementations for the same datatype actually are compatible
      4. The actual java value would not (necessarily) be the same as the CustomDatatype, e.g. we'd have "class DateInPast implements CustomDatatype<java.util.Date>"
      5. I believe this is equivalent to an API-level handler from my previous proposal.
      6. If you unload the module that provides one of these datatypes you have to treat existing persisted values as unvalidated strings, but this is no different from what happens in the current code.
    2. Refactor CustomDatatypeHandler by removing some methods and having it refer to a CustomDatatype
      1. Try to do CustomDatatypeHandler<DT extends CustomDatatype<T>> (and get rid of "String getDatatypeHandled()")
      2. get rid of validate/serialize (since these are now in CustomDatatype).
      3. keep render(String raw, String view).
      4. We'd have sub-interfaces representing available UI technologies that let you define how the HTML widget looks and how its HTTP submission is processed.
      5. This would handle a single value. minOccurs/maxOccurs would be handled by the API (validation-wise) and the UI framework (view-wise). If you want to override this and have a single UI widget that lets you choose all awards the provider has won, create a ListOfAwards datatype with a web handler.
      6. The only case that didn't used to break before, but will break now, is if you unload a module that defines a datatype, but you keep a module that defines its handler. (I think this is a feature rather than a bug.)
    3. interfaces CustomValueDescriptor
      1. defines datatype, handler, config
      2. (this is what I called CustomDatatyped above)
      3. GlobalProperty implements this
    4. interface RepeatableCustomValueDescriptor extends CustomValueDescriptor
      1. adds minOccurs, maxOccurs
      2. VisitAttributeType, PersonAttributeType, etc, implement this
    5. interface SingleCustomValue
      1. GlobalProperty implements this.
      2. (This covers what you were getting at in your proposal, with different interfaces for things that have maxOccurs=1 vs maxOccurs > 1.)
    6. interface RepeatableCustomValue
      1. an abstract base class implementation of this interface does what your CustomDataService is doing minus the persistence. I.e. given a Person, this gives you convenience methods to work with all their "Email Address" person attributes as a group. But the persistence is still handled when you save the Person who owns the attributes.

    ...

    1. Unknown User (r.friedman)

      Darius –

      I want to spend more time with your reply in conjunction with the attribute design, I probably should have done so earlier. I want to make sure you know I understand the time pressure to get this out for the meeting and the investment in code that you and others have made. I have gone on and refined my design because I didn't want to leave it half-done, I'll clean up the page later.

      I have tried to address your concerns. I think the statefulness issue can be addressed easily except in the case where a change fails validation, in that case I can't think of much to do but revert to the last saved value (i.e. lose the changes that failed) or to put the sets of values in the context or serialize and persist them. I think the sets of values for each type is an inevitable consequence of adding multiplicity. I have tried to add some extra methods so that in the vast majority of cases where the multiplicity is 0:1 or the datatype is a simple type, simple calls very much like those used currently are available. I also tried to improve the way the handler relates to the form controller. I still haven't addressed the global property listener.

      I find myself surprised that the CDSO has become the eggplant that ate Chicago; it applies to every attribute, property, or complex concept; it applies to every OpenMRS object. It does seem to break down somewhat naturally into 5 areas: the management of multiplicity; the instantiation of the handler and marshalling data for its use; the mediator between the form controller and the handler; the service for the custom datatype; and the interface for external stores. It looks like you are trying to separate some of these and assign them to the attribute type and the service, which is probably for the good.

      One thing that we should make sure to agree on – for the sake of data exchange and REST, where the datatype is an OpenMRSObject, its name should be that of the domain object and the value should be the UUID of the object.