O3: Test Result Trees Driven by Concept Sets

Development
, ,
#5

Update: I’ve created two example Convenience Sets (stored in OCL under the OMRS org) for two example use cases:

3-level example

  • Bloodwork Set (see in OCL here)
    • Chemistry
      • Serum Electrolytes Panel
    • Hematology
      • CBC Panel

2-Level example that handles a site’s custom view request

  • HIV Adult Lab Set (see in OCL here)
    • Set Members are the labs that clinicians at Ampath specifically want to see together in one table, for their Adult HIV patients. image

(thought you’d find this interesting @akanter and @michaelbontyes and @jamlung)

Now updating that content in dev3.openmrs.org

#6

As it turns out, defining lists of tests for rendering flowsheets was one of the initial use cases for concept sets when Dr. Clem McDonald invented them for Regenstrief in the 1970s… from which our concept model was derived. :slightly_smiling_face: So, using concept sets to define what should be shown on a flowsheet is one of the most natural uses of concept sets.

We already do something very similar for complex observations. We already have methods for getting a list of obs in ObsService. What you’re describing would probably be a new method to get observations for all descendants of a given concept set that meet criteria (e.g., patient & time range) and return them in something like a TreeSet.

2 Likes
#7

Really nice concept - as @grace says, giving non-coders the ability to create these sets gives great value and flexibility.

Question 1: Do you envision that the tree structure is preserved? I.e., if my desired concept tree is “complete blood count” which has seven child tests, plus “basic metabolic panel” which has eight other child tests, it becomes automatic or straightforward to display the results with the tree headers shown and appropriate indentation. I imagine that this is in mind – it greatly helps end-users stay oriented as they are going through a long list of results.

Question 2:

I don’t see any problem with this at all – it’s not uncommon for the same test to be relevant in a couple of different panels and convenience subsets. The question in my mind is that, if there are several variations on the same theme (if several different clinics have their own idea of their local “basic metabolic panel”), we may want some way of keeping them apart in a list. In other words, either by naming convention or some other sort of filing system, there may be universal trees, and clinic-specific trees, and you could easily find just the universal ones or just your clinic’s own special ones.

1 Like
#8

Hi All,

Here’s a very naive approach to how the API and data could look like. Right now I’m imagining a GET request with two parameters, one for patientUuid and one for the name of convenience set, lab set, or test to search for. Data would preserve the tree structure, and tests array would preserve ordering.

GET /concept
Query parameters: 
- patientUuid - uuid of patient to search for
- name - name of convenience set, lab set, or test to search for

Response:
{
    "name": "Bloodwork",
    "class": "ConvSet", // convenience set
    "tests": [],
    "sub-sets": [
        {
            "name": "Hematology",
            "class": "ConvSet", 
            "tests": [],
            "sub-sets": [
                {
                    "name": "Complete Blood Count",
                    "class": "LabSet",
                    "tests": [
                        {
                            "name": "Hematocrit",
                            "class": "Test",
                            "values": {values obj},
                            "attributes": {attributes obj}
                        },
                        {
                            "name": "Platlets",
                            "class": "Test",
                            "values": {values obj},
                            "attributes": {attributes obj}
                        },
                        {
                            "name": "Mean corpuscular hemoglobin",
                            "class": "Test",
                            "values": {values obj},
                            "attributes": {attributes obj}
                        },
                        ...
                    ],
                    "sub-sets": []
                }
            ]
        },
        {
            "name": "Chemistry",
            "class": "ConvSet", 
            "tests": [],
            "sub-sets": [
                {
                    "name": "Serum chemistry panel",
                    "class": "LabSet",
                    "tests": [
                        {
                            "name": "Serum calcium",
                            "class": "Test",
                            "values": {values obj},
                            "attributes": {attributes obj}
                        },
                        {
                            "name": "Serum glutamic-pyruvic transaminase",
                            "class": "Test",
                            "values": {values obj},
                            "attributes": {attributes obj}
                        },
                        {
                            "name": "Serum albumin",
                            "class": "Test",
                            "values": {values obj},
                            "attributes": {attributes obj}
                        },
                        ...
                    ],
                    "sub-sets": []
                }
            ]
        }
    ]
}

Please let me know your thoughts, Thanks.

3 Likes
#9

Is this only to be used to show related concepts that would be viewed together, or is this intended to also link together panels with their results? (CBC linked to WBC, RBC, etc…)?

#10

Doing this seems like it would strain the concept dictionary in the same way that, e.g., reference ranges do. In other words the actual panel-as-performed should probably still be an obs group and this is just a way for showing related obs.

Thanks for the outline @zacbutko!. I have a few persnickety comments on tihs:

  1. The GET would likely be against some URL like /obs/tree/ or maybe just /obstree/ not /concept, because ultimately what you’re getting back is really a tree of obs (tests) categorised according to their defining concepts (also /concept is already an endpoint in the REST API).
  2. I would name the url parameters patient and conceptSet. It may be better to allow conceptSet to take either a name or UUID.
  3. For API consistency, it’s probably best that we just map properties as we already do, so that, e.g., each Set is returned as a concept using either the standard ref view or a predefined custom view. You can see most of the properties, e.g., here. At the very least, it’s best if the attribute names are in line with existing attribute names (“display” for “name”, “conceptClass” for “class” where “conceptClass” is resolvable to an object if needed).
  4. For similar reasons, “tests” → “obs”. While the initial target for this is lab tests, this is actually a fairly generic feature to construct a tree structure of Obs built from nested concept sets, so it’s much more general than just tests.
  5. The results returned in the “obs” array should match our standard obs properties, so, e.g., what’s returned from here.
  6. Really, really needless, but “sub-sets” should probably be “subSets”.
  7. I’m a little confused on the distinction you’re trying to make between “values” and “attributes”. On the naming convention side of things: attributes have a specific meaning in the backend and Obs already have a property called value, so we might want names that are less ambiguous for those.

I’d imagine (for ease of developing the backend) we’d end up with something like this structure:

{
  "display": "Bloodwork",
  "concept": {
     // Bloodwork concept details
  },
  "obs": [],
  "subSets": [
     {
       "display": "Hematology",
       "concept": {
            // Hematology concept details
        },
        "obs": [],
        "subSets": [
           {
              "display": "Complete Blood Count",
              "concept": {
                 // CBC concept details
               },
               "obs": [
                  // this is just an array of standard obs ordered according to concepts
               ],
               "subSets": []
           }
        ]
     }
  ]
}

Does that seem acceptable?

2 Likes
#11

Thanks for clarifying so much @ibacher ! Yes, all of your changes make sense. I didn’t know any of the endpoint formats or data structure so I guessed at all of them. Thanks for putting this into a more usable format!

By naming this parameter conceptSet are you disallowing searching for individual tests / obs? What are you thinking should be the answer if conceptSet is passed “Hematocrit”. Also I’m in agreement that it should support both name and UUID, or if it’s limited to just one I would favor UUID for specificity.

Totally agree.

Definitely not needless. We should pick variable naming conventions and stick to them. I actually just found this in my mock code and fixed it :slight_smile:

I just copied this from some object format I saw in the front end somewhere, but that was probably after a lot of manipulation. Thanks for clarifying!

Neither of those links you posted are working for me? I get a 404 for both.

Thanks again for reviewing this!

2 Likes
#12

I suppose I hadn’t considered that possibility, primarily because we already have an endpoint that can get all the obs for a given patient for a single concept, e.g.:

https://demo.openmrs.org/openmrs/ws/rest/v1/obs?patient=90f7f0b4-06a8-4a97-9678-e7a977f4b518&concept=162169AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA

Or, using FHIR:

https://demo.openmrs.org/openmrs/ws/fhir2/R4/Observation?patient=90f7f0b4-06a8-4a97-9678-e7a977f4b518&code=162169AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA

But, if it’s useful to do so, I don’t see any reason we can’t build a “tree” of just one concept and obs.

Yeah, they were both pointing at https://qa-refapp.openmrs.org which seems to be down at the moment. I just restarted it, so hopefully it comes up soon. If not, I think if you change the links from qa-refapp.openmrs.org → demo.openmrs.org they’ll work.

Thinking a bit more on this, it might be useful to have some time-based component to this too, e.g., so we don’t necessarily have to load all the obs, but all the obs since Oct 10, 2019 or something.

1 Like
#13

I only bring this up because the component is supposed to be completely abstract / agnostic as to the (tree) structure of data being retrieved. In principle implementers could configure it to show a single obs as peers alongside convenience sets.

That’s a great idea! startDate and endDate would be useful parameters to add, with reasonable behavior when omitting either or both

2 Likes
#14

Funny, Burke just said the same thing to me 30 mins ago as well :joy:

#15

Thanks for tagging me @grace , in OpenELIS we use our internal data structures, but don’t have anything quite as modeled out as this, and are thinking about the same issue of convenience sets. As we are moving toward implementing OCL which opens up OE to be able to share a concept set with OpenMRS, it would be interesting to handle these convenience sets the same way.

One of our main filters is to filter by lab section (EG: Microbiology, Biochemistry, etc) but the docs won’t be very interested that that specificity.

1 Like
#16

@zacbutko this being a potentially large tree, i guess you need the minimum number of fields returned to the client. Would something like this be enough?

{
  "{Test Result}": [
    {
      "{Vitals}": [
        {
          "Weight": "55.6kg"
        },
        {
          "Height": "4.5m"
        }
      ]
    }
  }
}
#17

Hi @dkayiwa , I’m not sure I follow your question? I think the best proposal for data structure so far looks like @ibacher 's answer here O3: Test Result Trees Driven by Concept Sets - #10 by ibacher. Does this fit with what you are thinking?

#18

@zacbutko do you need all those fields on the client?

#19

@dkayiwa , I’m still not sure of your question? What fields are you referring to when asking “do you need all those fields on the client?” The frontend will need all tree structure as proposed by @ibacher as well as all observation / test data contained within those convenience sets as they pertain to the patient. The final UI will look something similar to this.

DL6_Results-LeftFilterTree
DL6_Results-LeftFilterTree1440×1258 156 KB

#20

@dkayiwa, like @ibacher, I would expect the response to look just like our other obs responses, but organized into a hierarchy defined by the set(s) requested. What is the response if you request an obs group (a concept that is an obs group with some observations beneath it) from the obs service? Do you get the obs group and its children?

@mogoodrich, PIH uses obs groups for several constructs, don’t you? Do you have any examples of fetching these from via REST? Or do you request the individual obs and build the groupings client side? Or are they only used within HTML Form Entry (i.e., calling the API server-side)?

What we need for this use case is slightly different than requesting an obs that happens to be a concept set. While asking for an obs for a concept set would ideally return any obs within an obs group where the target concept is the grouping concept, for this use case we actually want to return the set members (and any descendants) whether or not there are corresponding obs.

At the Java API level, this would be something like

TreeSet<ObsRow> ObsService.getObsTable(Patient, Concept[, DateRange])

The REST endpoint would be something like

/obstable?patient=:uuid&concept=:uuid[&fromdate=:date&todate=:date]

with a response similar to what @ibacher suggested.

@dkayiwa, does this help clarify?

#21

Thanks @burke for the response. Am following this up with @zacbutko on the pull request he raised to consume this data: [O3-1060] Data Timeline FilterSet by ZacButko · Pull Request #557 · openmrs/openmrs-esm-patient-chart · GitHub

1 Like
#22

Via REST, Obs group members are returned as the groupMembers property of the grouping Obs, so, e.g., to pick on one structure CIEL:1421 (Immunization history and how we implement Immunizations in the FHIR2 module) would result in something like:

{
    concept: "1421AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA",
    // other attributes
    groupMembers: [
    {
         // CIEL:984 = Immunization given
         concept: "984AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA",
         // CIEL:5864 = Yellow fever
         valueConcept: "5864AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA",
           // other attributes
     },
     {
          // CIEL:1420 Immunization sequence number
          concept: 1420AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA,
          valueNumeric: 1,
          // other attributes
     },
     // other obs in this obs group
    ]
}

However, at least for the view discussed in this thread, we’d want to avoid pointing to obs groups at all if possible since we’re ultimately displaying the value obs and the handling of an obs group is likely to be more complex. E.g., the above example obs group really wants a custom view to translate the obs group into a single immunization record. This flattening of an obs group is obviously somewhat dependent on the structure of the obs group itself.

#23

It seems possible to keep a relationship between a test data-point and an encounter. If this is passed through with the data the table could act as a hyperlink to navigate to the original encounter.

Does anyone know if this is desired behavior? From a front end perspective this is very quick to implement, given that the endpoint ties encounter data to each data point.

#24

Regarding actual values of test ‘leaves’ I think this structure provides most of the functionality that the timeline needs. Here is an example for individual test type ‘Hematocrit’.

...
display: 'Complete Blood Count',
obs: [
  {
    display: 'Hematocrit',
    conceptUuid: '1015AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA',
    meta: {
      datatype: 'Numeric',
      hiAbsolute: 100,
      hiCritical: null,
      hiNormal: 51.9,
      lowAbsolute: 0,
      lowCritical: 21,
      lowNormal: 32.3,
      range: '32.3 – 51.9',
      units: '%',
    },
    values: [
      {
        effectiveDateTime: '2020-05-07T00:00:00+00:00',
        value: 42,
      },
      {
        effectiveDateTime: '2020-05-07T00:02:00+00:00',
        value: 43,
      },
      {
        effectiveDateTime: '2020-05-08T00:03:00+00:00',
        value: 45,
        notes: 'Maybe there is some special note written here?',
      },
    ],
  },
  {
    display: 'Platelets',
    ...
  },
  ...

@dkayiwa does this work for you?

@burke , @akanter , @ibacher are there some other data that needs to be displayed on Timeline table? Simplifying the data format to the above removes about 90% of information currently sent for the purpose of the timeline, but I believe these are actually the only information the timeline needs.