O3: Test Result Trees Driven by Concept Sets

jdick · February 8, 2022, 4:03pm

As we move forward with OpenMRS 3.0 features, the test results feature remains unfinished with regards to the designs.

The idea is to allow the user to have a filter tree to allow them to quickly move through categories of lab results. See https://app.zeplin.io/project/60d59321e8100b0324762e05/screen/60d5ea5bf120a6089c660a68

I would like to propose supporting this via concepts. A user would have the ability to create a concept set of concept sets based on their implementation needs. The leaf nodes of such. concept set would have to be of an allowable class (e.g. test result or test set).

To support, this I would like to propose the development of a new backend feature which takes a “root” concept, recursively unfolds the tree (could be done on initial load by OpenMRS if commonly used concepts are provided ahead of time) and on a rest request, if given a root concept, a patient id and a time window, will provide the “populated result” tree. The tree will be used to define the filters available in the UI such that only those Test with results are present.

Looking forward to hearing others’ thoughts.

JJ

@corneliouzbett @ibacher @Mekom @PIH @OHRI @burke @dkayiwa @aojwang (this is not meant to be exhaustive please make sure all are aware.

akanter · February 8, 2022, 11:11pm

JJ, can you provide more details and examples of the tree structure and the definitions of the branches? I don’t have a Zeplin account…

grace · February 9, 2022, 12:02am

I love the idea of implementing this via concepts, because:

It’s something a non-developer (like me!) can do/manage/maintain
Ways of grouping tests would be managed in one place along with concepts, instead of a secondary, basically duplicated place apart from concepts (which also have to be maintained anyway)

Convenience Sets too, right? Because there are some things you might want to group together for convenience (e.g. “Orthopedic Program View” or “Site A’s HIV Clinic View”), but they aren’t truly a “Lab Set” the way a CBC or Lytes panel is.

But, this introduces the likely scenario that there will be duplication of concepts across a Lab Set and a Convenience Set. For example: (and @akanter I think this will help answer your question above):

Site A’s HIV Clinic wants clinicians to be able to filter and just see Viral Load, CD4, and WBC count (would be stored in OCL as “Site A’s HIV Clinic, Convenience Set”).
Site A clinicians also sometimes want to see a whole CBC panel (stored in OCL as “CBC, Lab Set”). The CBC panel includes the concept for WBC count.

Does it matter to you @jdick that there could be multiple ways of seeing the same specific lab? (I think this is fine but seemed worth mentioning.)

grace · February 9, 2022, 12:03am

p.s. @caseynth2 are you guys at OpenELIS managing Lab Results filters with concept sets, or considering that? Maybe you already have gone through modelling something like this.

grace · February 9, 2022, 4:43pm

Update: I’ve created two example Convenience Sets (stored in OCL under the OMRS org) for two example use cases:

3-level example

Bloodwork Set (see in OCL here)
- Chemistry
  - Serum Electrolytes Panel
- Hematology
  - CBC Panel

2-Level example that handles a site’s custom view request

HIV Adult Lab Set (see in OCL here)
- Set Members are the labs that clinicians at Ampath specifically want to see together in one table, for their Adult HIV patients.

(thought you’d find this interesting @akanter and @michaelbontyes and @jamlung)

Now updating that content in dev3.openmrs.org

burke · February 10, 2022, 5:27am

As it turns out, defining lists of tests for rendering flowsheets was one of the initial use cases for concept sets when Dr. Clem McDonald invented them for Regenstrief in the 1970s… from which our concept model was derived. So, using concept sets to define what should be shown on a flowsheet is one of the most natural uses of concept sets.

We already do something very similar for complex observations. We already have methods for getting a list of obs in ObsService. What you’re describing would probably be a new method to get observations for all descendants of a given concept set that meet criteria (e.g., patient & time range) and return them in something like a TreeSet.

jteich · February 10, 2022, 4:00pm

Really nice concept - as @grace says, giving non-coders the ability to create these sets gives great value and flexibility.

Question 1: Do you envision that the tree structure is preserved? I.e., if my desired concept tree is “complete blood count” which has seven child tests, plus “basic metabolic panel” which has eight other child tests, it becomes automatic or straightforward to display the results with the tree headers shown and appropriate indentation. I imagine that this is in mind – it greatly helps end-users stay oriented as they are going through a long list of results.

Question 2:

I don’t see any problem with this at all – it’s not uncommon for the same test to be relevant in a couple of different panels and convenience subsets. The question in my mind is that, if there are several variations on the same theme (if several different clinics have their own idea of their local “basic metabolic panel”), we may want some way of keeping them apart in a list. In other words, either by naming convention or some other sort of filing system, there may be universal trees, and clinic-specific trees, and you could easily find just the universal ones or just your clinic’s own special ones.

zacbutko · February 10, 2022, 4:24pm

Hi All,

Here’s a very naive approach to how the API and data could look like. Right now I’m imagining a GET request with two parameters, one for patientUuid and one for the name of convenience set, lab set, or test to search for. Data would preserve the tree structure, and tests array would preserve ordering.

GET /concept
Query parameters: 
- patientUuid - uuid of patient to search for
- name - name of convenience set, lab set, or test to search for

Response:
{
    "name": "Bloodwork",
    "class": "ConvSet", // convenience set
    "tests": [],
    "sub-sets": [
        {
            "name": "Hematology",
            "class": "ConvSet", 
            "tests": [],
            "sub-sets": [
                {
                    "name": "Complete Blood Count",
                    "class": "LabSet",
                    "tests": [
                        {
                            "name": "Hematocrit",
                            "class": "Test",
                            "values": {values obj},
                            "attributes": {attributes obj}
                        },
                        {
                            "name": "Platlets",
                            "class": "Test",
                            "values": {values obj},
                            "attributes": {attributes obj}
                        },
                        {
                            "name": "Mean corpuscular hemoglobin",
                            "class": "Test",
                            "values": {values obj},
                            "attributes": {attributes obj}
                        },
                        ...
                    ],
                    "sub-sets": []
                }
            ]
        },
        {
            "name": "Chemistry",
            "class": "ConvSet", 
            "tests": [],
            "sub-sets": [
                {
                    "name": "Serum chemistry panel",
                    "class": "LabSet",
                    "tests": [
                        {
                            "name": "Serum calcium",
                            "class": "Test",
                            "values": {values obj},
                            "attributes": {attributes obj}
                        },
                        {
                            "name": "Serum glutamic-pyruvic transaminase",
                            "class": "Test",
                            "values": {values obj},
                            "attributes": {attributes obj}
                        },
                        {
                            "name": "Serum albumin",
                            "class": "Test",
                            "values": {values obj},
                            "attributes": {attributes obj}
                        },
                        ...
                    ],
                    "sub-sets": []
                }
            ]
        }
    ]
}

Please let me know your thoughts, Thanks.

akanter · February 11, 2022, 3:06am

Is this only to be used to show related concepts that would be viewed together, or is this intended to also link together panels with their results? (CBC linked to WBC, RBC, etc…)?

ibacher · February 11, 2022, 2:50pm

Doing this seems like it would strain the concept dictionary in the same way that, e.g., reference ranges do. In other words the actual panel-as-performed should probably still be an obs group and this is just a way for showing related obs.

Thanks for the outline @zacbutko!. I have a few persnickety comments on tihs:

The GET would likely be against some URL like /obs/tree/ or maybe just /obstree/ not /concept, because ultimately what you’re getting back is really a tree of obs (tests) categorised according to their defining concepts (also /concept is already an endpoint in the REST API).
I would name the url parameters patient and conceptSet. It may be better to allow conceptSet to take either a name or UUID.
For API consistency, it’s probably best that we just map properties as we already do, so that, e.g., each Set is returned as a concept using either the standard ref view or a predefined custom view. You can see most of the properties, e.g., here. At the very least, it’s best if the attribute names are in line with existing attribute names (“display” for “name”, “conceptClass” for “class” where “conceptClass” is resolvable to an object if needed).
For similar reasons, “tests” → “obs”. While the initial target for this is lab tests, this is actually a fairly generic feature to construct a tree structure of Obs built from nested concept sets, so it’s much more general than just tests.
The results returned in the “obs” array should match our standard obs properties, so, e.g., what’s returned from here.
Really, really needless, but “sub-sets” should probably be “subSets”.
I’m a little confused on the distinction you’re trying to make between “values” and “attributes”. On the naming convention side of things: attributes have a specific meaning in the backend and Obs already have a property called value, so we might want names that are less ambiguous for those.

I’d imagine (for ease of developing the backend) we’d end up with something like this structure:

{
  "display": "Bloodwork",
  "concept": {
     // Bloodwork concept details
  },
  "obs": [],
  "subSets": [
     {
       "display": "Hematology",
       "concept": {
            // Hematology concept details
        },
        "obs": [],
        "subSets": [
           {
              "display": "Complete Blood Count",
              "concept": {
                 // CBC concept details
               },
               "obs": [
                  // this is just an array of standard obs ordered according to concepts
               ],
               "subSets": []
           }
        ]
     }
  ]
}

Does that seem acceptable?

zacbutko · February 11, 2022, 4:59pm

Thanks for clarifying so much @ibacher ! Yes, all of your changes make sense. I didn’t know any of the endpoint formats or data structure so I guessed at all of them. Thanks for putting this into a more usable format!

By naming this parameter conceptSet are you disallowing searching for individual tests / obs? What are you thinking should be the answer if conceptSet is passed “Hematocrit”. Also I’m in agreement that it should support both name and UUID, or if it’s limited to just one I would favor UUID for specificity.

Totally agree.

Definitely not needless. We should pick variable naming conventions and stick to them. I actually just found this in my mock code and fixed it

I just copied this from some object format I saw in the front end somewhere, but that was probably after a lot of manipulation. Thanks for clarifying!

Neither of those links you posted are working for me? I get a 404 for both.

Thanks again for reviewing this!

ibacher · February 11, 2022, 5:11pm

I suppose I hadn’t considered that possibility, primarily because we already have an endpoint that can get all the obs for a given patient for a single concept, e.g.:

https://demo.openmrs.org/openmrs/ws/rest/v1/obs?patient=90f7f0b4-06a8-4a97-9678-e7a977f4b518&concept=162169AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA

Or, using FHIR:

https://demo.openmrs.org/openmrs/ws/fhir2/R4/Observation?patient=90f7f0b4-06a8-4a97-9678-e7a977f4b518&code=162169AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA

But, if it’s useful to do so, I don’t see any reason we can’t build a “tree” of just one concept and obs.

Yeah, they were both pointing at https://qa-refapp.openmrs.org which seems to be down at the moment. I just restarted it, so hopefully it comes up soon. If not, I think if you change the links from qa-refapp.openmrs.org → demo.openmrs.org they’ll work.

Thinking a bit more on this, it might be useful to have some time-based component to this too, e.g., so we don’t necessarily have to load all the obs, but all the obs since Oct 10, 2019 or something.

zacbutko · February 11, 2022, 7:17pm

I only bring this up because the component is supposed to be completely abstract / agnostic as to the (tree) structure of data being retrieved. In principle implementers could configure it to show a single obs as peers alongside convenience sets.

That’s a great idea! startDate and endDate would be useful parameters to add, with reasonable behavior when omitting either or both

grace · February 11, 2022, 7:43pm

Funny, Burke just said the same thing to me 30 mins ago as well

caseynth2 · February 14, 2022, 5:52pm

Thanks for tagging me @grace , in OpenELIS we use our internal data structures, but don’t have anything quite as modeled out as this, and are thinking about the same issue of convenience sets. As we are moving toward implementing OCL which opens up OE to be able to share a concept set with OpenMRS, it would be interesting to handle these convenience sets the same way.

One of our main filters is to filter by lab section (EG: Microbiology, Biochemistry, etc) but the docs won’t be very interested that that specificity.

dkayiwa · February 15, 2022, 3:37pm

@zacbutko this being a potentially large tree, i guess you need the minimum number of fields returned to the client. Would something like this be enough?

{
  "{Test Result}": [
    {
      "{Vitals}": [
        {
          "Weight": "55.6kg"
        },
        {
          "Height": "4.5m"
        }
      ]
    }
  }
}

zacbutko · February 15, 2022, 11:34pm

Hi @dkayiwa , I’m not sure I follow your question? I think the best proposal for data structure so far looks like @ibacher 's answer here O3: Test Result Trees Driven by Concept Sets - #10 by ibacher. Does this fit with what you are thinking?

dkayiwa · February 16, 2022, 7:49am

@zacbutko do you need all those fields on the client?

zacbutko · February 16, 2022, 9:04am

@dkayiwa , I’m still not sure of your question? What fields are you referring to when asking “do you need all those fields on the client?” The frontend will need all tree structure as proposed by @ibacher as well as all observation / test data contained within those convenience sets as they pertain to the patient. The final UI will look something similar to this.

burke · February 18, 2022, 4:42am

@dkayiwa, like @ibacher, I would expect the response to look just like our other obs responses, but organized into a hierarchy defined by the set(s) requested. What is the response if you request an obs group (a concept that is an obs group with some observations beneath it) from the obs service? Do you get the obs group and its children?

@mogoodrich, PIH uses obs groups for several constructs, don’t you? Do you have any examples of fetching these from via REST? Or do you request the individual obs and build the groupings client side? Or are they only used within HTML Form Entry (i.e., calling the API server-side)?

What we need for this use case is slightly different than requesting an obs that happens to be a concept set. While asking for an obs for a concept set would ideally return any obs within an obs group where the target concept is the grouping concept, for this use case we actually want to return the set members (and any descendants) whether or not there are corresponding obs.

At the Java API level, this would be something like

TreeSet<ObsRow> ObsService.getObsTable(Patient, Concept[, DateRange])

The REST endpoint would be something like

/obstable?patient=:uuid&concept=:uuid[&fromdate=:date&todate=:date]

with a response similar to what @ibacher suggested.

@dkayiwa, does this help clarify?