OPTIONAL: 2021-07-21: Dictionary Manager Squad call (focus on peer support)

Dear community,

This week’s OCL squad call is optional for folks to attend. We want to make sure that people have the time and space they need to properly mourn our dear friend Kaweesi’s passing.

We will not have a usual agenda. Instead, people are welcome to drop in for peer support in grieving.

If people would find it a helpful outlet to discuss some technical issues, we can also unblock each other’s technical questions as-needed. But first we will focus on making sure that anyone who would like to talk or support one another has this space to do so.

When: Wednesday 6:30pm IST | 4 pm EAT | 2pm CET | 1pm UTC | 9am EST | 6am PST

Where: Join from computer or mobile: om.rs/zoomocl

@grace @paynejd @dkayiwa @ibacher @burke @ball @michaelbontyes @muhima08 @mogoodrich @akanter @tendomart @jwnasambu @tendomart @nkumar @irenyak1 @hadijah315 @patrickcmd

Following today’s call, I was working on a simple use case for customizing mappings (i.e., adding another CIEL concept as an answer to a simple CIEL question). While testing it out, I discovered two issues:

  1. It looks like CIEL’s Yes, No, and Unknown concepts have mappings to PIH 1065, 1066, and 1067, which are no longer correct. These should be PIH codes 1, 2, and 5, respectively. (@akanter)
     
  2. When adding Received HIV test result from CIEL to my test dictionary, I get the question, its 3 answers (Yes/No/Unknown), and I get PIH’s 1065, 1066, and 1067 concepts (which are Hep B, syphilis, and C-reactive protein because of #1). Why would the Dictionary Manager load these additional concepts? In short, I understand adding question & answers, but I don’t understand why we would cascade into loading concepts from mappings in the answers.

/cc @grace @suruchi @ibacher @akanter

This seems very strange behavior. For #1, I had a discussion about this with Ellen some time ago. If PIH has retired a mapping and created a new mapping that having >1 PIH mapping on the same concept would not be a problem (if only 1 of the mappings would be active). This would allow for retrospective data to be mapped to the current CIEL concept. However, in this case, it seems like an error if the PIH mappings are currently active and have completely different meanings. Could this have occurred because PIH concepts were renumbered when imported and this is causing problems with both #1 and #2?

The #2 issue would not normally happen. Dependencies are based only on internal mappings. If PIH is now considered internal and the concept_ids were changed, then this might happen.

So 2 is caused by:

  • PIH no longer being an external source in CIEL
  • The recursivelyFetchToConcepts() function which just loads the mappings (up to levelsToCheck deep) from concepts.

The recursivelyFetchToConcepts() is probably setup to do this with the intention of finding, e.g., concept set members and answers, both of which would need to be referenced for the concept to be useable (e.g., if the answers to a question concept are given by a convenience set, we also need those sets members).

We could probably make things work more as the are expected to just by limiting the version of fetchMappings() here.

Are there other mapping types we need to make sure are copied? I think CONCEPT-SET and Q-AND-A are the only mapping types that have “special” treatment.

Looking at the SQL dumps from @ball, it looks like PIH concept IDs for Yes and No are 1 and 2. Unknown has concept ID 5 in the latest SQL, but appears to have concept ID 173 in one of the earlier SQL dumps, so perhaps Ellen could clarify.

@ball can you confirm PIH’s concept IDs for Yes, No, and Unknown? Are these 1, 2, and 5, respectively; or, do the concept IDs vary by implementation (e.g., Unknown is 5 in some sites and 173 in others)?

I would be happy if we abolished the idea of “internal” vs “external” mappings. Mappings are just a link to something. Whether or not the entity being mapped is available or not (e.g., within OCL) is something that can (and will) change over time.

Another reason to abolish the notion of “internal” vs “external” mappings. This makes the behavior of Dictionary Manage unpredictable.

Reviewing list of map types in OpenMRS and OCL…

Click to see all map types in OpenMRS
$ http -b -a admin:Admin123 \
  'https://demo.openmrs.org/openmrs/ws/rest/v1/conceptmaptype' | \
  jq -r .results[].display | sort
Associated finding
Associated morphology
Associated procedure
Associated with
BROADER-THAN
Causative agent
Finding site
Has specimen
Laterality
NARROWER-THAN
SAME-AS
Severity
Click to see all map types on OCL
$ http -b \
  'https://api.openconceptlab.org/orgs/OCL/sources/MapTypes/concepts/?limit=1000' | \
  jq -r .[].id | sort
Access
After
Associated-finding
Associated-morphology
Associated-procedure
Associated-with
BROADER-THAN
CONCEPT-SET
Causative-agent
Causative-agent
Clinical-course
Component
Direct-device
Direct-morphology
Direct-substance
Due-to
Episodicity
Finding-context
Finding-informer
Finding-method
Finding-method
Finding-site
Has-active-ingredient
Has-definitional-manifestation
Has-dose-form
Has-focus
Has-intent
Has-interpretation
Has-specimen
IS-A
Indirect-device
Indirect-morphology
Interprets
Interprets
Laterality
MAY-BE-A
MOVED-FROM
MOVED-TO
Measurement-method
Method
NARROWER-THAN
Occurrence
Part-of
Part-of
Pathological-process
Priority
Procedure-context
Procedure-device
Procedure-morphology
Procedure-site
Procedure-site
Procedure-site-Direct
Procedure-site-Indirect
Property
Q-AND-A
Q-AND-A
REPLACED-BY
Recipient-category
Revision-status
Route-of-administration
SAME-AS
Scale-type
Severity
Specimen-procedure
Specimen-source-identity
Specimen-source-morphology
Specimen-source-topography
Specimen-substance
Subject-of-information
Subject-relationship-context
Surgical-approach
Temporal-context
Time-aspect
Using-access-device
Using-device
Using-energy
Using-substance
WAS-A

I believe CONCEPT-SET (for set members) and Q-AND-A (for answers to questions) are the two concept relationships in OpenMRS for which OCL uses mappings. These are the only mappings types I would expect the Dictionary Manager to follow when cascading.

The mappings on the master PIH concept server for Yes and No are indeed PIH:1065 and PIH:1066, which may be wrong, but is what is in place. It may not be possible to just create new external ids based on PIH concept primary keys, as this may not be backwards compatible with what our server actually has in place and what may be used, but I defer to @ball ( FYI @mogoodrich ).

I’ve create OCLOMRS-1015 to look into this. One caveat might be that we probably still want the SAME-AS and other mappings to show up in the new source (so that we have an accurate copy of the concept as it exist in CIEL), but not the actual concept itself.

Yes. To be clear, I would expect to get all mappings for any concept imported, but I would only expect an additional concept to be imported if it’s a set member or answer to a concept being imported. In short, answers & set member concepts get automatically included in the import but any concepts linked through SAME-AS, NARROWER-THAN, etc. relationships are not automatically included. For each concept imported, I would assume all of its “mappings” (using the OpenMRS meaning of the word mappings… meaning concept_reference_term mappings) would be included as part of the concept’s definition.

@burke The concept_id from the PIH mysqldump is from a local OpenMRS SDK instance. Those concept_ids are random and not associated with the PIH mappings.

Our “golden” PIH concept server is used to generate the mds packages and later the PIH EMR concept mysql import file to OCL. The concept_id on the “golden” server matches the PIH mapping.

“Golden” server:

SDK Instance used for OCL import: image

Our “golden” server has many old and unused concepts. I’d prefer not to use that as the source for OCL. It would require lots of cleanup.

Thanks @ball

To just second what she said @burke we should not be using the concept_id from the dump that @ball provided. We don’t maintain consistent concept ids across all our servers. The mapping code (and generally/always? the uuid) are what is consistent (as the screenshot Ellen posted shows).