While this might sound a bit like this thread, this is actually intended as a discussion of a different point about concept identity in the OCL module, specifically, how we relate mappings to concepts.
There are two different things that the OCL module uses to determine concept identity:
- The
external_id(in OCL) refers to the UUID of the concept. - The “URL” of the concept from OCL prepended with the name of the OCL server the OCL module is synchronised with.
The first of these is more important for determining how the module interacts with existing entries in the concept dictionary: essentially, if a concept exists with the same UUID as the OCL concept, the OCL concept will overwrite it.
The second of these, however, is important for determining how the module deals with OCL’s internal references.
Each object in OCL has a few system identifiers:
- a UUID field (which is so named for legacy purposes… it actually stores an auto-incrementing database identifier)
- an ID field (which stores the “mnemonic” for the concept, e.g. “1065” for CIEL:1065 or " 95941-1" for LOINC: 95941-1)
- a URL (which identifies the object in the REST API)
- A version URL (which identifies the specific version of the object in the REST API).
For the current version of CIEL: 1065 the relevant fields look like this:
{
"external_id": "1065AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA",
"id": "1065",
"name": "1065",
"owner": "CIEL",
"owner_type": "Organization",
"owner_url": "/orgs/CIEL/",
"source": "CIEL",
"type": "Concept",
"url": "/orgs/CIEL/sources/CIEL/concepts/1065/",
"uuid": "3678",
"version": "3678",
"version_url": "/orgs/CIEL/sources/CIEL/concepts/1065/470101/",
"versions_url": "/orgs/CIEL/sources/CIEL/concepts/1065/versions/"
}
We use the URL and version URL fields in the following ways:
- When importing a concept or mapping, we check to see if it was already imported and if so if this version was already imported. If it’s already imported, we simply skip the concept.
- When importing a mapping, the concept URL is used to determine which two concepts the mapping pertains to.
2 above comes about because the mappings we receive from the API look like this:
{
"external_id": "7180CCCCCCCCCCCCCCCCCCCCCCCCCCCCCCCC",
"from_concept_code": "155569",
"from_concept_url": "/orgs/CIEL/sources/CIEL/concepts/155569/",
"from_source_name": "CIEL",
"from_source_owner": "CIEL",
"from_source_owner_type": "Organization",
"from_source_url": "/orgs/CIEL/sources/CIEL/",
"from_source_version": null,
"id": "303992",
"map_type": "Q-AND-A",
"owner": "CIEL",
"owner_type": "Organization",
"source": "CIEL",
"to_concept_code": "1065",
"to_concept_url": "/orgs/CIEL/sources/CIEL/concepts/1065/",
"to_source_name": "CIEL",
"to_source_owner": "CIEL",
"to_source_owner_type": "Organization",
"to_source_url": "/orgs/CIEL/sources/CIEL/",
"to_source_version": null,
"type": "Mapping",
"url": "/orgs/CIEL/sources/CIEL/mappings/303992/",
"uuid": "303992",
"version": "303992",
"version_url": "/orgs/CIEL/sources/CIEL/mappings/303992/1226376/",
"versioned_object_id": 303992,
"versioned_object_url": "/orgs/CIEL/sources/CIEL/mappings/303992/"
}
Basically, we just leverage the from_concept_url and to_concept_url properties to work out the concepts being referred to.
So why does this matter?
Currently in the OCL module we alway pre-pend a server URL to the URLs so that we can distinguish between a concept from api.openconceptlab.org (the OCL server), api.staging.openconceptlab.org (the OCL staging server), or something like api.ocl.brown.edu (e.g. if, for whatever reason I were to stand up my own OCL instance). The way we determine bit to pre-pend to the URL is derived from the global property that the subscription URL (if any) is saved in.
This works fine in the case where the subscription module is used to subscribe to a live OCL instance and would work for the offline case if someone entered a subscription URL and then uploaded a zip file using the UI (the “offline mode” feature). Where this begins to fall down is in a few of the cases that we’re currently contemplating. For instance, if we import the zip file on start-up without setting the subscription URL (as can happen today or when we get things working with Initializer), then it’s possible to end up with concepts with inconsistent URLs. Likewise, @mseaton relatively recently opened a PR to add the ability to upload a single concept from OCL as a file downloaded from the Term Browser, which seems like a useful feature.
So, the question is, how should we support these use-cases where an actual subscription may not be present while still maintaining the ability to import mappings correctly?
Option 1: We stop prepending anything to the URLs and just use the URLs as they are in the files we get from OCL. This is by far the simplest and least disruptive option, but it would mean that we can’t distinguish between concepts loaded from two different instances of OCL, which may be a problem down the road if countries and implementers opt to host their own OCL instances rather than using the central server.
Option 2: We continue with things as they are but where the subscription URL isn’t filled in, we assume that the content came from https://api.openconceptlab.org, i.e., the OCL server that the OCL team at Regenstrieff supports.
Option 3: We continue things as they are today and let implementers deal with potential inconsistencies where data was both loaded from a zip file and a live subscription.