This post is to try to capture the issue we were discussing with Iniz and the openconceptlab module.
@suruchi has been working on creating an Iniz domain to load OCL-based zip files (eventually we’ll aim to support raw JSON files as well and loading the metadata for a subscription, but this is our initial target). As part of this work, we discovered that the openconcept lab module assumes that there’s an active subscription already setup when it tries to save concepts to the OpenMRS Concept dictionary. More exactly, the error we’re seeing is caused by this check for the type of validation that should be performed. By default, the global property that drives the validation type will only be set if someone has actually configured a subscription, which we don’t expect to be the case for a system that is being initially populated by Iniz (again, at a later state, we will add to Iniz the ability to automatically configure the OCL subscription, if any).
To unblock our work, we’re proposing to simply change the code in the openconcept lab module to default to assuming that we use ValidationType.FULL unless the global property has already been set.
@ibacher and @suruchi - this makes sense to me. I don’t see any reason why this would be a problem. If defaulting to full validation ends up not working for everyone (eg. it makes the import dramatically slower and this extra validation is not necessary in their setup), then finding a way to make this configurable may be something we want to look into.
There are a number of reasons, but chief among them is that the functionality to import the OCL JSON format already exists whereas exporting from OCL to Iniz formatted CSVs would be entirely new work (it’s also non-trivial work given the mismatch between OCL’s data model and the Iniz CSV format). Also, maintaining that converter would then become more work that needs to be done by the OpenMRS community.
Some other considerations are that the OCL format is more stable than the Iniz CSV format and that the OCL format allows us to maintain higher fidelity to the canonical representation of concepts in OCL. Among things that are hard to represent in Iniz’s format are: concept translations that are not fully specified or short (e.g., preferred names and alternative translations), mappings other than SAME AS, etc.
Thanks for explaining the situation, @ibacher. Given that the blocker really comes down to deciding how validation should be handled, I agree with the approach of defaulting to full validation when a subscription URL isn’t available.
@mksd was looking into expanding the CSV format in ways that was going to make it increasingly complex (evolving toward replicating the full data model in CSVs) and, at the same time, the OCL team was looking for a format in which to export concepts to OpenMRS. Since the subscription module already knows how to import OCL’s native JSON format (which can represent the full model with all features for concepts), the decision was to adapt iniz to support OCL’s JSON format by calling on the subscription module to handle the files.
So, it’s not really choosing JSON over CSV format; rather, it’s keeping CSV for the simple cases and also supporting OCL’s JSON format when you either need more features of concepts or your just pulling in concepts your managing in OCL (and don’t need to manage them in a spreadsheet).
