How to add a custom concept dictionary (csv file) to a module

Application Name: OpenMRS Version Number: 2.4 / 2.5

Question: I want to bind a custom concept dictionary (csv file) to my module during build in eclipse, and have all the concepts automatically added to the openmrs concepts dictionary on any instance where I drop the module. Could anyone help with this? Or could anyone show me a sample code/module I could replicate?

I would also like to know if there is a concept dictionary upload module?

In general, I would recommend that any concepts required for modules which would be distributed to the community… be added to the CIEL dictionary and therefore be part of a global load. However, new modules which require concepts which were not previously loaded into the local instance would require an MDS (meta data sharing module) or OCL package to add the missing concepts. @darius, do you agree?

@cpnwaugha, there is no way to do this using a CSV file out of the box. Concepts have a complex data model that can’t easily be captured in a spreadsheet.

I know of a few approaches to what you’re trying to do:

  1. Create a metadata sharing package containing the concepts you want (using the Metadata Sharing module) and distribute this as a ZIP file to be manually installed by the implementers.

  2. create a MDS package like in (1) and have this automatically installed by a module. You can see an example of this in older versions of the referencemetadata module. This approach allows these concepts to be added on top of an existing implementation’s concepts. https://github.com/openmrs/openmrs-module-referencemetadata/blob/referencemetadata-2.2/api/src/main/java/org/openmrs/module/referencemetadata/ReferenceMetadataActivator.java#L99

  3. Use the dataexchange module to import concepts defined as XML, as is done in current versions of the referencemetadata module. This will install the concepts with fixed conceptIds, so if you try to install on top of an existing implementation with its own concepts, they will be overwritten and the data will be corrupted. https://github.com/openmrs/openmrs-module-referencemetadata/blob/referencemetadata-2.5.0/api/src/main/java/org/openmrs/module/referencemetadata/ReferenceMetadataActivator.java#L68

  4. You can define concepts in code, and have your module install this at startup as PIH started to do here: https://github.com/PIH/openmrs-module-pihcore/tree/master/api/src/main/java/org/openmrs/module/pihcore/deploy/bundle/core/concept

All of these are hacky and/or awkward. In the future our hope is that (1) you define the concepts in Open Concept Lab, and then (2) you ask implementations to subscribe to them. But this isn’t ready yet.

1 Like

Thanks a lot @akanter and @darius for the timely responses. @darius I think we will try to implement (2) because we want to automatically achieve this since we’d be deploying to a good number of facilities. I will let you know how it goes.

@darius and @dkayiwa is https://github.com/openmrs/openmrs-module-referencemetadata/blob/referencemetadata-2.2/api/src/main/java/org/openmrs/module/referencemetadata/ReferenceMetadataActivator.java#L99 enough to pull the packages.xml file, and thus the zip files into the module? Because am not getting any positive result.

Is your module publicly available on a place like github such that we see the changes you have made?

Yes @dkayiwa. See it here: https://github.com/nwaughachukwuma/samplerepo.

I edited the packages.xml file to include UTH_Concepts, see it here: https://github.com/nwaughachukwuma/samplerepo/blob/master/smartacarecustomapp4/api/src/main/resources/packages.xml

I also added UTH_Concepts.zip file to the resources directory. I am just wondering how https://github.com/nwaughachukwuma/samplerepo/blob/master/smartacarecustomapp4/api/src/main/java/org/openmrs/module/smartacarecustomapp4/SmartaCareCustomAppActivator.java#L151 can pull in the UTH_Concepts.zip file or even the packages.xml file

I took a peek at some of the concepts in UTH concepts. There is overlap with CIEL in some cases and there are potentially problematic concepts like (Hypertension/Shock). Since CIEL is distributed as one of the sources of concepts for the OpenMRS community, I recommend that you choose from there when you can, and when you need to request a new concept (such as Doppler of Renal Vessel) you let me know. Terminology can be hard, and I just wanted to give you some feedback if people are going to want to integrate your concepts with a curated concept dictionary like CIEL.

Yes @akanter I am aware of the overlap, and it is because of the way we used them in our custom html forms. We didn’t want to alter the CIEL concepts markedly, because we feared they might be used by other forms. For example, the CIEL version of hypertension has class: Diagnosis and data type: N/A, while ours has class: Misc and data type: Boolean . I will review our custom concepts again in the light of what you have said i.e. use CIEL concepts where necessary (and possible), and also indicate new concepts like you new pointed.

At the moment we have to meet deadline, and we need a quick fix to the problem. I will very much need as much help as possible to solve this problem, and in the nearest future begin to implement the things you have pointed. Thanks again

@cpnwaugha can you send me a compiled omod of your module?

@dkayiwa see it here: https://www.dropbox.com/s/1cuv0zjrkqcd6lf/smartacarecustomapp4-1.0-SNAPSHOT.omod?dl=0

You need to be clear with your report, in order for us to be helpful. For instance, when you say no positive results, do you mean getting compiler errors? Module loading well but concepts do not get added? Module failing to install? Or something else?

Thanks for the correction @dkayiwa. Point noted. The module is loading well but concepts do not get added.

Do you get any errors in the tomcat log files?

Yes I do @dkayiwa. This is the pastebin link to the tomcat log file: http://pastebin.com/akUw7pCx

Look at line 676.

Your log shows the cause of the failure with this error message: ERROR - MetadataUtil.installMetadataPackageIfNecessary(157) |2017-01-03 11:43:07,044| Failed to install metadata package UTH_Concepts-1.zip java.util.zip.ZipException: invalid stored block lengths

The solution is what @mseaton gave in this thread: Why am I Unable to install bundle PackagesMetadata? You need to have it replace the resource filtering that you have in the pom.xml file of your api sub module.