[SOLVED] Generation of Meta-data like Reference Application for Concept Installation

I am trying to leverage meta-data sharing of data like the reference application in the referencemetadata module to enable automatic configuration of encounters, concepts, locations. My question is how is this data generated

Concepts - https://github.com/openmrs/openmrs-module-referencemetadata/blob/master/api/src/main/resources/Reference_Application_Concepts-17.xml

Thanks in advance

I’m not sure, but I searched the repo and based on this it seems to be using dataexchange’s DataImporter.

Thanks @pascal this seems to work perfectly, as I have duplicated concepts so the metadata sharing module was failing to export them and I cannot change the concepts at this time.

@pascal Can I assume that the zip files are generated using metadata sharing module?

An interesting question to @raff What is the future of the data exchange module, is it going to be integrated into the metadata sharing? Will it be extended to include encounters, and other metadata?

I’m not sure.

Stephen, the Data Exchange module was just a hack to preserve concept IDs from CIEL when importing them into the Reference Application. The Metadata Sharing module does not let you do that. The Data Exchange module is going to be deprecated as soon as openconceptlab.org is ready for production. See https://wiki.openmrs.org/x/XYA5B

At this point I would recommend you to use https://wiki.openmrs.org/display/docs/Metadata+Deploy+Module

1 Like

@raff thanks for the info about deprecation. I am looking for a way to export my concepts into an XML file that can be version controlled, metadata sharing cannot do that because of duplicated concepts hence cannot validate

If you really want to have duplicate concepts in your dictionary then maybe, you could try backporting https://issues.openmrs.org/browse/TRUNK-4626 and disabling validation for the time of export/import. It’s just an idea, not a recommendation :wink:

@raff will look into that, but yes I am stuck with duplicate concepts (legacy issues) for now, not sure if metadata sharing allows for export without validation @mseaton @mogoodrich any ideas here?

This reminds me of the errors I had. After doing a successful upgrade, when I installed the ref app modules it will not stop talking about duplicate concepts. http://pastebin.com/VVjVT1de

@raff I am running into an error with the data importer

http://pastebin.com/bECkvTbj

I tried it on a blank OpenMRS database (new install with tables) and also from the database in which I ran the export and the error is the same. Does the importer update existing concepts it the UUIDs or ids match existing ones.

Generally, the recommended approach for forcing multiple servers to have the same configuration, and having this configuration on source control, is to use the Metadata Deploy module.

You can see an example of how PIH uses it here:

You’ll notice that by default the metadata is described in Java code, rather than a CSV, like you might prefer. In KenyaEMR they use CSV with metadatadeploy for their Kenya master facility list (which was long).

The underlying code for that is:

KenyaEMR’s code to use this underlying function is:

Part of the benefit of the metadatadeploy approach is that it forces someone to actually look at the metadata as they’re describing it, and this tends to lead to fixing mistakes, rather than blindly pushing them out to lots of servers. The downside is that it takes some time.

The dataexchange module allows you to blindly export from a database, and push into other databases. If you insist that it’s necessary to push out concepts that fail OpenMRS validation, this is probably your best choice.

That said, pushing out concepts that fail validation is a bad idea and I recommend strongly against it. It’s only going to cause you more pain in the long run.

@raff I have gone the route that you and @darius proposed using metadata sharing rather than a blind export of the concept dictionary.

I have hit a blocker details https://gist.github.com/ssmusoke/b2ff614981fb3d02fc43

  1. There is an error exporting the concept name tags - I thought it was a data issue, but fixed the concept_name_tag_map table by removing duplicates

  2. There seems to be a Concept 126, that is being searched for which does not exist in the database, but the export task does not state where this concept is being used for

Thanks in advance

  1. Seems like a bug in ConceptNameTagValidator in openmrs-core. What openmrs version are you using? Is it possible for you to upgrade to SNAPSHOT if we fix it in core?
  2. The bad row is hard to track down based on logs indeed. We need to add more logging in metadatasharing to better indicate what is being validated. Let me address that right away.

2nd is addressed in https://issues.openmrs.org/browse/META-352. Please test the omod attached to issue.

  1. @raff I am using 1.11.5 and yes it is possible for me to use a snapshot version

  2. The fix in the attached omod works very well, as I found the concept was in a concept set which is now fixed.

1 Like

@raff The attached omod does not have the labels resolved from the messages file, even when I build from source and install in the local maven repo

Thanks, I’ll have a look…