Export concepts using data exchange to preserve IDs for later updates of CIEL using mdsbuilder

@raff when i look at the commit history https://github.com/openmrs/openmrs-module-dataexchange/commits/master this feature was added and released in version 1.3 but with a MetadataSharingParser class whose parseConceptIds method body just throws a not supported exception, from the first time of its release up to the latest release.

Do you happen to know why in all subsequent releases the MetadataSharingParser class just throws that exception?

It was a placeholder before implementing it, which I did later and also deployed to mdsbuilder an omod, but apparently I failed to push changes to upstream and do a proper release to maven. Now after restoring mdsbuilder we no longer have even the binary.

@raff in your estimation, how long do you think it would take some one to reimplement this feature?

I’d say < 2h. It’s a matter of parsing header.xml from a zipped metadata package to extract concepts ids. The rest is done in https://github.com/openmrs/openmrs-module-dataexchange/blob/master/omod/src/main/java/org/openmrs/module/dataexchange/web/controller/DataExchangeController.java#L69

Hi @raff @dkayiwa ,

Any updates on the feature for the data-exchange module?

I may have time to implement that by the end of this week.

So, my understanding is RefApp 2.9 release is blocked by the need to implement parseConceptIds(InputStream) method to

  • treat input stream as zip input stream
    ZipInputStream zis = new ZipInputStream(fin);
    
  • find and read header.xml contents into memory
  • parse as XML document
  • return values from the following xpath(s) into a Set:
    • /package/items/org.openmrs.module.metadatasharing.Item[classname='org.openmrs.Concept']/id
    • /package/relatedItems/org.openmrs.module.metadatasharing.Item[classname='org.openmrs.Concept']/id
  • close the input stream before returning (to clean up)

Am I close? Is it safe to assume the header.xml can be loaded into memory?

Burke, yes, that’s it and it’s safe to load header.xml into memory.

Committed in https://github.com/openmrs/openmrs-module-dataexchange/commit/39f6ec49bbfd930a2aa8cb787356954c2b224aa3

Now it’s a matter of releasing the module, testing and deploying to mdsbuilder. I’m not setup to do a release on my machine so please do that and I can update mdsbuilder to include the fixed version.

Awesome @raff! I’ll try releasing the module now(if I have the necessary permissions) :smile:

@dkayiwa , do we have a CI plan for this module? I am not really sure about the procedure for the releasing this particular one.

I have created the CI plan and released version 1.3.3 of the module: https://ci.openmrs.org/browse/DEX-DEX-3

@dkayiwa , you’re awesome! :smiley: , @raff can you update mdsbuilder please?( I have a feeling that the updation part is something that I can also do but I don’t want to take chances here ). Also , should we be updating the database on mds with the latest CIEL dump file which was released just 2-3 days back?

@reubenv, updated. I’d just use the CIEL that’s been loaded already. Updated deploy instructions at https://github.com/openmrs/openmrs-contrib-ansible-docker-compose/blob/master/files/mdsbuilder/README.md

1 Like

@raff @dkayiwa , I was just trying to continue with the release and I still get an error. Here’s a link to the same,

Thanks

Are you able to reproduce this on your local openmrs instance?

@raff i have just tried exporting a header.xml package on qa-refapp. On clicking Submit, is it supposed to end up with the xml displayed here? https://qa-refapp.openmrs.org/openmrs/module/dataexchange/exportPackageContent.form

Am using Chrome.

Do you think I should set a build for that?

Corrected the release instructions. You can now pass a .zip file and not a header.xml.

1 Like

@reubenv any update on this?