Integration with OpenEMPI

@craigappl, we haven’t lost interest in the great work your team has been doing with MPI integration. :slight_smile:

I see the iSantéPlus openmrs-registration-core is 100 commits ahead of the community repo. How can we help get this work folded back into the community repo?

1 Like

@nathaelf, do you have any thoughts on this? Thanks! -Burke :burke:

1 Like

@nathaelf or @craigappl I don’t want to lose this thread. Any thoughts on @burke post about folding iSantéPlus back into the community repo?

Hi,

I’m no longer on the iSantePlus team and don’t have the technical skills to perform the merge. The challenge with the merge is that all of the commits have been made to master along with a few other enhancements that have to do with fingerprints. The person would have to either go through and cherry-pick all of the commits related to the MPI or go through and manually rework the files that have been touched. I’m happy to talk about the work if anyone is able to pick it up.

Craig

1 Like

Hi @craigappl and @pgesek,

I watched the video training session you guys recorded and am trying to set up a the docker container for openEMPI (https://github.com/SEDISH/ohie-cr)

I cloned the repo, went inside and ran the command:

$ docker build -t uwitech/ohie-cr .

and received the following message:

Step 1/28 : FROM uwitech/ohie-base Get https://registry-1.docker.io/v2/uwitech/ohie-base/manifests/latest: unauthorized: incorrect username or password

would greatly appreciate if someone could give me some advice on how to proceed? Thank you so much :slight_smile:

Hi @craigappl @pgesek @mogoodrich @mseaton @ball,

Just to highlight some sticky points I’m facing now and the things I’ve tried so far.

Regarding the above message about the unauthorised error, it was that I had not logged in to docker via the cli, and that was easily fixed. (thanks to guidance from @mseaton) (https://docs.docker.com/engine/reference/commandline/login/)

Goals


My overarching goal is to merge the MPI functionality into the main branch of the reference app and associated modules, e.g. registration core, registration app.

My intermediate goal was to set up all the necessary components, I.e. iSantePlus client, OpenEMPI Docker, etc (I know there are some other necessary components e.g. XDS, PIX, PDQ endpoints? but I haven’t figured out what all those things are yet) but basically I wanted to set up all these components on my local machine and be able to use all the MPI functionality and have a “ground truth” where everything works and start refactoring from there.

Questions/ Requests


  1. Is there a way to build the iSantePlus m2Sys Biometrics Module?

    • Demo/Sample environments/instances of BioPlugin server 8 and Cloudscanner? And is there a way to use this module without the hardware of the M2Sys fingerprint reader? e.g. just a way to feed the system IDs/Keys?
    • Looking at the code the MPI functionality is very intertwined with the fingerprint functionality and the iSantePlus registration core and app modules would not work without it.
  2. Are there other training videos you guys made on these modules/this functionality that could help me to understand the concepts and implementation? I watched the video linked above twice but I’m very new to OpenMRS and all of this so it would be really helpful to have more guidance on how all of this works and all the parts fit together :slight_smile: If there are sequence diagrams to show how all the classes interact and in what flow that would be very very awesome, or if we could set up a time for a call @pgesek

What I’ve Tried So Far


1: Set up Open EMPI Docker

(Done)

2: Set up iSantePlus instance locally

(Roughly following the README on https://github.com/IsantePlus/openmrs-module-isanteplus) (Thanks to guidance from @mogoodrich)

  • Used SDK to set up an OpenMRS platform (V 2.0.5)
  • Copied modules from Reference Application (V 2.5) and replaced those that were created by the SDK for the platform
  • Updated the openmrs-distro.properties with that list of modules
  • Used mvn openmrs-sdk:deploy to install the htmlformentry 3.3.2 module
  • Git cloned the isanteplus module
  • Tried to use the sdk to install the isanteplus module, but got an error stating that it required the xds sender module.
  • Git cloned the xds sender module
  • mvn clean installed the xds sender module (successful)
  • Used the sdk to install the isanteplus module (successful)
  • Ran the server, it seemed to work, had new UI but it did not look like the UI of the demo server that @ball shared with me (OpenMRS logo not iSantePlus logo, no outgoing exceptions icon on home page etc)
  • Tried to add a patient through my local instance and then see if it was added to Open EMPI but it didn’t work.

3: Set up iSantePlus instance locally (Beyond the instructions)

My hypothesis was that this was not the complete set up of the iSantePlus server even though I followed the instructions in that module because I had not installed the other modules listed in Notes on iSantéPlus Activity and Push/Pull Architecture with SEDISH by @craigappl, namely: isanteplus/registrationcore, isanteplus/registrationapp, isanteplus/xds-sender and isanteplus/outgoingmessageexceptions modules. So I thought I should install them on the server

  • Used sdk to install the xds sender module (successful)
  • Used sdk to install the outgoing exceptions module (successful)
  • Used sdk to install the registration core module (successful)
  • Tried to use the sdk to install the registration app module (Unsuccessful)
    • It produced an error saying that it required the m2sys biometric module
    • Because I dont have a BioPlugin server (as explained above) this is where I got stuck…
  • I thought I might continue to try to get the server to work even without the isanteplus versions of the registration core and registration app and went down several error fixing rabbit holes but ultimately realised that I was getting too many errors that I didn’t know how to fix and that it probably wouldn’t work anyways…

I’m thinking now that if I am not able to have a working local instance of iSantePlus running and then work backwards from there, I might have better luck starting from the latest core OpenMRS modules, understand the mechanics of the iSantePlus code super thoroughly and then work forwards from there…? Thoughts anyone?

Anyways, thanks for your time and effort in reading my clumsy attempts at this, hope you will be able to share some advice :slight_smile:

-Shao Yuan

1 Like

Hi Shao,

I’m sorry I haven’t been around to support this. I am attending a conference and will be back online tomorrow. I will help get you unblocked. I’m located in Seattle and can also schedule a call for Monday. Please send me a PM.

Craig

1 Like

Hi All,

Just a quick update before I move on to the next step. Following @craigappl’s advice I continued to try to set up the local iSantePlus instance and set the global properties to match that of the iSantePlus hosted demo to hit the same openHIM/ Interoperability endpoints. I’m happy to say that I’ve got it working. I’m just going to detail the versions of modules I have installed as well as the global properties I changed for future reference.

Platform

OpenMRS platform (V 2.0.5)

Modules

image

Specific modules that I had to upgrade before the whole thing worked: uicommons 2.4.0, appui 1.9.0-SNAPSHOT I also had a strange problem where for some reason my reportingapp module’s pom.xml initially listed that it was dependent on reportingcore-1.8.0 which is what it is in the main branch but in the iSantePlus repo the latest is 1.7.2-SNAPSHOT, so that was creating a problem but I re pulled from github and reinstalled it via deploying through the sdk directly in the cloned folder.

Changed Global Properties

registrationcore.local_mpi_identifierTypeMap.Biometrics Reference Code

from: 5a597bcc-26ad-11e8-b467-0ed5f89f718b:2.25.300969590489438061583573695579607328089:NI

to: e26ca279-8f57-44a5-9ed8-8cc16e90e559:2.25.300969590489438061583573695579607328089:NI

Audit Info

mine: UUID: 5d808438-ae32-4765-b853-fd2a1d4970eb

demo: UUID: 77f50b0b-db43-4708-bbfc-44093188dbe0

(I’m thinking actually I probably didn’t need to do the one above, the only thing I needed to set was the api implementation below)

registrationcore.mpi.implementation from: blank to: registrationcore.mpi.implementation.PixPdq

The global property registrationcore.mpi.personIdentifierId exists in my local and is blank but does not exist in demo.

I did not key in all the biometric global properties and just left that blank.

Remaining Errors

I getting the following errors:

ERROR - MetadataUtil.installMetadataPackageIfNecessary(157) |2018-06-11 17:10:09,932| Failed to install metadata package iSantePlus_Registration_Concepts-1.zip

Caused by: org.openmrs.api.db.DAOException: Error saving org.openmrs.Concept [1065AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA]

Caused by: org.openmrs.api.APIException: Cannot save Concept #1065. Failed with the following exception:

Caused by: org.openmrs.api.DuplicateConceptNameException: ‘Oui’ is a duplicate name in locale ‘fr’

Full error here: https://pastebin.com/Yt4V6E9n

Next Steps

  1. Set up a new OpenMRS platform 2.0.5 with the SDK
  2. Copy over modules and distribution.properties file (I want to leave this module as it is and work on a different one so I can always compare behaviour with this local version as I change the other local version)
  3. Slowly remove the biometrics files and changes from the second local version, document what I’m removing and regularly check whether everything is still working until I end up with only changes for the MPI functionality.

-Shao Yuan

1 Like

Awesome, thanks @shaoyuan!

Hi @craigappl and @pgesek,

So I’ve got slightly stuck again, I’m wondering if you have any thoughts on this:

Error

In the local iSantePlus server that I said I got working, (before any editing of the code) when I try to register a new patient I get an error about not being able to find the concept CIEL 1054

(This image shows the patient I am trying to register)

[Notice also in the registration date field somehow the css has become part of the output text, I have no idea why that is happening…]

The console output of the error is available here: https://pastebin.com/Yg5EjA50

And you can see the form (patient data) that is being submitted here: https://pastebin.com/gKtsQhTa

Line 55 is where it has

obs.CIEL:1054: CIEL:1067

When checking the concept dictionary on my local server I cannot find CIEL 1054, when I check it on the Demo server I can find it:

Just out of curiosity I searched for it on the PIH-Mirebalais concept dictionary and it was not there, however, it was referenced so I’m not sure how that works? (See red arrow)

Question

Would either of you know in which module or file is this concept instantiated/installed/ where it comes from?

I just noticed in the global properties the following values:

labintegration.hl7.regForm.civilStatus CIEL:1054

might it be from the lab integration module?

Some other discrepancies in the global properties I noticed:

  • metadatadeploy.bundle.version.org.openmrs.module.haiticore.metadata.bundles.HaitiAddressBundle is 10 on my local and 11 on the demo
    • This is interesting because I thought I installed the latest 1.1.0-snapshot version of the Haiti core which is higher than what is on the demo server yet this metadata bundle is lower on my local server?
  • registrationcore.mpi.personIdentifierId is blank on local, and this variable does not exist in the demo
  • registrationcore.mpi.personIdentifierTypeUuid is f54ed6b9-f5b9-4fd5-a588-8f7561a78401 on demo and this variable does not exist in the local server
  • registrationcore.openmrsIdenitfier.uuid is 05a29f94-c0ed-11e2-94be-8c13b969e334 on local, there is no such variable on demo

Would you know the source of these discrepancies? it seems like we are not running the same registrationcore module version? but we are running the same version! So I am confused…

Modules

These are the modules I have installed on my local server, those in yellow are below the versions in the demo server and those in green are above.

image

Additional Setup

I also had to add values to two more global variables (based on the global variables in the demo server):

  • xdssender.openmrs.username
  • xdssender.openmrs.username

Other Progress

I think I successfully cut out the fingerprint/biometric code in the isanteplus module and registration app module, I forked these repos from the isanteplus branches of them and have committed the changes there.

My Local edited version has the same behaviour as my local base version, (meaning exactly the same errors etc) so that is good. I hope to resolve these errors before proceeding with the edits on registrationcore.

Thanks for your time and input :slightly_smiling_face:

-Shao Yuan

Hi Shao,

The missing concepts mean the CIEL dictionary needs to be loaded. OpenMRS ships with a minimal CIEL dictionary. You can get the CIEL dictionary from the community dropbox. I would install everything in the following order: Start with a fresh installation of the OpenMRS WAR file without any modules, once installed, load in the CIEL dictionary to the SQL database, start tomcat to load OpenMRS (Indexing will take a long time), shutdown tomcat, load the modules into the modules folder, start tomcat and hopefully things will work.

Craig

1 Like

Hi Craig,

Thanks for the advice, while waiting to gain access to the CIEL dictionary I’ve been merging registrationcore, registrationapp and coreapps with the main branch, so far I’ve done registrationcore and registrationapp and they seemed to be working mostly fine but I faced some problems with coreapps.

When I run

$ mvn clean install

in the cloned isanteplus/openmrs-modules-coreapps directory I get the following errors: https://pastebin.com/gpSprsNk

in short, they are caused by two things:

Running org.openmrs.module.coreapps.fragment.controller.visit.RetrospectiveVisitFragmentControllerTest ERROR - RetrospectiveVisitFragmentController.create(76) |2018-06-15 13:24:49,637| Unable to add retrospective visit java.lang.NullPointerException at org.openmrs.module.coreapps.fragment.controller.visit.RetrospectiveVisitFragmentController.create(RetrospectiveVisitFragmentController.java:57) at org.openmrs.module.coreapps.fragment.controller.visit.RetrospectiveVisitFragmentControllerTest.test_shouldSetEndDateToCurrentTimeIfEndDateIsCurrentDay(RetrospectiveVisitFragmentControllerTest.java:160)

and

testShouldNotDisplayAdmissionLocationAfterEnteringDischargeDisposition(org.openmrs.module.coreapps.htmlformentry.EncounterDispositionTagHandlerComponentTest) Time elapsed: 17.249 sec <<< ERROR! java.lang.NoClassDefFoundError: Lorg/openmrs/module/xdssender/api/service/CcdService;

I asked @mogoodrich and we tried to fix the second error by adding the dependancy on xdssender-api into the pom.xml of the whole project as well as the pom.xml within the api folder. Do you have any ideas on where the issue might be coming from/ how we might fix it?

Best regards, Shao Yuan

Hi @dkayiwa,

I have a question relating to the emrapi module and the coreapps module

So the process that we are testing out now is to setup a new distribution 2.8.0 of the reference application using the SDK. After we run the setup, before running the server, we swap out the registrationapp, registrationcore, coreapps modules that we have been working on and merged, and emrapi-1.25.0-SNAPSHOT because that is required by the latest version of coreapps that we merged with (1.20.0-SNAPSHOT). We then proceed to do openmrs-sdk:run on the server for it to complete the setup.

However! when we create a patient on the fully set up server, we get an error that

Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Table ‘reference_8.conditions’ doesn’t exist

because this table doesn’t exist, because it was not set up by liquibase? referencing your commit

I was wondering why the conditions table wouldn’t be created in this case because I believe the platform that reference app 2.8.0 uses is 2.1.3? or at least in the openmrs-distro.properties it is stated that version=2.8.0 war.openmrs=2.1.3

What are your thoughts on how we should reconcile this?

Best regards,

Shao Yuan

That looks like a result of this bug: https://issues.openmrs.org/browse/TRUNK-5407

So platform 2.1.4 should have the fix.

@shaoyuan and since I don’t think 2.1.4 is out yet, you can try setting the war to 2.1.4-SNAPSHOT

Thanks for your input @mogoodrich and @dkayiwa,

That seems to have fixed it. So I used the sdk to set up the reference application 2.8.0 distribution, then used openmrs-sdk:deploy to deploy the platform 2.1.4-SNAPSHOT to the server and then added in my merged files and the conditions table was set up so I didn’t get that error anymore. thanks!

hi @pgesek,

So we were wondering about the mllp sender (openmrs-module-registrationcore/api/src/main/java/org/openmrs/module/registrationcore/api/mpi/pixpdq/MllpSender.java).

So we could change the global property: registrationcore.mpi.Hl7implementation from registrationcore.mpiHl7v2HttpSender to registrationcore.mpiHl7v2MllpSender in order to use it, but under what circumstances would someone want to do this? Is it for talking directly to OpenEMPI or also for routing through OpenHIM? and if so what other configuration/setup needs to be done to get this to work? We could test it and help create some documentation for it. Thank you!

Best regards,

Shao Yuan

Hello,

the MLLP sender will use the MLLP protocol as the name suggests. Yes, MLLP can be used to directly interface with OpenEMPI or other PIX/PDQ implementation through a TCP socket. Since MLLP does not provide any security on its own, additional methods for securing the channel need to be employed, a VPN or an SSH tunnel for example.

We opted for HTTP and basic authentication instead. The HTTP sender sends the HL7v2 message through HTTP to OpenHIM, which then routes the message to OpenEMPI using MLLP. In our case the OpenEMPI ports that accept MLLP messages (3600 & 3700) are not open to the world.

Regards, Paweł

Hi Pawel,

Thanks for that clarification!

So right now I’m looking over the tests and beginning to understand some of the code better and I have a few questions. I have three suggestions marked by the :star: but I’m not sure whether there are considerations that I am not seeing…

  1. Currently there is only one test class in the pixpdq folder, org.openmrs.module.registrationcore.api.mpi.pixpdq.PdqSimilarPatientsSearcherTest ideally there should be about one test class for each class in that folder correct?

  2. Referring to org.openmrs.module.registrationcore.api.mpi.pixpdq.PdqSimilarPatientsSearcher

    1. Referring to the find method, for constructing the identifierQueryParams for the PDQ message, only the last identifier to be added to the set of patient identifiers is used for the query, this leads to the following behaviour in the iSantePlus SEDISH Demo:

      1. Start Registering a patient, set ST Code to 6757, press enter, it submits the PDQ and gets back one similar patient, Jean Barron.

      2. Next, set NAT990999283 for Code National, press enter, it will submit this new ID as the identifierQueryParams.

      3. However, this is where I think the undesirable behaviour is: if you go back and change the ST Code and press enter, for example, changing it to 98761, it will submit a PDQ message but the identifier it sends is the Code National and not the recently changed ST Code, thus the matching patient it gets back from the MPI is still Jean Paul and not Jean loubeau

      • I also noticed that this pattern is different than for the given name and family name, where it will send one if it only has one, and will send both it has both.
      • :star: Might it be better to send all available Identifiers? According to the documentation, you can send multiple sets of identifiers in the query And I verified this by testing with a local OpenEMPI instance by sending a PDQ with both the ST Code and Code National.
      • In fact, I was wondering about the design considerations for having distinct identifierQueryParams vs nameQueryParams and sending the query with preference for name, and if no name is present then for identifier. (See lines 77 to 87 in find method)
      • :star: Would it be better to send all available demographic data? So as the registration form gets filled up you send more and more data of all the types? Right now, for example gender and addresses are not currently used for PDQ.
    2. In the toMatchList method, is there a reason that there are no checks for matching identifiers? Or phrased differently, that matching identifiers do not get added to the getMatchedFields List?

Thanks for your help in clarifying these things :slight_smile:

  • Shao Yuan

Also, a question about PixPatientExporter: (org.openmrs.module.registrationcore.api.mpi.pixpdq.PixPatientExporter)

Currently, the method exportPatient assumes that if the mpiPatient that is returned is not null, it means that a patient with the sam IDs already exists. What we have found is that if a patient with the same MPI Global Identifier Domain ID exists, it will not create the patient that was sent, but the PdqPatientFetcher will get back the patient originally in the MPI. I have explained the situation here: https://wiki.openmrs.org/display/docs/Registration+Module (that I am in the process of editing) under the section: Errors -> Patient Identifier Collision

given that there is no error in the PIX response (as shown in my screen shots in the other page) I think we can get rid of this undefined behaviour by doing a check to see if the name, gender and address fields of the patient passed as input to exportPatient is the same as that of the mpiPatient returned by pdqPatientFetcher.fetchMpiPatient? If they do not match we can throw a new MPI exception.

Does this seem like a good approach?