Integration with OpenEMPI

@craigappl, FYI - I tried looking at this recording but it requires special permissions.

Thanks @mseaton, I updated the link and tested it in Chrome incongnito.

Thanks @wyclif

I finally got around to reviewing this yesterday. Your assessment is correct and we are going to work on improving how the matching algorithm is handled.

Here’s the background: The PDQ search needs to build a HL7 message while the form is being filled by the user (In the PDQSimilarPatientsSearcher controller). This follows the current practice in OpenMRS that searches the database for patients. When the patient’s name is entered, the controller builds the message and the query is fired to the EMPI which returns ‘n’ results. These results are parsed and scored in the match list.

Our next step is to review how we build the query message to streamline the number of results that are returned from the MPI so that we can leave that logic up to the MPI. We expect to do this over the next few weeks.

Craig

FYI @pgesek, @nathaelf

Sounds good and thanks for the background info!

@craigappl, we haven’t lost interest in the great work your team has been doing with MPI integration. :slight_smile:

I see the iSantéPlus openmrs-registration-core is 100 commits ahead of the community repo. How can we help get this work folded back into the community repo?

1 Like

@nathaelf, do you have any thoughts on this? Thanks! -Burke :burke:

1 Like

@nathaelf or @craigappl I don’t want to lose this thread. Any thoughts on @burke post about folding iSantéPlus back into the community repo?

Hi,

I’m no longer on the iSantePlus team and don’t have the technical skills to perform the merge. The challenge with the merge is that all of the commits have been made to master along with a few other enhancements that have to do with fingerprints. The person would have to either go through and cherry-pick all of the commits related to the MPI or go through and manually rework the files that have been touched. I’m happy to talk about the work if anyone is able to pick it up.

Craig

1 Like

Hi @craigappl and @pgesek,

I watched the video training session you guys recorded and am trying to set up a the docker container for openEMPI (GitHub - SEDISH/ohie-cr: OpenEMPI)

I cloned the repo, went inside and ran the command:

$ docker build -t uwitech/ohie-cr .

and received the following message:

Step 1/28 : FROM uwitech/ohie-base Get https://registry-1.docker.io/v2/uwitech/ohie-base/manifests/latest: unauthorized: incorrect username or password

would greatly appreciate if someone could give me some advice on how to proceed? Thank you so much :slight_smile:

Hi @craigappl @pgesek @mogoodrich @mseaton @ball,

Just to highlight some sticky points I’m facing now and the things I’ve tried so far.

Regarding the above message about the unauthorised error, it was that I had not logged in to docker via the cli, and that was easily fixed. (thanks to guidance from @mseaton) (https://docs.docker.com/engine/reference/commandline/login/)

Goals


My overarching goal is to merge the MPI functionality into the main branch of the reference app and associated modules, e.g. registration core, registration app.

My intermediate goal was to set up all the necessary components, I.e. iSantePlus client, OpenEMPI Docker, etc (I know there are some other necessary components e.g. XDS, PIX, PDQ endpoints? but I haven’t figured out what all those things are yet) but basically I wanted to set up all these components on my local machine and be able to use all the MPI functionality and have a “ground truth” where everything works and start refactoring from there.

Questions/ Requests


  1. Is there a way to build the iSantePlus m2Sys Biometrics Module?

    • Demo/Sample environments/instances of BioPlugin server 8 and Cloudscanner? And is there a way to use this module without the hardware of the M2Sys fingerprint reader? e.g. just a way to feed the system IDs/Keys?
    • Looking at the code the MPI functionality is very intertwined with the fingerprint functionality and the iSantePlus registration core and app modules would not work without it.
  2. Are there other training videos you guys made on these modules/this functionality that could help me to understand the concepts and implementation? I watched the video linked above twice but I’m very new to OpenMRS and all of this so it would be really helpful to have more guidance on how all of this works and all the parts fit together :slight_smile: If there are sequence diagrams to show how all the classes interact and in what flow that would be very very awesome, or if we could set up a time for a call @pgesek

What I’ve Tried So Far


1: Set up Open EMPI Docker

(Done)

2: Set up iSantePlus instance locally

(Roughly following the README on https://github.com/IsantePlus/openmrs-module-isanteplus) (Thanks to guidance from @mogoodrich)

  • Used SDK to set up an OpenMRS platform (V 2.0.5)
  • Copied modules from Reference Application (V 2.5) and replaced those that were created by the SDK for the platform
  • Updated the openmrs-distro.properties with that list of modules
  • Used mvn openmrs-sdk:deploy to install the htmlformentry 3.3.2 module
  • Git cloned the isanteplus module
  • Tried to use the sdk to install the isanteplus module, but got an error stating that it required the xds sender module.
  • Git cloned the xds sender module
  • mvn clean installed the xds sender module (successful)
  • Used the sdk to install the isanteplus module (successful)
  • Ran the server, it seemed to work, had new UI but it did not look like the UI of the demo server that @ball shared with me (OpenMRS logo not iSantePlus logo, no outgoing exceptions icon on home page etc)
  • Tried to add a patient through my local instance and then see if it was added to Open EMPI but it didn’t work.

3: Set up iSantePlus instance locally (Beyond the instructions)

My hypothesis was that this was not the complete set up of the iSantePlus server even though I followed the instructions in that module because I had not installed the other modules listed in Notes on iSantéPlus Activity and Push/Pull Architecture with SEDISH by @craigappl, namely: isanteplus/registrationcore, isanteplus/registrationapp, isanteplus/xds-sender and isanteplus/outgoingmessageexceptions modules. So I thought I should install them on the server

  • Used sdk to install the xds sender module (successful)
  • Used sdk to install the outgoing exceptions module (successful)
  • Used sdk to install the registration core module (successful)
  • Tried to use the sdk to install the registration app module (Unsuccessful)
    • It produced an error saying that it required the m2sys biometric module
    • Because I dont have a BioPlugin server (as explained above) this is where I got stuck…
  • I thought I might continue to try to get the server to work even without the isanteplus versions of the registration core and registration app and went down several error fixing rabbit holes but ultimately realised that I was getting too many errors that I didn’t know how to fix and that it probably wouldn’t work anyways…

I’m thinking now that if I am not able to have a working local instance of iSantePlus running and then work backwards from there, I might have better luck starting from the latest core OpenMRS modules, understand the mechanics of the iSantePlus code super thoroughly and then work forwards from there…? Thoughts anyone?

Anyways, thanks for your time and effort in reading my clumsy attempts at this, hope you will be able to share some advice :slight_smile:

-Shao Yuan

1 Like

Hi Shao,

I’m sorry I haven’t been around to support this. I am attending a conference and will be back online tomorrow. I will help get you unblocked. I’m located in Seattle and can also schedule a call for Monday. Please send me a PM.

Craig

1 Like

Hi All,

Just a quick update before I move on to the next step. Following @craigappl’s advice I continued to try to set up the local iSantePlus instance and set the global properties to match that of the iSantePlus hosted demo to hit the same openHIM/ Interoperability endpoints. I’m happy to say that I’ve got it working. I’m just going to detail the versions of modules I have installed as well as the global properties I changed for future reference.

Platform

OpenMRS platform (V 2.0.5)

Modules

image

Specific modules that I had to upgrade before the whole thing worked: uicommons 2.4.0, appui 1.9.0-SNAPSHOT I also had a strange problem where for some reason my reportingapp module’s pom.xml initially listed that it was dependent on reportingcore-1.8.0 which is what it is in the main branch but in the iSantePlus repo the latest is 1.7.2-SNAPSHOT, so that was creating a problem but I re pulled from github and reinstalled it via deploying through the sdk directly in the cloned folder.

Changed Global Properties

registrationcore.local_mpi_identifierTypeMap.Biometrics Reference Code

from: 5a597bcc-26ad-11e8-b467-0ed5f89f718b:2.25.300969590489438061583573695579607328089:NI

to: e26ca279-8f57-44a5-9ed8-8cc16e90e559:2.25.300969590489438061583573695579607328089:NI

Audit Info

mine: UUID: 5d808438-ae32-4765-b853-fd2a1d4970eb

demo: UUID: 77f50b0b-db43-4708-bbfc-44093188dbe0

(I’m thinking actually I probably didn’t need to do the one above, the only thing I needed to set was the api implementation below)

registrationcore.mpi.implementation from: blank to: registrationcore.mpi.implementation.PixPdq

The global property registrationcore.mpi.personIdentifierId exists in my local and is blank but does not exist in demo.

I did not key in all the biometric global properties and just left that blank.

Remaining Errors

I getting the following errors:

ERROR - MetadataUtil.installMetadataPackageIfNecessary(157) |2018-06-11 17:10:09,932| Failed to install metadata package iSantePlus_Registration_Concepts-1.zip

Caused by: org.openmrs.api.db.DAOException: Error saving org.openmrs.Concept [1065AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA]

Caused by: org.openmrs.api.APIException: Cannot save Concept #1065. Failed with the following exception:

Caused by: org.openmrs.api.DuplicateConceptNameException: ‘Oui’ is a duplicate name in locale ‘fr’

Full error here: Failed to install metadata package iSantePlus_Registration_C - Pastebin.com

Next Steps

  1. Set up a new OpenMRS platform 2.0.5 with the SDK
  2. Copy over modules and distribution.properties file (I want to leave this module as it is and work on a different one so I can always compare behaviour with this local version as I change the other local version)
  3. Slowly remove the biometrics files and changes from the second local version, document what I’m removing and regularly check whether everything is still working until I end up with only changes for the MPI functionality.

-Shao Yuan

1 Like

Awesome, thanks @shaoyuan!

Hi @craigappl and @pgesek,

So I’ve got slightly stuck again, I’m wondering if you have any thoughts on this:

Error

In the local iSantePlus server that I said I got working, (before any editing of the code) when I try to register a new patient I get an error about not being able to find the concept CIEL 1054

(This image shows the patient I am trying to register)

[Notice also in the registration date field somehow the css has become part of the output text, I have no idea why that is happening…]

The console output of the error is available here: register patient error console output - Pastebin.com

And you can see the form (patient data) that is being submitted here: https://pastebin.com/gKtsQhTa

Line 55 is where it has

obs.CIEL:1054: CIEL:1067

When checking the concept dictionary on my local server I cannot find CIEL 1054, when I check it on the Demo server I can find it:

Just out of curiosity I searched for it on the PIH-Mirebalais concept dictionary and it was not there, however, it was referenced so I’m not sure how that works? (See red arrow)

Question

Would either of you know in which module or file is this concept instantiated/installed/ where it comes from?

I just noticed in the global properties the following values:

labintegration.hl7.regForm.civilStatus CIEL:1054

might it be from the lab integration module?

Some other discrepancies in the global properties I noticed:

  • metadatadeploy.bundle.version.org.openmrs.module.haiticore.metadata.bundles.HaitiAddressBundle is 10 on my local and 11 on the demo
    • This is interesting because I thought I installed the latest 1.1.0-snapshot version of the Haiti core which is higher than what is on the demo server yet this metadata bundle is lower on my local server?
  • registrationcore.mpi.personIdentifierId is blank on local, and this variable does not exist in the demo
  • registrationcore.mpi.personIdentifierTypeUuid is f54ed6b9-f5b9-4fd5-a588-8f7561a78401 on demo and this variable does not exist in the local server
  • registrationcore.openmrsIdenitfier.uuid is 05a29f94-c0ed-11e2-94be-8c13b969e334 on local, there is no such variable on demo

Would you know the source of these discrepancies? it seems like we are not running the same registrationcore module version? but we are running the same version! So I am confused…

Modules

These are the modules I have installed on my local server, those in yellow are below the versions in the demo server and those in green are above.

image

Additional Setup

I also had to add values to two more global variables (based on the global variables in the demo server):

  • xdssender.openmrs.username
  • xdssender.openmrs.username

Other Progress

I think I successfully cut out the fingerprint/biometric code in the isanteplus module and registration app module, I forked these repos from the isanteplus branches of them and have committed the changes there.

My Local edited version has the same behaviour as my local base version, (meaning exactly the same errors etc) so that is good. I hope to resolve these errors before proceeding with the edits on registrationcore.

Thanks for your time and input :slightly_smiling_face:

-Shao Yuan

Hi Shao,

The missing concepts mean the CIEL dictionary needs to be loaded. OpenMRS ships with a minimal CIEL dictionary. You can get the CIEL dictionary from the community dropbox. I would install everything in the following order: Start with a fresh installation of the OpenMRS WAR file without any modules, once installed, load in the CIEL dictionary to the SQL database, start tomcat to load OpenMRS (Indexing will take a long time), shutdown tomcat, load the modules into the modules folder, start tomcat and hopefully things will work.

Craig

1 Like

Hi Craig,

Thanks for the advice, while waiting to gain access to the CIEL dictionary I’ve been merging registrationcore, registrationapp and coreapps with the main branch, so far I’ve done registrationcore and registrationapp and they seemed to be working mostly fine but I faced some problems with coreapps.

When I run

$ mvn clean install

in the cloned isanteplus/openmrs-modules-coreapps directory I get the following errors: https://pastebin.com/gpSprsNk

in short, they are caused by two things:

Running org.openmrs.module.coreapps.fragment.controller.visit.RetrospectiveVisitFragmentControllerTest ERROR - RetrospectiveVisitFragmentController.create(76) |2018-06-15 13:24:49,637| Unable to add retrospective visit java.lang.NullPointerException at org.openmrs.module.coreapps.fragment.controller.visit.RetrospectiveVisitFragmentController.create(RetrospectiveVisitFragmentController.java:57) at org.openmrs.module.coreapps.fragment.controller.visit.RetrospectiveVisitFragmentControllerTest.test_shouldSetEndDateToCurrentTimeIfEndDateIsCurrentDay(RetrospectiveVisitFragmentControllerTest.java:160)

and

testShouldNotDisplayAdmissionLocationAfterEnteringDischargeDisposition(org.openmrs.module.coreapps.htmlformentry.EncounterDispositionTagHandlerComponentTest) Time elapsed: 17.249 sec <<< ERROR! java.lang.NoClassDefFoundError: Lorg/openmrs/module/xdssender/api/service/CcdService;

I asked @mogoodrich and we tried to fix the second error by adding the dependancy on xdssender-api into the pom.xml of the whole project as well as the pom.xml within the api folder. Do you have any ideas on where the issue might be coming from/ how we might fix it?

Best regards, Shao Yuan

Hi @dkayiwa,

I have a question relating to the emrapi module and the coreapps module

So the process that we are testing out now is to setup a new distribution 2.8.0 of the reference application using the SDK. After we run the setup, before running the server, we swap out the registrationapp, registrationcore, coreapps modules that we have been working on and merged, and emrapi-1.25.0-SNAPSHOT because that is required by the latest version of coreapps that we merged with (1.20.0-SNAPSHOT). We then proceed to do openmrs-sdk:run on the server for it to complete the setup.

However! when we create a patient on the fully set up server, we get an error that

Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLSyntaxErrorException: Table ‘reference_8.conditions’ doesn’t exist

because this table doesn’t exist, because it was not set up by liquibase? referencing your commit

I was wondering why the conditions table wouldn’t be created in this case because I believe the platform that reference app 2.8.0 uses is 2.1.3? or at least in the openmrs-distro.properties it is stated that version=2.8.0 war.openmrs=2.1.3

What are your thoughts on how we should reconcile this?

Best regards,

Shao Yuan

That looks like a result of this bug: https://issues.openmrs.org/browse/TRUNK-5407

So platform 2.1.4 should have the fix.

@shaoyuan and since I don’t think 2.1.4 is out yet, you can try setting the war to 2.1.4-SNAPSHOT

Thanks for your input @mogoodrich and @dkayiwa,

That seems to have fixed it. So I used the sdk to set up the reference application 2.8.0 distribution, then used openmrs-sdk:deploy to deploy the platform 2.1.4-SNAPSHOT to the server and then added in my merged files and the conditions table was set up so I didn’t get that error anymore. thanks!