I wonder if we might be able to find a time next week to talk through possibilities for fingerprint integration architecture together. This is an area that I have been focused on over the past week, and would love to connect on some of our design ideas. I’m particularly keen to look at whether there are aspects that can be developed in common, even if the underlying fingerprint platform (M2Sys, Neurotechnology, etc) is different (i.e. can we abstract this away).
This is a perfect time to discuss the path forward with fingerprinting. Here are some updates on our side:
The fingerprint software deployed in the iSanté system is being upgraded from a proprietary fingerprint template to a ISO standard template.
M2Sys is working with UGP to convert the existing fingerprint templates to the ISO standard so they can be universally accessed regardless of the printing technology
We’re working with M2Sys to setup a dev/testing environment of their fingerprint server that will be deployed at each iSantéPlus clinic.
M2Sys has provided us documentation with the Web Services calls to the biometric server
Architecture
We will retain the system architecture where we have a windows 2008 server at each clinical site running the M2Sys technology that hosts the web service.
We need to develop a link between OpenMRS and the local M2Sys server in the patient registration, search and clinician facing dashboard.
We haven’t addressed this yet, but we need to figure out how we get fingerprints from the local fingerprint server to the national fingerprint server when there is internet connectivity and deduplicate patient ids when a newly registered patient already has a fingerprint in the national fingerprint server.
Core Dev Work
The Haiti Core module already has the biometric identifier added as a patient identifier. This will act as the target for the identifier, but I don’t yet understand the new M2Sys technology well enough to know what should be stored in this space.
We will need to build one or two widgets in OpenMRS that 1) allow us to register a patient and 2) allow us to search for a patient.
There will be a lot of dev work around interacting with the national fingerprint server. This is not yet scoped. We have the design specification for the legacy M2Sys replication process, but the design has limitations that prevent real-time interaction with the national fingerprint server.
I’ll send an email to setup a time for a Skype call next week.
This is another part of registration that the CDC is keen to see incorporated.
Is there any chance we could interact with the device RESTfully? I’m not expecting that devices would come with native REST support; rather, whether there are any devices with an API that would allow us to hack a RESTful wrapper around them whether to perform the operations headlessly or, if necessary, trigger the device’s UI to perform its operation.
I also know calls to localhost have issues. I’m picturing something like:
@burke and @craigappl, maybe this is a good topic for a design call spot this week?
Burke - to your point, this is the approach that we have started to take. Our Haiti team has built a small spring boot application that runs easily on the client, and is responsible for interacting with the fingerprint device. This exposes a REST API with endpoints: “/is-scanner-connected” and “/scan”. The “is-scanner-connected” endpoint can be used to determine whether fingerprinting options should be enabled in the UI. The “scan” endpoint waits for a fingerprint to be sent from the device, and extracts this into a template.
I started throwing together a small module last week to expose a REST endpoint on the server side to do identification and matching.
We are using the Verifinger Extended SDK from Neurotechnology and a Futronic FS88 scanner in our setup, but my hope is that we could build the client-side REST API and server-side REST API to be independent of the underlying fingerprint libraries, and then for us to be able to plug in Neurotechnology, M2Sys, or other libraries into this as an implementation requires. From what I recall, Ampath also uses Neurotechnology via Muzima, is that right?
I haven’t given much thought yet to how this would interact with a remote master fingerprint database, but I would expect this would be similar to how we would interact with any other element of an HIE, like an MPI for example.
Let me know if you all think that this would be useful to have in a design call spot week. Or I’m happy to have it at another mutually agreeable time in a public forum as well.
The UgandaEMR implementation already implements it this way. give us few weeks to fully document in order to share. You may be re-eventing the wheel if you start on a new project.
@craigappl thanks for all the insights. We definitely should talk in more detail about your plans as we are working on this now as well. Would be good to get an overview of what is known about the entire fingerprint architecture planned for Haiti so we can incorporate it into our plans.
I’m going to start working on a local fingerprint server spec early next week. (Note that interaction with a national fingerprint server is out of scope for now). I see the API work that you created in https://github.com/PIH/pih-biometrics. Are you working from a spec that you could share?
It looks like the latest commit includes a local fingerprint server that interacts through a REST API instead of relying on something created by Neurotechnology. As I look at the M2Sys software, their Bioplugin server seems to accomplish a lot of what has been developed in the pih-biometrics module so far.
Beyond this module, it seems like we would want to add a capability to registration-core and a widget to the registrationapp that can interact with these REST endpoints. Does that align with your thoughts?
@craigappl, thanks for pinging me on this. Collaborating on this, and sending an update on our progress, was on my list for late last week but there were a few things I wanted to tie off first.
To give a brief summary of how things evolved, I first started out creating an OpenMRS module that exposed REST web services, and which had an interface and default implementation of the various services required. My thought was that we could support any number of “engines” - Neurotechnology, M2Sys, etc - by providing the necessary implementations of the interface within this module. I got this working in a POC, and then decided that I wasn’t thrilled with the dependency on the various underlying platforms. For example, to just build the module, one needed to have the Neurotechnology SDK jars (and to run the unit tests, one needed a valid license). So I started down a different path that led to the pih-biometrics library that you see in github, which is more of a standalone fingerprint server +/- client application that uses the Neurotechnology SDK.
As you indicate, my thinking now is that we would mainly want to focus our shared energy on a few areas:
Client-side capabilities built into the registration app and/or coreapps which provide the relevant widgets to:
Search for a patient by fingerprint
Save fingerprint reference in a new patient registration
Enroll / associate a fingerprint with an existing patient (i.e. as an action from the registration dashboard)
Potentially delete/update an existing fingerprint.
TBD - associate multiple fingerprints or other modalities
Standard modeling, if possible:
Agree on a common model for storing a “fingerprint” in OpenMRS (eg. a patient identifier, person attribute, or complex obs that contains a reference to a unique subjectId in the fingerprint server)
REST representations of all relevant resources (template, match, etc), and either standard endpoints that any compatible fingerprint server would need to support, or a way to configure the client UI components with different adapters
To that latter point, one approach I have been considering is whether we could create a Biometric Javascript library (likely using Anguar components), written in such a way that we could develop a common set of UI components for the registration app, but which could be configured as needed by an implementation with a JS object that contains it’s service-specific code. For example, an object that has an “enroll” function, which in our instance would call an endpoint writting for pih-biometrics, and in your case would utilize a different endpoing in the M2Sys server.
I’d love to get a time on the calendar to actually talk about this a bit. Is there any time that could work for you and your team this week?
My current task is to scope the fingerprint solution so we can define a direction and set of tasks. I expect we will have to create a lightweight wrapper around the M2Sys BioPlugin commands that will allow us to interact with the server based on the standard API endpoints that we create in registration-core.
Additional Client Side Capabilities:
We will need to be able to change the patient fingerprint identifier when we merge patients in OpenMRS.
In the long run, we would want to have a function that searches the OpenMRS fingerprint database for duplicates and returns the registrationID (or patient ID) to OpenMRS so we could compare those records for merging.
Notes on standard modeling:
The fingerprint in the M2Sys system is a base64 string of the ISO template. Each finger has a position number and we transport that by xml.
Do we need to store the fingerprint in OpenMRS, or should we just store a link to the identifier from the fingerprint server? In the M2Sys technology, this is called a “RegistrationID” I would prefer to store this registration ID as a patient identifier. In the M2Sys technology, we have to generate this ID and post it with the fingerprint scan.
We need to also add a REST endpoint that shows whether the fingerprint server is actively connected to the OpenMRS instance. If it’s not connected, we will control UI elements.
The M2Sys technology requires that we collect two prints “LeftEnrollTemplate, LeftFingerType” as well “RightEnrollTemplate, RightFingerType”
I’m still in the information gathering stage on the M2Sys technology. I don’t yet understand how the fingerprint reader interacts on the client side with the browser. I have an email out to them to get more information.
Looking forward to discussing in person. I’ll try to put together some additional design documentation, but in an attempt to respond to your points, see below:
Yes, this is essentially what I have done in pih-biometrics for Neurotechnology. Neurotechnology also comes with an out of the box product, called NServer. I chose to simply roll my own since the API was pretty straightforward to work with and it gives me more control and understanding of what is going on. But it is really just a lightweight wrapper application of the underlying Neurotechnology service, in order to expose a set of REST web services that we can access from OpenMRS. So I think we are aligned here, and even though we’ll both have to write our own wrapper applications, I hope we can come up with a harmonized API.
I hadn’t really thought much about this merge use case, but it is an interesting one. One question overall is - is it a problem (or could it be a benefit) to have multiple fingerprint records associated with a single patient? Do we really need to merge these, or can we keep them both as separate identifiers, which might both return valid matches?
I have also based our API thus far on a base64 encoded text representation of the template. Neurotechnology is capable of supporting (and converting between) it’s internal proprietary template format, ISO, and ANSI, from what I can tell. I’m not sure if there is an advantage to storing fingerprints in ISO format if matching is more efficient and/or accurate in the proprietary format, and as long as it can import/export templates in ISO format for integration with other systems. I’d be interested in any thoughts or experience that you or others may have on this.
In the design that I have been working up, the fingerprint itself is not stored in OpenMRS. It is not even necessarily in MySQL (I have used Sqllite in my initial setup). The way I have been thinking about it - the fingerprint server is conceptually distinct from OpenMRS, and could even contain records of persons that are not stored in the OpenMRS system. Perhaps there are other systems that need fingerprinting at a facility that do not exactly overlap. So I have deliberately tried to keep any notion of an “OpenMRS person/patient” out of the fingerprinting server itself. The main drawback to this that I can tell is that registering a patient and/or storing a reference to the patient’s fingerprint is transactionally separate from registering the fingerprint with the fingerprint server. So it is theoretically possible to end up with patients with no fingerprints registered, or fingerprints that are registered before a patient is created (and perhaps that patient registration never completes).
What M2Sys calls the “RegistrationID” is called the “SubjectID” in Neurotechnology, and so this is the verbage that I have used in my initial design. We can call it whatever we think is most correct / whatever the standard in the industry is, if there is a generally agreed one, it doesn’t really matter to me, but it sounds like we are aligned around this.
In the pih-biometrics system, I associate a completely independent SubjectId/RegistrationId with the fingerprint, and then expect we will associate that with the patient. Whether this is stored as a patient identifier, person attribute, or obs - I’m not sure. A patient identifier makes decent sense to me, and if you have this requirement, then that should work. I don’t know if you have particular constraints around the nature of this SubjectId/RegistrationId. In my initial pass, I am simply generating a new UUID and using that, and I haven’t seen any indication yet that this is a bad choice.
Agreed. I have added this to pih-biometrics as the “status” endpoint, which contains various properties (which we will likely need to discuss and agree upon), including “status = AVAILABLE/NOT_AVAILABLE” as a first pass.
This is an area I am very interested in discussing more and understanding best practice. Neurotechnology seems to support bundling multiple modalities into a single template - one creates a subject, and adds one or more fingers with or without a position (one can also add things like palm prints, iris scans, voice recognition, etc to this as well), and then generates a template from all of it . Of course, one could also decide to limit a template to just a single fingerprint, and generate a template from this, and do this several times for the same patient, storing different patient identifiers for different fingerprints, though in this case I might advocate more strongly for storing these as Obs, not sure.
Hope this helps and we can find a common way forward!
Do you have information on how the fingerprint reader interacts with the web browser? This is a gap in my understanding with the M2Sys technology. Effectively, we need to use JavaScript to build the REST call to enroll the patient on the client side. As you mentioned previously, this would likely be a widget in coreapps.
Also, does your fingerprint server have any type of authentication needed? (I don’t see any in the M2Sys docs)
This is down the road a bit…I think the patient identifier makes most sense when we are trying to associate the fingerprint identifier with the national fingerprint system or the Master Person Index. The HL7 v2 message we’re targeting for the MPI has a patient identifier domain in the PID section that we would want to use to push this identifier to the MPI. So, when a patient is registered in the clinic, we would create the PID section with the iSantePlus ID, the code national, the code ST and the BiometricID. We haven’t yet worked out the process of moving the template.
Yes, in our case we are building this out in a smaller wrapper application as well (in fact, I’ve put it in the same application, not sure if this is what I want long term or not). Basically, the idea is that any client that wishes to utilize the fingerprint scanner would need to be set up to run this small wrapper application (manually or as a service). This exposes a web service (just like the fingerprint server) with a few different endpoints. See the README here.
Right, so basically we’d have a web service at “localhost” for the scanning operation, and a web service coming from the server for the matching/enrollment operations, and the widgets that we build into coreapps would need to hit these two different web services. This will require working out some CORS stuff, but should be doable.
I don’t think there is anything built into Neurotechnology, and I haven’t added anything to our wrapper application (yet), but this is something I was wondering about as well. We really just need a shared secret between the OpenMRS server and the fingerprint server, so I hope this will be relatively easy to add as needed in within our lightweight wrapper applications.
The Neurotechnology VeriFinger SDK that we are using supports this. From a snippet of their documentation:
The Fingerprint BSS component allows conversion between Neurotechnology proprietary fingerprint templates, ISO/IEC 19794-2:2005 (/2011), ANSI/INCITS 378-2004 and ANSI/NIST-ITL templates
What I haven’t determined yet if is it makes sense for us to do a wholesale adoption of ISO as a format for all operations (scanning, matching, enrolling, etc), or if we should stick with the Neurotechnology proprietary format in general, and then convert to/from ISO as needed when use cases arise for needing to share fingerprint data externally. I have this as an open question we have yet to resolve. A few considerations:
I’ve read that using the internal proprietary format provides somewhat better performance, as it is optimized for the library
The components needed to convert to ISO may have additional licensing costs than would otherwise be necessary at the client-side. It might make more sense to minimize the licensing requirements at each scanning client, and instead do ISO conversion as needed at the server-side, as this might be more cost-effective.
I’d be interested in your thoughts and any experiences from the community as to these considerations.
