Lucene patient search by identifier of a particular identifier type

We have tried integrating lucene search for patient with our Bahmni patient search api and it gave us a good performance boost.

But one thing that we weren’t able to achieve is, searching patient by identifier of a particular identifier type. We have extra identifiers configured and we sometimes want to limit the search only to a particular identifier type.

I tried making patientIdentifierType as a lucene field in PatientIdentifier.class and we were able to search by a particular identifier type. Can this be incorporated in openmrs-core? I can raise a pull request.

1 Like

Having this as a configurable option looks great. So a pull request would be cool! :slight_smile:

Please create an issue in JIRA and then a pull request so we can have a look and comment. Thanks! :slight_smile:

Out of curiosity, could you please share results of benchmarks you did for patient search? It may motivate more people to move to platform 2.1.0 :slight_smile:

1 Like

Patient search api in Bahmni performs both partial and exact match search by identifier, name, person attributes, patient program attributes.

As the first step, I tried integrating lucene for patient search using identifier (partial and exact match) in our api.

Below is the info on benchmark for patient search by identifier: Total number of patients: 186944 Index size on disk for PatientIdentifier: 8.2 MB Response time of Lucene patient search api : 172 ms Response time of our old patient search api : 14577ms

Index size on disk for PersonName: 40MB Index size on disk for PersonAttribute: 6.8MB

3 Likes

Thanks @preethi_s for sharing! It’s quite impressive boost in performance. By chance did you measure how long it took to create the index at startup?

Please remember to share here a link to the issue once you create it.

Sounds to me like @preethi_s is saying that they’ve implemented their own lucene based patient searching and not really using that which was recently added to core, is this true?

@wyclif yes you are right. we have slightly different usecases like searching patient by partial match of identifier, by patient program attributes, etc. So we have our own patient search api.

@raff I did not notice the time taken to build lucene index for the first time. I can update you when we try in a different server sometime.

I have raised a issue for supporting search by particular identifier type: TRUNK-5012 . I will pick it up soon.

Also when can we expect openmrs 2.1.0 to be released?

@preethi.s, I understand you must have leveraged at least the Lucene indexing of patients that we setup for platform 2.1.0.

When integrating Lucene search with your search API, did you use Lucene directly, Hibernate Search or LuceneQuery? I’d highly recommend you to look at LuceneQuery and its usage in e.g. PersonLuceneQuery as it’s targeted at simplifying and optimizing queries.

Would you mind pointing me to your search API code so that I could try to look into incorporating it in openmrs-core? I’d love us to be in a place where openmrs-core search API is flexible enough to meet needs of most implementations. We could work together on improving performance and it would benefit more.

I don’t know the platform 2.1.0 release date. I asked the same question here.

1 Like

@raff One of the reasons for writing our own patient lucene search is that our api supports wildcard search of identifier, which was not available in findPatients.

I have taken a look at LuceneQuery class that was already there but I went ahead and used Hibernate-search’s dsl way of querying things since I felt it was easy and it was just a spike on how patient search using lucene would be performant for Bahmni.

Here is the code that I have written.

Hi @preethi.s and #bahmni team.

We have an issue with the way the patient identifier types are being queried. For i18n related reasons we would like not to rely on their names but on their UUID instead (see also this thread on the same subject). So I went ahead and made some changes to the Lucene query fetching the patient identifiers but I can’t make it work.

Perhaps some Lucene experts or yourself could shed some light, or maybe @raff?

This is and example of a query that works taken out of the integration tests:

+identifierAnywhere:*hos1225* +voided:false +patient.voided:false
  +identifierType.name:"openmrs identification number"

And this my modified query that does not return anything:

+identifierAnywhere:*hos1225* +voided:false +patient.voided:false
  +identifierType.uuid:1a339fe9\-38bc\-4ab3\-b180\-320988c0b968

Clearly the filter +identifierType.uuid:1a339fe9\-38bc\-4ab3\-b180\-320988c0b968 is not doing the job, any idea why?

We don’t store uuids in the Lucene index. https://github.com/openmrs/openmrs-core/blob/master/api/src/main/java/org/openmrs/BaseOpenmrsObject.java#L74 would have to be annotated with @Field. I’d support that change.

Thanks @raff, we have solved the issue that lead to the above by another route. So I’ll leave that on hold for now. But thanks, I was suspecting that it could be something with the Lucene config as I really had no more clues as to how to make that query work.

Hi @raff am trying to extend the work @preethi.s had started on leveriging lucene for patient search on Bahmni. It would be optimal to use the openmrs-core search API to handle this but I am encountering the same problem of wildcard searches on patient identifier. Is this something that is being worked on or identified to be worked on in future?

Hi, you are more than welcome to add support for that in core!

It’s best to start from creating an issue in the CORE project describing the use case. I can add more details to assist you on what needs to be changed in code, if needed. Just mention me in the issue.

@ekirapa is this what you’re looking for?

org.apache.lucene.search.Query identifierQuery = queryBuilder.keyword()
  .wildcard().onField("identifierAnywhere").matching("*" + identifier.toLowerCase() + "*").createQuery();

Basically we want to have this logic somewhere in OpenMRS Core as well.

As @raff suggested, would you mind creating a TRUNK ticket for this? Let me know when it is done so that I can link it to BAH-273.