Implementing full-text search in OpenMRS

sopnopriyo · October 7, 2016, 2:31am

This is Shahin here , studying bachelor of software engineering in University of Malaya, Malaysia. I am very interested to contribute to OpenMRS as QA Engineer. I have sound knowledge in multiple programming languages such as Java, PHP, Python and some popular web framework like Spring, Laravel. Recently, I have build-up knowledge in Elastic Search. I think it would be great if I can use that knowledge to implement full-text search functionality in OpenMRS.

teleivo · October 7, 2016, 10:45am

Welcome!!

OpenMRS uses Apache Lucene to provide indexed search.

You can look at the issues in JIRA and look for related things to work on:

https://issues.openmrs.org/issues/?jql=project%20%3D%20TRUNK%20AND%20text%20~%20"lucene"

Havent found any wiki on the topic, but you could browse the code.

For ex.

github.com

openmrs/openmrs-core/blob/master/api/src/main/java/org/openmrs/api/db/hibernate/search/LuceneQuery.java

/**
 * This Source Code Form is subject to the terms of the Mozilla Public License,
 * v. 2.0. If a copy of the MPL was not distributed with this file, You can
 * obtain one at http://mozilla.org/MPL/2.0/. OpenMRS is also distributed under
 * the terms of the Healthcare Disclaimer located at http://openmrs.org/license.
 *
 * Copyright (C) OpenMRS Inc. OpenMRS is a registered trademark and the OpenMRS
 * graphic logo is a trademark of OpenMRS Inc.
 */
package org.openmrs.api.db.hibernate.search;

import java.util.ArrayList;
import java.util.Collection;
import java.util.Collections;
import java.util.HashSet;
import java.util.List;
import java.util.Set;

import org.apache.lucene.analysis.Analyzer;
import org.apache.lucene.index.Term;

This file has been truncated. show original

Maybe someone else can give some more insight

darius · October 7, 2016, 3:29pm

@sopnopriyo, glad to heard you’re interested! If you want to work on implementing full-text search, you should call that work development (not QA Engineering).

In OpenMRS Platform 1.11 we started using Lucene for searching concepts. This was a large piece of work, and you should be able to see all the details from the link here: https://issues.openmrs.org/browse/TRUNK-2999

What we’d like to do is to apply this same approach to patient searching. I see that you came across the ticket https://issues.openmrs.org/browse/TRUNK-425, and this is where the work should happen. (That ticket seems to have some very old patches on it, that predate the work on TRUNK-2999, so you can peek at them, but probably don’t use them directly.)

So, at first I suggest you take a look at the work that was done as part of TRUNK-2999, and then ping back again when you have an understanding of it. (Ultimately we’ll want to specify some clear Acceptance Criteria about how we’ll want a lucene-backed patient search to work.)

darius · October 7, 2016, 3:31pm

PS- @sopnopriyo, I moved this message to the general Development category. If you end up wanting to talk about QA, then start another thread in the QA category, but as we’re talking about adding a new feature, this belongs in Dev.

sopnopriyo · October 8, 2016, 5:21am

Hi @darius

Thanks for your reply. I will try to understand the code and get back to you on this.

darius · October 10, 2016, 2:24pm

@sopnopriyo, it’s probably worth saying: if you’re not familiar with hibernate this may be a bit tricky; so if you’re running into issues, then please do reach out for help on the IRC channel.

sopnopriyo · October 11, 2016, 5:42am

Hi @darius, I am familiar with JDBC template and Hibernate . I am having some issues with setting up the development environment. Once done, I will ping you.

Thanks, Shahin

darius · October 11, 2016, 8:20pm

I recomment using the SDK (https://om.rs/sdk) for setting up your dev environment.

sopnopriyo · October 12, 2016, 12:06pm

Still No luck. Ended up with the following error

darius · October 12, 2016, 4:36pm

I think you are behind a proxy, and you need to configure maven for this. Google should tell you how. (It’s probably documented on some other OpenMRS Talk thread also.)

sopnopriyo · October 15, 2016, 12:45pm

Hi @darius , Finally I have been able to set up the environment. Besides, I have looked into the trunk you mentioned earlier. Let’s discuss about the task.

Thanks, Shahin

darius · October 15, 2016, 5:23pm

@sopnopriyo, do you have specific questions? (I’m not actually familiar with the code in question, although I know the general idea of what we’re trying to do.)

Can you suggest, based on peeking at the way this was implemented for concepts, how you suggest to implement it for patient search?

Then anybody can reply to specific questions or proposals.

sopnopriyo · October 15, 2016, 6:59pm

Hi @darius,

Thanks for your reply. The coding part I can manage by myself I guess since there is already some implementations and If I need help I will inform you later for sure .

Let’s discuss about the requirements and overall picture of the task.

Then, I will come up with a plan for implementation and discuss with you.

Thanks, Shahin

darius · October 25, 2016, 8:47am

@sopnopriyo, sorry for the delay, I was traveling and then at all-day meetings last seek.

I have updated the ticket description on TRUNK-425 to include some Acceptance Criteria describing the work. (I wrote these quickly so they are not perfect!)

Is this good enough to get started? Do you want to discuss further?

sopnopriyo · October 26, 2016, 2:06am

@darius, No worries. As a starting point , the supplied information is okay. So, I am going to start with creating index for Patient Entity and will keep you inform time to time

Thanks, Shahin

darius · October 26, 2016, 2:46am

One thing to keep in mind: Patient extends Person in our domain model, and really we’d like lucene-based searching with both Patient and Person.

(I haven’t stated this very clearly in the ticket, and I haven’t had time to think through all the implications.)