Travis CI failure after upgrading lucene sub Libraries

Hello friends, can someone advice me on how to go about this issue https://issues.openmrs.org/browse/TRUNK-5528? basing on the ticket description am supposed to upgrade the

org.apache.lucene:lucene-queries … 4.10.4 -> 8.0.0
 

org.apache.lucene:lucene-queryparser … 4.10.4 -> 8.0.0

I have upgraded the sub libraries to the current version as per this pull request https://github.com/openmrs/openmrs-core/pull/2947/commits/80e73a63258471202083c48e7bc04c9f85914b64 but the travis CI fails. Kindly how can I sort out the travis failure?

@mozzy, @ruhanga, @k.joseph, @ibacher, @bistenes

Whenever i see this error “can not find symbol” , especially when you ddint change any thing else

/home/travis/build/openmrs/openmrs-core/api/src/main/java/org/openmrs/api/db/hibernate/search/LuceneQuery.java:[322,22] error: cannot find symbol

The compiler cant find the given class or method , its usually an incompatibility between the java version and the library

Thanks @mozzy are you trying to say I try a lower version?

1 Like

sure , you can try that

Am doing it right away!

1 Like

you could rather first find out which library is compatible with java 8

@mozzy My friend, I have been faithful upgrading from the latest release which is 8.4.1 to 5.0.0 version but all the versions the builds are failing. This is the pull request for upgrading

org.apache.lucene:lucene-queries … 4.10.4 -> 5.0.0
 

org.apache.lucene:lucene-queryparser … 4.10.4 -> 5.0.0

https://github.com/openmrs/openmrs-core/pull/3121. Feel free to advice me.

cc @mozzy, @dkayiwa, @ruhanga, @ibacher

I am trying to solve the Travis CI failure by upgrading indexes to Lucene 5. I have tried to read on how to upgrade Lucene 4.x indexes by using the IndexUpgrader tool, so that I remove the backward-codecs module from classpath but I have failed to access the Lucene 4.x indexes. Can someone help me on this please?

Instead of worrying about obtaining the right reference, etc., I think we just need to increment the number stored in OpenmrsConstants.SEARCH_INDEX_VERSION. This should trigger a one-time rebuild of the Lucene search indexes when the application is upgraded.

More usefully, let’s look at what’s failing here. So, in line 2797 of the log file we see the actual exception at the heart of this: java.lang.NoSuchFieldError: LUCENE_3_1. Looking at the exception log, we can see this happens at org.apache.lucene.analysis.util.CharacterUtils.getInstance(CharacterUtils.java:50), so line 50 of the relevant source code for the Lucene CharacterUtils class.

The CharacterUtils class itself is in turn contained in the lucene-analyzers-common dependency. Loading the source code for the version included in OpenMRS Core, we see that this line looks like this:

@Deprecated
public static CharacterUtils getInstance(Version matchVersion) {
    return matchVersion.onOrAfter(Version.LUCENE_3_1) ? JAVA_5 : JAVA_4;  // <- this is line 50
}

The Version class referenced here is org.apache.lucene.util.Version and defined in the lucene-core dependency. And, looking at the source code for the version included in OpenMRS, there indeed is no Version.LUCENE_3_1 field.

So how did that happen?

Well, it turns out the lucene-analyzers-common dependency is not defined in the OpenMRS POM directly, but rather defined by the hibernate-search dependency, which in turn depends on lucene-analyzers-common version 4.10.4. So we’ve got dependencies from two different versions of Lucene hanging around on the classpath.

One way to handle this would be to add an entry for lucene-analyzers-common to the OpenMRS POM. But that wouldn’t solve the issue of the HibernateSearch version depending on Lucene version 4. So what we really need to do is to upgrade HibernateSearch to a version that is compatible with Lucene 5.

Here, we can use https://mvnrepository.com to do a little sleuthing. lucene-analyzers-common is used by 600 artifacts, among which is hibernate-search-engine. We’re currently using version 5.3.0.Final which depends on Lucene 4.10.4. 5.4.0.Final also depends on Lucene 4.10.4, so to get to a version of HibernateSearch that is compatible with Lucene 5.x, we need to migrate to the 5.5.x series. I would think 5.5.8.Final makes the most sense. But note that this depends on Lucene 5.3.1. So try bumping the HibernateSearch version and the Lucene version to 5.5.8.Final and 5.3.1 respectively.

When you try to build that locally, there are likely to be some issues related to the OpenMRS code that uses HibernateSearch and Lucene. See if you can get anywhere with those.

3 Likes

Thanks so much @ibacher for great workaround in this thread, hey @jwnasambu are you still on this, i would like to volunteer following @ibacher comments if possible really

My friend, am still working on it that is why I asked for help. I will only surrender when am done.

:grinning: dont surrender my friend take your time, @ibacher information will push you somewhere i know

I have tried bumping Hibernate Search version to 5.5.8.Final and the Lucene version to 5.3.1and this is the error log https://pastebin.com/mj99K9f5. Building a core needs patience of a saint. It has been taking me at least 7hrs. Is there away of reducing on time spend running the core after making changes?

Hmmmm… 7 hours is excessive. Some thoughts on what you can do:

  1. Continue using Travis and just push the changes you’re testing to a PR (it might be better if you used a draft PR when doing this)

  2. Instead of mvn clean install run mvn -pl :api -am clean install. This will only build the api module, which is still quite heavy, but might build quicker in your environment.

Looking at the build log, I recommend option 1. Could you zip up the files in: C:\Users\JULIE\OpenMRS-Tickets\openmrs-core\api\target\surefire-reports and attach that file here?

The Travis CI is still falling as per the PR https://github.com/openmrs/openmrs-core/pull/3121. I tried to zip and share the file surefire-reports but I failed because of the unauthorized format of the content.

Please use pastebin for your log.

You are right. I tried to use paste bin but it couldn’t copy the content @ibacher requested me to zip up the files from: C:\Users\JULIE\OpenMRS-Tickets\openmrs-core\api\target\surefire-reports that is why the mess.

@dkayiwa and @ibacher how do I go about this issue. I have run out of options but I desire to complete this ticket.