Escape from dependency hell

Development
1

Hi everybody,

Last week I had a hard time with several dependency issues.

Problem

The hardest one appeared when I tried to integrate optaplanner into openmrs. Optaplanner is a large library with several transitive dependencies. In the end I found out that some classes weren’t loaded from ecj-3.7.2.jar, but from core-3.1.1.jar. The first resides in my modules lib folder the latter is an outdated library imported from maven jetty plugin.

I had another problem with jodatime and found out that the modules that are packaged with openmrs-distro-referenceapplication use three different versions of this library:

  • Htmlformentry: 2.1
  • emr and others: 2.2
  • reporting: 2.3

In my case the classloader sometimes decided to load DateTime from emrapi regardless if the library was present in my lib folder or not. I found the following ticket that describes this problem:

In a comment @bwolfe described the classloading process:

  1. Module asks OpenmrsClassLoader for class string X
  2. If OCL has loaded class X already, returns class X
  3. If OC has not loaded class X, look in ModuleClassLoader for requesting module (and its jars).
  4. If not found, look in ‘required modules’ of requesting module class

At the moment it is a bit like using global variables. You can’t be sure that the classes during testing are the same that are used during runtime. It depends on the loaded modules.

Suggested Solution

Before I continue I want to say that I am not an expert on class loading, but I wanted to discuss possible solutions with you.

From my point of view dependency issues can be divided into the following groups:

  1. make sure that related modules are compiled, tested and run with the same versions.
  2. make sure that module dependencies on other libraries are independent from each other

@ 1 This problem already broke the install process of the reference application - RA-331: The referencemetadata module was compiled and tested against idgen 2.7, but was distributed with idgen 2.8-SNAPSHOT. A message signature changed causing an exception at runtime, although all tests passed during compilation.

Suggestion: Create a common parent pom that defines all version numbers for each module. Let all modules derive from this file.

Benefit: Increased trust, that the application won’t break if I use the exact same module versions.

@ 2 In this case we have to distinguish between:

  • entities that are private to this module
  • and those that are shared between modules

I think the first one could be solved by using independent class loaders, that don’t load classes from other module libraries. Otherwise a single dependency in some other module can break the application.

The second one could be solved in a way that the library is added to the common parent pom. This means that modules are compatible with each other because they include the same version of this dependency into their private classpath.

Conclusion

The reference application already consists of several modules. This number will certainly rise in future. In order to ensure high quality and maintainability we have to discuss this issue.

I hope I could describe my concerns in an understandable way – otherwise I am happy to clarify!

I am looking forward to hear your opinions!

Cheers, Lukas

2 Likes
2

About #1, we started to look into having modules in the reference application inherit other module versions from the whole distro, rather than declaring them locally. I don’t think we ran into any problems with this, so we want to do this in more modules. But for now, I don’t think we implemented it in many places. The one example I found in a quick search is:

About #2, I do think it makes sense to explore making our module classloaders a bit more independent. E.g. a module should only be able to load a class from:

  • itself
  • a module it depends on (directly, or transitively)
  • OpenMRS core

It sounds like we currently also allow the scenario where a module gets a class from an independent module that it doesn’t depend on, because that happens to have loaded a similarly-named class first.

3 Likes
4

@darius thanks for your reply!

that’s great! We should go through the list of modules and see which one is still defining the versions locally (e.g. emrapi)

I would even go one step further and suggest that the classloader should only be allowed to load classes of a dependent module if it starts with org.openmrs.module.*

Otherwise dependency issues are still likely if two modules, that I depend on, use libraries in a different version. That would be the case for jodatime and the modules htmlformentry and emr. (see first post)

What do you think?

Is anyone familiar with frameworks like OSGi and could tell us how they are trying to solve this problem?

5

Note that we can’t actually update all modules to share these same dependency versions. E.g. the idgen module intentionally can run against older versions of OpenMRS (1.6.5), whereas some modules that “belong” to the reference application may only run on the latest.

Modules are not required to be underneath the org.openmrs.module package (though AFAIK all of them are).

6

I was afraid that it isn’t as simple as I thought :smile: Do you think we should discuss this topic in the developer forum?