I believe our module framework was created with the first release of OpenMRS over 20 years ago and not much changed over the years… It’s been serving us well over the years and contributed greatly to OpenMRS success! It hasn’t changed not because it’s so flawless, but it’s the foundation of our module ecosystem that is used by so many modules, which makes it hard to introduce changes without breaking modules.
The key issues that I encountered over the years:
ModuleClassLoader leak classes into OpenMRS core and other modules leading to linkage errors when e.g. module includes a jar dependency that is already in core. You may try to mark jars as provided in your module, but if dependencies change in a new version of OpenMRS core you may still run into issues and need to release module for the specific version of core. The separation of module class loaders didn’t really work…
Hot reloading of modules at runtime during development isn’t working reliably leading to classloading issues or server going down and is painfully slow, if you have a lot of modules. These days it’s probably faster and safer to restart the server instead of trying to hot-reload a module when developing.
Loading modules slows down all server startups as it requires additional Spring context refresh.
It’s hard to develop and automatically test modules supporting different OpenMRS versions. Effectively many modules are tested with the version of OpenMRS that they were originally created for, with just a handful of tests for a part of functionality that was developed to support a new version of core.
It’s really hard to run module tests in the presence of other modules, thus issues with incompatible modules are usually discovered at runtime.
I believe there is a way to significantly improve our module framework with compile time module dependency resolution and producing a distro jar/war file with all modules and its dependencies included. We could have a single build for a distribution that runs tests from all modules against the very specific version of OpenMRS core and the server loading up modules with the same classloader as core. It would disable hot reloading of modules, but we could enable Spring devtools with partial classloader re-load for application classes, which is a tested Spring solution for rapid development. There would still be a possibility to take a module jar and put it inside the application directory to be loaded without using a distro build, but it would only be recommended in simple cases where you don’t expect any dependency issues.
It’s a big change that needs to be implemented without breaking existing modules. Initially, I’d add it in a way that works in parallel with existing module framework and use it for any new module developed. In the next phase I’d explore a possibility to support old modules in the same way.
The change does entail a significant effort in adjusting build processes and tools and obviously has a learning curve. Is there any interest in the community for this kind of undertaking? Or did we all learn to live with the current state of things and there’s not much pressure to change?
So, since we have platform 3.0 currently in development, this does feel like an opportune time to think about precisely this sort of change. I’m all in favour of this, especially as I think it’s very likely to help out with two stubborn issues we’ve never really had solutions for, namely, the issue you’ve outlined in 1 (TRUNK-5344, which has been stubbornly unresolveable) and potentially the perennial issue of OpenMRS “hanging” during the context refresh after all modules are loaded—or if not solving this, at least giving us better tools for exploring why that is happening (I have written one or two nasty monkey patches to work around issues from this), so I’m all in favour of this.
It would be nice if we could continue to support existing modules, but I don’t think we need to make that a hard requirement. If we need to make adaptations for a new module system, as long as those are well-documented, I think it should be ok.
Since platform 3.0 already has breaking changes for most of the modules, i also agree that we do not have to make it a hard requirement to support existing modules.
What i would personally like most about the module framework rewrite is, improving the openmrs startup time. @raff do you have an estimate of the percentage improvement in startup time that this could deliver?
@raff as with all tradeoffs in software, apart from disabling hot reloading of modules, is there anything else that we would miss in the old module framework, as we to transition to the new one?
@dkayiwa, we won’t have separate classloaders for modules so they will be able to see classes from other modules at runtime. The separation isn’t working reliably anyway in the current framework as classes leak through the core classloader as soon as they are used in Spring context, which is only using the core classloader.
I cannot think of any other tradeoff.
I don’t have any estimate on how much faster the server would start up. I think the biggest benefit would be to have a reliable dependency resolution, fixing classloader issues and enabling a way to move to Spring Boot and use of Spring devtools.
I think this is probably the biggest gain. It seems like we’ve been leaving a lot on the table in terms of DevX for Java development that would be improved if we could take advantage of some of the things Spring Boot enables.
One more technical question I have: currently the two main functions of the Module ClassLoader are:
Unpacking any dependencies (the lib folder) into a per-module cache
“Hiding” conditional resources as appropriate (e.g., how we support multiple API versions in a single module).
Any preliminary thoughts on how those would be handled?
@mogoodrich@raff - Since I mostly work on the implementation side, I don’t know specifically if this would be an issue but would moving to a single classloader create potential issues with namespace collisions? Apologies if the current design already covers for this, I’m still a novice on the code side, just thinking about potential runtime issues.
No custom unpacking. Dependencies would be resolved during distro build and put with all core dependencies in a standard fat jar /war location.
We would rely entirely on conditional beans and the OpenmrsProfile annotation. It’s the standard SpringBoot approach when adding features to auto-register beans given conditions are met.