Enhance OpenMRS SDK to support Dockerized Module-Based Distribution Builds

Development Quality Assurance
, ,
1

Hi all,

Following the great conversations in the threads below — which represent a significant step forward in achieving more consistent and modern software development and deployment practices — I’d like to propose an additional improvement to help fully realize the underlying goals:

:magnifying_glass_tilted_left: Why The above Efforts Matter

Both development and deployment workflows fundamentally require a correctly provisioned environment — typically involving compatible versions of Java, Maven, and system-level dependencies. Docker allows this environment to be defined and shared consistently, avoiding “it works on my machine” problems and making CI/CD pipelines more predictable at the same time improving developer experience.

The benefits can be summarised as, being able to:

  • Provide a standardized, reproducible runtime across teams and platforms.
  • Eliminate local/host system dependency issues.
  • Improve developer onboarding by reducing setup complexity.
  • Make it easier to test against OpenMRS distributions or module versions.

It is from the last point above that I’d like to discuss an area to improve on the OpenMRS SDK. The OpenMRS SDK already provides powerful features for managing development servers, creating distributions, and scaffolding modules.

During module development, being able to also build deployable module based distributions can be great addition to the above efforts. Here is where this would lead to — being able to generate a run-able distribution based on a module’s config.xml file which consequently involves resolving the transitively required modules. With this, we would eliminate the need to have extra distro-properties.xml or docker-compopse.yml files residing within modules.

:gear: Proposed Enhancements to the OpenMRS SDK

  1. Introduce a lightweight function to build a distribution based on a module’s config.xml file, into the OpenMRS SDK.

  2. Extend existing OpenMRS SDK commands to Support 1 above, such as:

     mvn openmrs-sdk:build-distro -Dmodule-based       # Build module based Dockerized distribution

The command can be run locally or executed within a mounted Docker container. This improvement is optional and opt-in for devs and, as usual, allows for execution in CI environments

Devs are allowed to define or override Docker Compose instance files as part of their project deployment setups, enabling custom local dev stacks.

Looking forward to your thoughts and feedback @ibacher, @dkayiwa, @wikumc, @raff, @mseaton, @dev5, @dev4, @dev3, and anyone else interested in improving our tooling

9 Likes
2

Sounds exciting. Cant wait to hear feedback.

2 Likes
3

Great Idea — resolving a deeply nested transitive dependency tree for the required modules is definitely one of the more complex pieces of this proposal. That said, it’s also a necessary step if we want to support truly self-contained module-based distributions.

Perhaps , we could leaverage the existing fucntionality where the SDK will generate either a Platform-based distribution or Reference-App based distribution depending on the module’s config.xml file and just plugin a few extra required modules not part of the distribuiton.

Definitely this is an area where thoughtful design and contributions would make a big impact.

5 Likes
4

Correct, @mozzy. You can think of this as enabling the dynamic generation of a distribution where the developer only needs to specify the module they’re working on.

One challenge, though, is ensuring the availability of necessary initialization metadata to successfully run the modules. I guess this shouldn’t pose a major hurdle in most cases, since many modules already include their default metadata via Liquibase migrations and/or config.xml definitions.

5

We generally discourage distributing module metadata via liquibase.

1 Like
6

Ahh, right. Thanks for the clarification @dkayiwa… :slight_smile:

7

It is a good idea and opens up a lot of possibilities for us to add features to the SDK based on specifications in a module’s config.xml file. I definitely see value in parsing out a list of module and core requirements for a given module, by walking the config.xml dependency tree recursively. It’s actually quite surprising that this has never been part of the SDK.

Before we dive it though, it would be good to clearly lay out what problem we are trying to solve so that we can best identify the specific features that we should add to the SDK. It might be that it is best to develop a few small, independent features that can be composed together, each of which is independently useful.

For example, outside of the CI goals listed here, another obvious use case would be If one deploys a module or a distribution into an existing SDK instance, the SDK should be able to check whether all of that module’s / distribution’s dependencies are present, and if they are not, should prompt the user to add what is needed, iterating over each missing dependency and presenting the user with a list of compatible versions of each that they could install.

1 Like
8

Indeed, @mseaton — the intention behind this feature is to complement and carry forward the efforts initiated by @corneliouzbett, specifically:

Absolutely — your suggestion to have the SDK automatically detect and resolve missing dependencies when deploying a module or distribution is spot-on.

From the perspective of implementers and testers, this would be really valuable. It would help assess the dependency safety of a module or distribution before runtime issues arise.

While this extends beyond the original motivation, it’s a highly complementary enhancement. It can easily be scoped as part of the broader effort to improve the SDK’s module/dependency handling — involving interactive prompts or automated resolution where feasible.

9

@ruhanga i see that you have started working on migrating from XStream to Jackson. Does that mean that this work, of being able to compile modules within docker, is complete?

10

This has already been achieved on our Bamboo CI, @dkayiwa. However I’m still working on the module-based Docker distribution with the SDK, albeit with a lower priority at the moment. That said, two things:

  1. The Jackson migration has taken precedence since it seems to be blocking some other upgrades.
  2. I’d also like to keep the discussions around the shift to Jackson moving forward.

With this in mind, I’ll make sure to get the module based distribution-deployment work completed as soon as possible as well.

11

@wikumc has finished Java upgrades for a number of modules but cannot have their snapshot versions run on Bamboo CI. May be you did that already and i am just not aware. If you already did so, can you share the instructions on what he should do with the rest of the modules that he is working on? This is of a higher priority than the XStream to Jackson upgrade work.

12

Correct @dkayiwa. I should have posted an updated list of those modules already configured in Bamboo to run with Java 21. This I have done on the migration thread.