Hi @dkayiwa@ibacher, I would like to suggest a change and I’m curious to hear your thoughts about it.
As of its branch 2.x we would like Reference Demo Data to depend on the Ref App 3.x config (as test dependency) ⇒ This would mean that this config is to be made available as a deployed artifact that can be depended upon, which is not the case right now (but it used to be the case, AFAIK).
The reason being that, at that point, Spring tests of the Reference Demo Data module will
Invoke Iniz’s API to load the 3.x config (this would actually validate it btw).
Perform tests that ensure that the demo data was generated as expected in the context of that config.
I wonder if you think this would work and make sense as a process? Thoughts?
@ibacher, the current Ref App 3.x is configured to package all artifacts in a single zip. I would as well, prefer having the configs as a separate/child artifact making it light weight and easily consumable by other projects like the referencedemodata for instance.
Also better having the configuration as an entirely separate artifact because that’d ensure stable releases/versions before being consumed by the reference application (I also understand that this could have implications to the way the dependent docker images are built and deployed but this shouldn’t be a ‘hard nut to crack’, a potential subject for more thoughts).
So, anything included in the referenceapplication-distro artifact that’s anything other than the configuration is only there by mistake. The only reason we even publish that artefact is to make the configuration available for Ozone, so please feel free to put whatever exclusions need to be in the assembly.xml to make it match that…
Thanks for clarifying @ibacher, Ozone not only needs the configs but the module/war artefacts of the Ref App as well. Referencedemodata is also being added to the basket but for configs solely in tests. Not sure if this would actually impact/help other implementations depending on the Ref App 3.x…
cc @PIH, @UWash, @dev3, @dev4, @dev5
I don’t think this is accurate. A “distribution” artifact is intended to be an entire distribution - the war and all modules, owas, and other artifacts needed to run an entire distribution. For convenience, it seems, we have been including the configuration directly in the distribution project. There is nothing wrong with this, but there is nothing in place to package this configuration independently of the entire distribution zip.
At PIH, we keep our configuration explicitly separate from our distribution for a variety of reasons:
CI can build and deploy our configuration whenever it changes without necessarily building and deploying an entire new distribution each time
We use the same distribution “binary” (war, modules, owas, etc) with different sets of configurations for different countries
Now, one does not necessarily need to keep configuration in a separate repository for this to work. One could build and deploy multiple maven artifacts in the same project - using a maven submodule might be easiest. But I personally prefer a separate github repository - it’s unambiguous.
So… if I’m understanding what @mksd is asking for, he is suggesting that we move the configuration that is currently in the refapp distribution into a new project, maybe “openmrs-config-referenceapplication”. Then, the openmrs-distro-referenceapplication would include a specific version of the openmrs-config-referenceapplication as a dependency, and the build process would be responsible for copying in that dependency, unpacking it, and including the contents in the final distribution zip.
Currently, we aren’t publishing a real distribution artefact for the 3.x RefApp in that sense. The RefApp distro artefacts are actually called referenceapplication-package (this is used by the SDK). Previously, we had a setup that was publishing something more similar to a distribution artefact, but this was hard to do without rendering the build unstable.
While I understand that we want to promote the use of published Docker images (and we certainly want to leverage this fully within Ozone and moving forward), having a way to access deployed artefacts remains important and useful.
As a matter of fact all our implementations as well are constructed as you described it for those at PIH: the configuration is a separate Maven project, with its own versioning, etc. May or may not be overkill (we had long discussions about this @mksrom and I), but anyway that’s been our pattern, historically at least.
Just to clarify a bit, I don’t mean to say that everyone should be using the RefApp Docker images or anything, just that the priority around the 3.x RefApp has been in getting us a stable environment that can be reliably deployed and that gives the people working on the 3.x frontend a stable backend with well-defined metadata, rather than on developing something that reflects how deployments should be setup.
The more I’m thinking about this, though, the less certain I am about how the demo data module figures into this… If we’re going to test the deployment of the metadata (and we should), that should presumably just be part of the configuration packages themselves, maybe using something like the Initializer validator.
I agree, and indeed we should have a validation job for the config as a step of the distro build, that was always the use case for the Validator.
Then, Reference Demo Data’s build would also run tests that check that its demo data can be generated against the Ref App 3.x config (specifically this is our focus here). However this entails loading the config again within Reference Demo Data’s Spring tests, which is a repetition to what the Validator would do during the distro build. Fine IMO, but curious to hear others comments.