Can even replace Artifactory!
Hi @mseaton,
Iāve been extremely busy withā¦ the whole thing going on, and I just could come here now.
Hm. That sounds like something is wrong on the nexus setup. Can you please try āJFrogā (with the slash at the end) to see if you have the same problem? Iām curious about your setup, because itās supposed to work.
Travis is a cloud CI. It works extremely well for independent artefacts (one pipeline that doesnāt interact with others). I particularly dislike having to cut a different branch to create a release (as it breaks the whole definition of a pipeline), but it does have its purposes.
Itās definitively not going to happen any time soon (at very least), as we have snapshots, that triggers snapshots, that triggers snapshots 
We donāt have independent pipelines, what can lead to certain complications.
We also have the deployment views, that helps us a lot during releases and after, as we have static environments.
Itās probably not worth moving at all for now.
What doesnāt block you at all in moving your own builds to travis or any other cloud CI!
Bullshit. Every CI can run arbitrary commands, thatās what they are, remote code executors. Each CI tool has their own strengths. Some have better triggers in place depending on how you like to run your deployments.
Hey @cintiadr,
Thanks so much for the detailed response. A few quick follow-upsā¦
I made this commit, which resulted in failed deploy here due to 401 Unauthorized.
I changed back to the URL that uses jfrog.io, and my deployment passed.
Well, now you can see above Iāve moved from Travis to Github Actions for my current experimentation
At this point, my goal is simply to set up a CI structure that will work best for a country-level distribution project, with minimal infrastructure that is owned and administered by a single organization and which is transparent and accessible for all to adopt. OpenMRS could be that if we want OpenMRS Bamboo to be used for that purpose, but it is not clear thatās what OpenMRS Bamboo is intended for, and also this seems like an opportunity to experiment with these CI tools that are more intrinsically linked to the source code.
As for any broad-scale OpenMRS migration to Travis, Iām not pushing for thatā¦just trying to take the temperature of the room, given nearly 4 years has passed since @raff wrote up this post and he and @ssmusoke provided working examples for several modules. As you sayā¦
ā¦that is probably true. As you detail in your blog, I think another solution would need to have clear, compelling benefits that are greater than the cost of migration, which (given our resources) is probably close to impossible at this point. But if some groups start experimenting with new approaches and find use cases for which they work quite well and are compelling, then this can be informative to the community and might provide opportunities for us to migrate opportunistically where the situation makes sense.
My guess is that there are likely at least some modules in OpenMRS Bamboo that are not part of any kind of dependency building pipeline, and only exist there to build and test upon each commit, and (maybe) to deploy snapshots and releases to Artifactory. These might be good candidates to pull out of Bamboo and into a Travis-based process, and see how that impacts things.
Speaking of Travis, Iām going to show my ignorance here, but can you clarify what you mean by your comments here?
Thanks again for your thoughts, and no rush on responding, we are all busy!
Best, Mike
1 Like
Thanks so much for the detailed answer. Itās a lot easier to debug
The URL you are using goes to /artifactory/. The URL the mavenrepo is redirecting goes to /openmrs/
Transfer failed for https://openmrs.jfrog.io/openmrs/modules-pih-snapshots/org/openmrs/module/rwandaemr/2.0.2-SNAPSHOT/rwandaemr-2.0.2-20200413.022553-2.pom 401 Unauthorized -> [Help 1]
What does it mean? I donāt know. Itās not I really know what Iām doing here, but it could be related to the permissions I gave to your CI Artifactory user. Eventually I will make sense of it, but I donāt think itās a bad idea to keep the JFrog URL - at least for the time being.
I raised ITSM-4271 to investigate that in the future. Downside of having the URL to Jfrog domain is that we are āhardcodedā on it during maven releases, so itās some sort of vendor lock-in (for some use cases).
The definition of a pipeline is: you checkout the code, you build an artefact (a java .war file, a docker image) and you keep promoting the artefact through your pipeline. You never rebuild the artefact. The further you go up the pipeline, the more confidence you have you can deploy it safely to production.
Each pipeline will release to production independently of others.
The reason why you never rebuild your artefact is that you are completely isolated of environmental changes (build environment; upgrades to dependencies or platform). We all know how rebuilding an artefact (from the same commit) in a different day can lead to different results.
So how you do in travis? You commit to master, you run unit tests, build your artefact, run integration tests, run manual tests, UAT tests, whatever. When you are happy with the result, you decide to deploy to productionā¦
ā¦ you update branch āproductionā to have that commit. That will, guess what, create ANOTHER artefact. 
So you have to redo every single of your tests, because you are dealing with a different artefact all together.
Itās small, but itās a risk you are taking.
You donāt have to follow that workflow. I.e., you can add steps to Travis to build and deploy the generated artefacts to where you want (e.g., I have some Travis builds that create a docker image, push it to Docker Hub and then tag the āreleaseā on GitHub, though that tag only means āthis was the source used to build release xā).
That said, I agree that Travis isnāt the best tool to use when you need to coordinate multiple artefacts and we would need to find a compelling reason to move off Bamboo (having all builds on a single dashboard with a single reporting infrastructure is nice).
Thanks @cintiadr I think I understand what you mean better now. I donāt knowā¦regardless of whether we use Bamboo, Travis, Github, or another CI server, I feel like we can implement pipelines. Iām not yet seeing where one limits us more than others. I do think Bamboo will make certain pipelines easier to manage.
Unfortunately, with our use of Maven snapshots, our release processes will always lead to at least one new commit pre-release (to change the pom from SNAPSHOT to non-SNAPSHOT, update the SCM tag, etc), and then will execute a completely separate build and test process from the newly created tag in github in order to initiate the release. So at least for us in OpenMRS we pretty much never release the artifact that has been tested against as you describe. But this seems to me to be a symptom of our usage of Maven Snapshots more so than a particular limitation or quality of our CI tool.
Not to expand the scope of this thread, but there are a number of things that one uncovers that can be improved in our CI process, once looking under the hood. One thing I have noticed is that many of our artifacts build against stable dependencies, but yet we have dependent builds in Bamboo (pipelines) that run through and execute whenever dependent projects are built. For example, in Bamboo the HtmlFormEntry plan will kick off builds of HtmlFormEntry19Ext, HtmlFormEntryUI, ReferenceApplication Module, and ReferenceApplication distribution. Yet, if you look at the actual referenceapplication distribution code, you will see that it depends on a stable version of htmlformentry - 3.10.0 currently. So really, there is no reason it needs to rebuild just because a new version of htmlformentry 3.17.4-SNAPSHOT has been deployed. This doesnāt really lead to problems per se, though it does create a large number of completely unnecessary builds, and reinitiation of downstream processes that might be watching for new versions of the referenceapplication distribution, even though this has not changed at all.
It would seem the intent of the above was likely to ensure that when any downstream dependencies are updated to require a snapshot, that the CI pipeline would ensure it is tested properly. But there is likely a better way to accomplish this. What we should really be looking at is building/adding a āMaven Dependency Triggerā for each of our modules, rather than our current process of having parent projects blindly trigger child project builds.
@dkayiwa and @raff Iām interested in your take on the above.
This is one thing I have on my radar to enable in the Github CI jobs that Iāve started to spike on. Iām not sure if Bamboo has a trigger option like this available out of the box - Iāve personally only seen and used the repository polling trigger. Iām wondering if this is a new goal we might add to our SDK Maven Plugin - one that can run on a polling schedule, identify if any dependencies have changed (snapshots or otherwise) since the last build, and if so, kick off a new build.
Best, Mike
My understanding is that the reference application distribution is supposed to depend on snapshot versions of all modules, until the time of releasing. After releasing, like version 2.10.0 which happened a week ago, we are supposed to have again bumped all modules to the next snapshot versions. I guess that is why our CI process was set up that way.
This does strike me as a strange release process, precisely because of the issue @cintiadr identified, i.e., we move one set of artefacts into UAT, but then cut a release withā¦ a completely new set of dependencies
with whatever changes have gone into master on each module added to the release. This can have bad consequences (as with including the COVID-19 concepts in the 2.10 RefApp).
My impression is that the more normal process across the industry would be to do a āfeature freezeā at some point prior to a release and only build against module versions ready at the time of the āfeature freezeā and only updating those modules to fix bugs identified in the release testing process.
The problem is, that this workflow would require giving up OpenMRSās allergy to āforkingā (that is, creating version branches) and would work best with a more rigorous application of SemVer (i.e., we should be more willing to bump module major versions, default to bumping minor versions and use patch versions to trackā¦ wellā¦ bug fix releases).
Hi @ian - it isnāt as bad as all that. What you are describing does not happen to my knowledge. The versions that are tested are (or should be) the versions deployed. We donāt ājust take the latest from masterā unless that latest from master is what has been tested against (eg. the latest snapshot). For core, there are release branches that are cut for all new minor releases (1.9.x, 1.10.x, etc) which typically represents a pre-release alpha. We donāt do the same for maintenance branches, but havenāt had problems with this to my knowledge. For modules it is certainly possible for us to run into trouble if we arenāt careful.
1 Like
@mseaton Youāre right I was being a bit hyperbolic. What I meant to say is that if RefApp 2.11 was being developed with, say, HFE 3.11.0-SNAPSHOT the more usual thing would be to release it with 3.11.x, where x is a patch version released to fix whatever bugs were found while testing 2.11.0-SNAPSHOT, even if HFE has moved on to 3.12.0 (because new features were added during the release cycle). I do realise this hasnāt traditionally caused issues with the RefApp once itās released, Iām just wondering if itās the most efficient way of ensuring timely releases of the RefApp.
Of course, this may be looking at the RefApp the wrong way, i.e., that itās some sort of product instead of a demo of the current capabilities of OpenMRS. The current processes work very well in that role.