Update CI builds to build with Platform 2.0

2.4 Reference Application has been released with 1.11.6 platform and Platform 2.0 released has also been released!

We plan on having all the platform 2.0 packaged with Reference Application 2.5 (September 2016). The modules of Reference Application 2.4 have already been tested on platform 2.0.

Going forward, we need the CI and automated builds for Reference Application to be built with Platform 2.0 instead of Platform 1.11/1.12.

Can @cintiadr or anyone from the QA team take a look at performing this update ?

@maurya So why do CI builds for Ref App 2.5 not also be built for 1.11.x and 1.12.x as those are the versions being used in the field by implementors for the foreseeable future.

This is one of the reservations that I raised for in a separate thread on having Ref App 2.5 be focused on Platform 2.0 cc @dkayiwa

@ssmusoke, as far as I know of Continuous Integration, they are builds to test the most current software for any issues. This helps identifying issues for the commits being done regularly before releasing it.

I do agree that most implementations are using 1.11.x and 1.12.x. But once a version is released it has had many continuous Integration builds done and the version is considered released for production.

Now, for Ref App 2.5 Platform 2.0 is being packaged with it, for it to be tested, it would need the CI configured to identify issues before releasing it.

As @dkayiwa mentioned 3 latest platform releases are supported by OpenMRS, so, if you face any issues for the modules any implementation uses, they can report the issues and they will be resolved. That said, unless a module developer uses new Java 8/Hibernate 4.x/Spring 4.x features/functionality in their modules they should work on 1.11.x/1.12.x builds or even earlier builds without issues.

Having builds being performed for 1.11.x and 1.12.x as well as 2.x will need additional build plans as well as servers with very high probability of facing the repetitive issues that will come up in 2.x builds. This will also increase undue burden on the developers monitoring them.

@maurya My understanding is that Ref App 2.5 will also run on 1.11.x and 1.12.x so if there are no CI builds for it, what is the confidence in using it on those platforms if the implementors have to find the issues in the field?

The purpose of the CI platform is to reduce the number of issues, especially regression ones, in the software as the cost of finding these is x10.

Since the core developers are too busy to monitor the older branches, is it possible to help implementors (like in our case) setup our own hosted CI platforms to monitor the branches of interest cc @burke @darius @paul

@ssmusoke you can volunteer to setup CI plans for the reference application running on 1.11.x and 1.12.x It shouldn’t be hard. I can mentor you to do it if you have the time. :smile:

1 Like

Whoever takes this up, a good starting point would be https://ci.openmrs.org/browse/REFAPP-DOM which runs snapshot versions of the reference application modules and platform 2.0.0-SNAPSHOT which has just been released as 2.0.0 It deploys on http://uat01.openmrs.org/

It is currently run manually instead of automatically after each commit to a module or platform, and is based on https://github.com/openmrs/openmrs-distro-referenceapplication/tree/platform-2.0-support

@dkayiwa on implementor hosted or openmrs infrastructure, because according to this thread those are being done away with to be replaced by 2.0 platform builds

My understanding is that replacing the current one to use platform 2.0 does not prevent you from creating a new one based on platform 1.12.x or anything else. Ask for a CI account and you do the needful.

@dkayiwa Thanks for the clarification, I happily volunteer to setup the CI plans for 1.11.x and Ref App 2.5. I am will read through the documentation provided above and get back to you tomorrow. Thanks

1 Like

You are as awesome as always! :smile:

Sorry for being late to comment on this thread! We should clarify what outcomes we are aiming to achieve here, and for how long we want to commit to them.

Historically, we have CI build and run automated tests on each commit to the reference application distro. As Maurya points out, we are moving the refapp distro to be based on the latest 2.x platform release. In theory the only thing we have to do here is to change one value in pom.xml of the reference application distro to indicate this new version of openmrs-core, and the existing CI plans will automatically be using platform 2.0. @maurya, you can do this yourself. (In practice things might break if the int/qa servers don’t have Java 8. In this case the infra team will need to help.)

Further @ssmusoke is asking that OpenMRS maintains CI plans that run the reference application against platform 1.x. We never did this before (e.g. when switching the refapp from 1.9.x to 1.10.x to 1.11.x), but given the larger changes from 1.x to 2.x I sort of agree with doing it, but let’s not forget that is a new thing to impose on the OpenMRS community.

Practically, I guess what are talking about is to take the current reference application modules + an older version of openmrs-core, and run the current refapp functional test suite. But what happens if the tests break? Are we saying that reference application package must be able to run against platform 2.x and 1.11 indefinitely? I don’t agree with making that commitment.

Right now, groups that build other distros from reference application components run their own CI servers, and have their own test suites to ensure that their configurations keep working even as the components evolve. E.g. PIH build its EMR on platform 1.10 with many refapp components, and Bahmni uses platform 1.12 with a few refapp components. And when something breaks for one of them they’ll raise that issue on OpenMRS channels and find a way to resolve it. Uganda should take the same approach: you are building your own distro, so that distro needs its own test suite, run in its own CI. And when someone else makes a change that breaks Uganda’s CI you should share this info and work out a solution with them.

As far as the individual components that make up the reference application go, these will continue to work on platform 1.x as long as feasible (and I’m practice that will be a long time), and if something like the coreapps module we to go to platform 2.0, that would be discussed at length.

So, if you want to set up an OpenMRS CI plan like Daniel suggests, you can try this, but I’m uncomfortable committing the community to keeping this CI plan green.

@darius thank you for the detailed answer: My continous question has been and now based on your answer still is: How long will the commitment from OpenMRS be to keep this plan green for Ref App modules, after which Uganda shall be forced to move installations (over 1000) to Platform 2.0 which is a rip and replace upgrade especially because of Java 8 requirement and a few others (probably unknown at this time).

Also while we can find funding for infrastructure to run CI plans, we do not have in-house capability to set-it up.

Let me highlight a distinction. (1) The reference application is built out of individual modules (and soon, OWAs and JS libraries). (2) the reference application is a specific distro.

For (1) I don’t think we have an official position on which OpenMRS versions are supported by modules. We should write down some guidelines, for OpenMRS-supported modules. But I cannot think of a single time when an existing supported module dropped support for older-but-still-supported versions of the platform. In practice, modules keep supporting older platform versions as long as their contributors are willing to do so. :slight_smile:

For (2), each version of the reference application is built on one specific platform version, and it would require extra work, and change the philosophical focus, for OpenMRS to ensure that multiple configurations work, and not just one. (And I’m opposed to OpenMRS taking this on: when I say we shouldn’t commit to keeping a build green, I mean only this one.)

So, I think what you really want is about (1) i.e. modules to keep supporting platform 1.x, but the approach we have been discussing more about (2) i.e. the reference application itself still supporting 1.x.

@dkayiwa what do you think? What is the most practical way to achieve what Uganda’s real need is?

This is a good point. It would be great if someone could do a proof of concept of setting up an OpenMRS distro and running its functional tests on a hosted CI solution like Snap CI or Travis CI. Then a group like the Uganda one could copy the pattern, providing its own core and module versions, and writing its own tests, without having to actually set up and manage a server.

@darius the reference application modules that the Ugandan distro uses, run on both platform 1.x and 2.x Even after making the reference application CI point to platform 2.0, most of these modules are being run in custom distros like PIH, and others, who would report if we made a commit that breaks any of these modules on 1.x

So @ssmusoke you do not have to move to Java 8 if you are not ready to. Just keep using the 1.x platform while taking advantage of the most recent changes in the reference application modules. Even if you do not have a custom CI, you can always run the tests locally every time you try to switch to a newer version of a reference application module in your distro.

I have updated the distro pom to build against 2.0.0. I have also updated our build plan to build with Java 8. The build is being successful, but there are a number of basic UI tests failing due to timeouts. This can be viewed in this build - https://ci.openmrs.org/browse/REFAPP-OMODDISTRO-4806

Can you provide any direction on where the issue might be? (@dkayiwa, @darius? )

@maurya, I looked at the tomcat log artifact of the build, and I see this snippet:

Caused by: java.lang.UnsupportedClassVersionError: org/openmrs/web/Listener : Unsupported major.minor version 52.0 (unable to load class org.openmrs.web.Listener)

I guess this means that the server is not running Java 8.

Also if you look at int-refapp.openmrs.org you’ll see it’s not running correctly. I don’t know how that machine was built, but we need to update it to run Java 8.

@maurya the failure of those tests is caused by a timeout because int-refapp.openmrs.org is not running. Just as Darius suggested, can you ensure that int-refapp.openmrs.org is up again?

I do not have the access to server, requested access, will update it as soon as I receive access to the server.

@darius @burke question - I am looking to setup a CI plan that runs the Ref App 2.5 against the 1.11.x branch however there is need to deploy to a server to run the automated tests, please can you advise whether a server can be made available even if its once a week to run this plan. If not can the plan be executed on a server outside the OpenMRS Inc infrastructure using Bamboo?

@ssmusoke, my feeling is that we shouldn’t really be doing this on OpenMRS servers; it’s not a service OpenMRS can or should sustainably provide to every distro. (And we don’t have any available servers now.)

What I recommend is that you pair with @dkayiwa (or some other volunteer!) on doing a proof of concept of setting up a CI pipeline on a hosted service like SnapCI or Travis CI, and share how you did this so that others can replicate it.

Snap CI is a ThoughtWorks product, that is especially built for CD, e.g. you have pipelines that can automatically or manually deploy the same artifact to different environments. It’s a good (very modern) option if in the future you might go beyond CI, and want to deploy the same tested artifacts to staging and production machines.

Travis CI is very widely-used, and would also be good at what you’re aiming to do now. (I don’t think it has the same first-class support for CD as Snap does, but you don’t need that now, and you might not ever.)

Both of them have a free tier for open-source repos, and will run your build and tests in a fresh virtual machine. (You don’t get a free persistent server, though.)

[Edit] PS- I’m suggesting this approach because I presume you don’t want to set up and host bamboo yourself. And given the power you can get for free from Snap and Travis, there’s no reason you should.