CI within OpenMRS - Bamboo vs Travis vs GitHub Actions vs Azure Pipelines

I’m hearing from Twitter that they are currently not granting any credits right now as management is deciding a few things. Hopefully they will approve us in January, but I wouldn’t expect this month.

Yeah, that’s the cache. It’s mostly us being mean to ourselves. Docker builds are available, but I never used them. Also, there’s monitoring when we are running out of disk, but I don’t have others actively looking.

But the main reason we went with travis and bamboo is because travis can be ran in forks, something I don’t think Bamboo is capable of. Moving our releases and all the miscellaneous builds will be a little bit of an adventure. I’m not sure that’s actually worth doing at all. But there are lot of builds that can be easily moved without too much harm or work.

I’m fairly sure we can get that from agents usage in Bamboo, but probably require a database query or something like a plugin.

I attempted to add more, but it’s hard to justify it now, really.

Technically, the docker builds are supposed to be there to address that, but there was never really someone driving it (and I didn’t want to complicate the builds more than people would be willing to). I also can always add nvm or similar tools, which is usually good enough. That said, maybe moving those builds to cloud CIs might be just easier.


I think there a lot of builds that would benefit from moving from Bamboo to a cloud CI, but we just have to be really careful transporting their releases. I believe we will still want to keep bamboo for the time being, and we will learn a few things that weren’t easy to do in a cloud CI.

2 Likes

We also never started using bamboo’s version of CI as code, same way.

Personally, I’ve got no desire to move stuff that’s working on Bamboo somewhere else. If nothing else, it’s nice to have one place to check build status rather than having to look at 40 different repositories (that’s the one thing we lose by moving away from Travis… there’s a dashboard for the whole organization).

Yeah, the email I got from Travis is identical to the emails I’ve seen on Twitter (“We are awaiting further directives from management regarding the soon-to-be-released OSS/Education plan”). I’m not really holding my breath…

Sounds like it would solve most of the issues I raised :smile:, but that does sound like a big effort to get going.

1 Like

I’ve been making some progress with this. For those who are interested, I’ve started this conversion for several OpenMRS repos. I’m basically setting them up with this, plus or minus a few variations. The script I’m using to do this is this (with some manual variations). I’ve been avoiding any repositories that have a setup beyond just some relatively straight-forward Maven commands, as those will require more effort and, in all probability, generating new secrets to migrate them. The repositories that have been converted so far can generally be found through this GitHub query.

4 Likes

Thank you @ibacher for your work on this! This has been in the back of my mind when I heard of GitHub actions since I have seen build times increasing for our PR builds on Travis. Also since GitHub actions are so close to the code/access to GitHub APIs (GitHub script). Plus when we had some tests only failing for devs on Windows it did not take me long to run them on PRs on Windows using GitHub actions.

So please let me know if I can support you in that effort :grin:

1 Like

Thanks @ibacher for this move. Any update on this and please let me know where I can support you in this effort

1 Like

is this per project/user or what! this seems a limit still

As I noted above, the limit only applies to private repos. At least for now, GitHub actions is free for public repos. In any case, it’s a per-repo limit.

I’ve converted quite a few repos over to using GitHub actions already. You can find the list of unconverted repos through this GitHub search and we should probably target repos based on this listing.

I generally only converted repos that used a very simple Travis file, generally the default Java build, though I can see a couple I missed. The remaining simple builds can be converted using this script. For build that aren’t simple, they’ll probably need more care to understand what the build is doing and how to replicate it.

3 Likes

I have also written one that compiles uitestframework, distro and runs qaframework e2e tests at:

A travis equivalent for this is available at: openmrs-contrib-qaframework/.travis.yml at master · openmrs/openmrs-contrib-qaframework · GitHub

3 Likes

@k.joseph does this mean that QAFramework PRs will now auto-run via GitHub Actions rather than Travis?

1 Like

We could keep both running until travis finally closes. but i think you are raising a good point, we can disable travis

1 Like

Upcoming call to review remaining unconverted repos

FYI to everyone following this thread… in 1 week, we’re having a get-together on next week’s Monday design time to go through the unconverted repos and double-check for critical ones that need to be moved from Travis to GitHub Actions. (Per discussion on the TAC last week.)

When: Mon March 29, at 8pm EAT / 7pm CET / 5pm UTC / 1pm EDT / 10am PDT

Where: Launch Meeting - Zoom

GCal link here

2 Likes

That’s been the strategy I’ve taken with core… it still builds on both GitHub Actions and Travis. That way, whenever Travis pulls the plug, it should be painless enough to remove.

1 Like

BTW @ibacher is there any way to speed up the builds in github actions? has anyone noticed increased build times with the shift or perhaps this is mainly affecting our ui tests?

1 Like

To be honest, I haven’t really been paying much attention to that kind of thing. My initial impetus for moving off Travis was to be able to support builds on Windows and after that it’s mostly been about realising that Travis was coming to its useful EOL, but I’m sure there are things that can be optimised further.

3 Likes

Well, it looks like the party is over and GitHub Actions are no longer unlimited and free. :frowning_face: image

From our experience with Travis, I’m guessing our community can expend 2000 minutes of GitHub Actions in a few days (or less), right? So, now, after all our efforts to migrate to them, GitHub actions will begin failing to execute for us. What options do we have? With some talented devops resources to assist @cintiadr, maybe we could harden CI (e.g., either to reliably clean up builds on our CI agents or to build and tear down agent VMs on demand)?

/cc @ibacher @mseaton

1 Like

I don’t think anything has materially changed w/r/t GitHub Actions. That limit has been there since we started this thread and we certainly haven’t had builds stopped due to minutes.

Bamboo is a good option and maybe we should migrate the deployment-specific stuff there to keep our use of GitHub Actions confined to builds on PRs (which might be migratable to Bamboo, but isn’t supported out of the box)).

@burke my interpretation of this is that the 2000 actions minutes / month only applies to private repositories, and that public repositories are unlimited and free. “GitHub Actions usage is free for both public repositories and self-hosted runners.”

Looking at our openmrs organization account settings, and clicking on the “Billing & plans” tab, it shows “Usage this month” at 0 minutes, and only appears to be looking at Private repos.

I don’t see anything at either the organization level or under any of our repository levels under either Settings or Actions (eg. for openmrs-core) that shows any metrics as to how many minutes that repository has used in the current month, or any way to monitor this. I would expect to see this if it was something that was limited or throttled.

Let’s do a little more checking and see what we can uncover before we move away from actions.

1 Like

Ok. Whew. :relieved:

We got notified from GitHub that we hadn’t paid our bill. It appears GitHub is trying to charge us $757 USD. There was an option to downgrade to a free account (which I did), but I think somewhere along the line we had accidentally agreed to an upgraded account. I don’t know if the bill was per user per month (our usage report only shows about $240 USD since 9 Dec 2020). In any case, we’ll get that sorted out.

What a relief that the GitHub Actions quota doesn’t apply to public repositories. That’s not clear in GitHub’s billing screens.