Today, I had a productive call with @kdaud to clarify various aspects of the openmrs-contrib-qaframework, including its structure, failing tests, progress on 2.x E2E, pending work, and implementation issues.
Before the call, I familiarized myself with the QA Framework technical documentation and two helpful YouTube videos[1] [2]. During the call, Daud expertly explained the project’s directory structure and test configurations, including the transition from legacy to BDD tests.
Here are my key takeaways from the discussion:
The bamboo integration is not working properly and has been disabled due to resource consumption on the CI server, and @raff has been asked to look into it.
Critical functionalities of 2.x have been covered with E2E tests
Tests are failing randomly, but not due to outdated tests but rather issues with running them on Github actions
Going forward, I have identified the following action items:
Check the status of bamboo integration resolution with @raff
Resolve random test failures on GitHub - @piumal1999
@raff, could you please give us an update on the status of the bamboo integration resolution? I couldn’t access the plan as the bamboo dashboard seems to be broken.
If you have any questions or comments, please feel free to share them below. Thank you!
Bamboo was running out of disk space due to an issue with backups not getting processed as expected, @ibacher was able to get it running, and @cintiadr mentioned she’d take a look at the backup process issue when she gets a chance. So Bamboo (ci.openmrs.org) should be running again and hopefully not going down again.
Ultimately, I think the backup backlog was caused by the Jetstream networking errors on Jan 27th. Hopefully that’s a one-off. Realistically, if this was the only fallout from that, I think this shows the overall robustness of our infrastructure.
However, I discovered that there’s an issue with setting up the 3.x from scratch. Some module seems to be blocking startup procedure. I’m investigating that right now. I was able to reproduce the issue locally as well. I’ll update you once I find the fix.
Also, have discovered the plan CONTRIB-QA, where 2.x E2E tests were configured. It seems it is triggered when there’s a commit to the GitHub - openmrs/openmrs-contrib-qaframework repo. As far as I know, the job is disabled due to resource consumption on the CI server.
I think it would be better if we could eliminate the CONTRIB-QA plan and integrate the 2.x E2E tests into the 2.x platform and refapp plans.
@raff just to be clear… you’re using Distribution 3.x as the “mothership”/official plan, right?
this is not actually the build we are using for O3 (the 3.x E2E one was a temporary clone for purpose of practicing setting up automated tests within our ci); the one to use is Reference Application - Distribution 3.x: Plan summary - OpenMRS Bamboo - this is the one that builds dev3 and is triggered by GitHub actions.
Distribution 3.x: Main plan, will remain the plan, that builds & deploys to dev3
Distrubution 3.x E2E: Exists so we can try out integrating the steps that run the tests as part of the build. We did this as a cloned plan b/c we didn’t want to stop deployments while we figured out the E2E tests (e.g. right now it’s currently failing, which would have meant nothing would be deploying to dev3). Ideally we want these tests running on the official Distribution 3.x build (then we’d remove this cloned plan entirely) and per @jayasanka today we are almost ready to do this.
Distribution 3.x Releases: New, separate plan @ibacher recently created, designed for handling O3 releases. Shouldn’t be manually run; will only execute when we intentionally kick off a formal O3 RefApp Release. E.g. see Ian’s note here from today about a new O3 release version.
Thanks @raff this is extremely helpful to know - actually it might explain some of the issues @moduguvikram has been running into with trying to setup the O3 RefApp. I’ve filed an issue in our new “Operation Stabilize” epic for stabilizing the O3 RefApp; assigned this issue to you Raff : [O3-1864] - OpenMRS Issues
I merged some and following up on the pending ones with review comments.
The module is configured at a specific time to auto-trigger a plan every after 24 hours. However, any push done on the module by default auto triggers a ci plan.
@jayasanka what could be the ideal of having the tests integrated into two modules?
For right now, the workflow with having two jobs is more maintainable… The central problem is that we need to keep dev3 up-to-date with the latest changes to any of the frontends. We do that by building against the next tag from NPM, ensuring that we’re always pulling in the latest versions. However, for releases, we need fixed version numbers so that the release builds are repeatable. The easiest way to do this is via two commits, one which updates things to fixed version, the other which restores the next tags. In any case, at the present moment, it does need to be a manual choice about which versions are being included in a release, so having this workflow saves that.
I also don’t see any clear advantages at this point and at the current level of maturity to trying to segregate the releases to a separate branch. This is because most of the churn in the repo, right now, is updating the app configuration, which are commits that we’d then need to also do to the release branch (but, again, not every commit would necessarily be an appropriate candidate for merging into a release branch, e.g. if we were to add a new frontend module to the 3.x app, which happens from time-to-time, we wouldn’t necessarily want that in the release…).
