Urgent OpenMRS QA improvement request: Preferred Testing Framework

Dear folk,

The Current QA team is advancing towards automating our OpenMRS Quality Assurance processes for our tools and distributions and we are kindly requesting for your partnership to help us improve the current process and guide the selection of best sustainable tools for the framework.

We are urgently interested in selecting a quality assurance library to write our framework in. We have great interest in Selenium Vs Puppeteer but open to any other competitive option. We are doing some analysis and hope to soon conclude our selection. We welcome your contribution at this stage before we start automating test scripts for a list of requirements that you can still contribute to as well.

We are also evaluating possible options to spin up the tests. Which would be your preferred approach to packaging and running this testing framework?

Please kindly confirm if you are willing to partner with us so we can keep in touch with you.

Your timely (before 28th/Jan/2020) response will be highly appreciated.


OpenMRS has been using Selenium for Reference Application for quite some time, see

I’m not quite sure what you are intending to test. Do you want to extend UI testing beyond Reference Application? What are your goals?

Tests for RA are written with the help of

They are run every commit by our CI, which triggers travis-ci paired with SauceLabs, see e.g. https://ci.openmrs.org/browse/REFAPP-OMODDISTRO-INTTESTS-9452 and https://travis-ci.org/openmrs/openmrs-distro-referenceapplication/builds/639760177

Tests are run against MySQL+Chrome 48, MariaDB+Chrome 48, MySQL+Firefox 42, MariaDB+Firefox 42.

It’s quite extensive setup. There was a wiki page describing the whole setup, but I couldn’t find it now.

My recommendation would be to keep using the existing setup and focus on covering more cases. There’s been quite some effort involved in putting together the UI test framework based on Selenium. The choice of library is less important than proper use of it and following good practices.


Dear @raff

Am so glad you have responded and thanks for granting me saucelabs access, The current QA team is consuming the former efforts and supporting a predefined set of requirements through a more generic framework aimed to make it easier to extend to other interfaces and distributions beyond the reference application. @christine who is QA lead will elaborate on further goals.

I am sure once more people are comfortable proceeding with Selenium, we will greatly embrace the uitestframework.

I am reviewing that extensive setup and will get back to you if I have any questions.

I don’t have much experience between Selenium and Puppeteer, but I would agree with @raff… or in particular, though moving to another framework shouldn’t be out of the question, considering the migration work, etc, there needs to be a strong reason to move away from the incumbent framework (Selenium).

One thing that might constitute a “strong” reason, in my option, would be if there exists a tool that allows writing UI tests without having do to much or any coding, so we could expand the number of people that could write tests beyond those with Java experience.

1 Like

@raff, yes it is true that there is extensive work done with regard to Quality Assurance for OpenMRS product. It was noted especially in terms of development, QA is well-established and is actively in use. However, there is a gap end user testing. With this in mind, the team is looking at the already available resources (test cases on Jira and automated test cases among others) to determine what can be reused or redevelop. The goal of the QA team is more focused at the inclusion of implementers and stakeholders in the testing of OpenMRS products for purposes of sustainability.

There is more information on what the QA team is planning to achieve here.


@christine can you please drop this week’s recording here ?

@tendomartn, I don’t yet have the call recording but here are the meeting notes. If you would like to catch up on all our previous meetings, please find the notes here.

@christine @k.joseph Selenium is the king of the hill, and you cannot go wrong, I would suggest leveraging the existing investment and building on top of that rather than try something new, which will always end up without the tooling support Selenium already has

The adage to follow If it ain't broke, don't fix it