The purpose of this thread is to continue the discussions on the issue MF-899 which is to identify the challenges on running E2E tests against PRs. Currently, we don’t have any mechanisms to run relevant workflows when we open a PR from an ESM repo.
As I understood, this issue is for finding a way to configure the micro frontend repositories to run the relevant Github workflows(E2E tests) for every pull request. Am I correct? @jayasanka
I did some research and found that, in GitHub actions, it is possible to write a workflow to trigger a workflow in a different repository and to get the result. Basically, it will trigger the workflow as same as a manual trigger.
Using that, we can run workflows in QA Framework when someone creates a pull request in ESM repositories. But there is a problem with that approach.
Since we are running the E2E tests with the demo server (OpenMRS) in default, the tests will not consider the commits in the Pull Request. It will always run the E2E tests against the deployed version.
Also, in microfrontend pull request checks, it only builds the app (doesn’t deploy).
cc: @jayasanka
Thanks, @piumal1999 for the detailed description.
As you said, triggering a GitHub workflow on the QA framework won’t help since it just runs on a pre-compiled server.
One way we could achieve this is by, building the product using the PR > spin a local server > checkout to the openmrs-contrib-qaframework repo > run the relevant test/tests using npm scripts.
spinning a local server would be challenging, but can’t we use the proxy?
@jdick @bistenes @vasharma05 @ibacher @grace do you have any idea to address this problem?
According to the discussion we had in 2021-11-03T18:30:00Z Microfrontend Squad Call, we decided to try running the E2E tests within the local environment created with a pull request build.
I tried the previously mentioned solution and it works!
I updated the workflow of my forked openmrs-esm-patient-chart repository to run the microfrontend and run the E2E Tests.
This is the script I added to the workflow
(Btw, I noticed that it took longer than usual to pass the E2E tests. In default, the timeout was set to 5000ms. So I had to change it to 25000ms to pass the tests.)
Since the initial testing was successful, I need to clarify a few other things.
Let’s consider the openmrs-esm-patient-chart repository. If someone sends a pull request to that repository, we have 2 ways to run the E2E Tests.
The 1st option is easy to implement and has a 100% guarantee. But it will take more time and resources to complete the Github Checks.
For the 2nd option, we will need to find the E2E tests related to the chosen micro frontend repository.
What do you think? @jayasanka @bistenes
The idea of running only E2E tests related to the microfrontend being changed is nice, but the nature of the E2E tests is that they run across microfrontends. There’s no easy way to map which parts of the E2E tests correspond to which part of the codebase. So I’d just run the whole thing.
Amazing work @piumal1999! and thanks @bistenes for your input.
You can use cypress run command to run all tests at once. (probably you might need a tag because there are 2.x tests)
see: Command Line | Cypress Documentation
For the base URL and other configs. I think we can use environment variables. However, it’s not a priority we can add it as an improvement later.
Thanks again! Looking forward to the PR. 
Thank you @bistenes and @jayasanka
Yeah I’ll use this method
I tried with the command cypress run --spec src/test/resources/features/refapp-3.x/*/*.feature with wildcards. So it will run only the tests in refapp3.x directory.
Btw, some tests are failing due to timeout errors(even after setting the timeout to 120000ms)
Also it takes around 1 hour to complete the workflow
This is freaking. however, I think it takes long time because you have increased the timeout. As I understand these tests are failing not because it takes time to load, but it redirects to the login page for some reason. We need to investigate why.
Thanks @jayasanka @piumal1999 , this workflow especially refapp 3.x patient Registration have been failing for some good time . Is it because of the loginRedirects, probably you can revert that from within the code.
@sharif the patient registration is failing because there’s a bug in the registration flow. It has been already reported here: [O3-867] Patient registration throws error when registering patient - OpenMRS Issues
Hi @jayasanka, I checked the video recordings of failed recordings, but there weren’t any login redirections. I’ll try running those individually to see whether there’s any issue.
Btw, does OpenMRS use a paid version of GitHub actions in those repositories?
No. OpenMRS is open source software and our GitHub Actions allowances are currently those provided to any open source repo. If we need something more specialised, we might need to look into moving things over to our Bamboo instance.
