RefApp UI tests and SauceLabs

I’d like to see if I can figure out why the UI tests are flaky. I found this conversation and others:

One thing I do not understand: Looking at Bamboo build log, it appears that the UI tests are actually being run on SauceLabs, but I don’t see where that is actually hooked up. How is that done? I looked at the Bamboo plugins, but I don’t see SL there. Can I run the UI tests on SauceLabs myself? Is there a SauceLabs OpenMRS-Testing account? Or it looks like the tests are just being run under Rafa’s account? Please advise.

1 Like

Never mind the question about how the UI tests are run at SauceLabs, I see it now in the UITestFramework. But the other questions about SL accounts and such are still valid.

Lee

Ignoring SauceLabs for now, running tests locally…

I observe that all the recent UI test failures on bamboo are actually this:

 javax.ws.rs.BadRequestException: HTTP 400 Bad Request

which has nothing to do with the actual Selenium UI tests, but rather is coming from the REST call the tests make during setup to create test Persons and such. I was able to reproduce this after repeated trials and I’m pretty sure the problem is in the birthdate field - the test was using 0-11 for the month instead of 1-12. When I fixed that locally, I could not get the error to happen. So that’s one fix I definitely think should go in. It’s actually in the uitestframework.

I also found we’re using a rather out of date version of Selenium, so I’d like to update that too. Also the chromedrivers. Oh, and I found a missing dependency on commons-io that only shows up when you use chrome.

I know I used to have commit privileges for uitestframework and ui-tests in refapp, but perhaps I should do a pull request for now because it’s been so long - get Rafa or someone to code-review.

Let me know.

Lee

oops, I forgot how git and github work, so I apparently went ahead and committed the birthdate fix.

Lee

Thanks for looking into this Lee.

Generally you can decide whether something should be a PR or a direct commit.

Since you did commit the code, you should probably trigger a build of the distribution project on the CI server to verify things. (Or at least make sure it is triggered by updates to uitestframework.)

Note that I had just released a version of uitf, so you will need to switch the distro’s pom.xml to refer to the new snapshot.

-Darius (by phone)

··· On Mar 13, 2016 1:19 PM, "Lee Breisacher" wrote:

leebreisacher https://talk.openmrs.org/users/leebreisacher Tester March 13

oops, I forgot how git and github work, so I apparently went ahead and committed the birthdate fix.

Lee

Visit Topic https://talk.openmrs.org/t/refapp-ui-tests-and-saucelabs/5263/4 or reply to this email to respond

In Reply To leebreisacher https://talk.openmrs.org/users/leebreisacher Tester March 13 Ignoring SauceLabs for now, running tests locally… I observe that all the recent UI test failures on bamboo are actually this: javax.ws.rs.BadRequestException: HTTP 400 Bad Request which has nothing to do with the actual Selenium UI tests, but rather is coming from the REST call the tests mak…

Visit Topic https://talk.openmrs.org/t/refapp-ui-tests-and-saucelabs/5263/4 or reply to this email to respond

To stop receiving notifications for this particular topic, click here https://talk.openmrs.org/t/refapp-ui-tests-and-saucelabs/5263/unsubscribe. To unsubscribe from these emails, change your user preferences https://talk.openmrs.org/my/preferences

or, click here <reply@talk.openmrs.org?subject=unsubscribe> to unsubscribe via email.

The commit did trigger a build of uitf (of course), which then triggered a build of refapp-distro, but I’m confused about how the version dependencies work. The refapp-distro pom has:

	<uitestframeworkVersion>1.7.2</uitestframeworkVersion>

And the uitf pom has:

<version>1.8-SNAPSHOT</version>

So that means the refapp will not pick up the latest uitf, right? How is this supposed to work?

Lee

You’re correct. Current changes to uitf are building the 1.8-snapshot version, but the distro pom declares that the distribution will use an earlier version.

You should change the distro pom to depend on the latest version of uitf (1.8-SNAPSHOT), so the distro build can see your latest changes.

··· On Mon, Mar 14, 2016 at 7:02 PM, Lee Breisacher wrote:

leebreisacher https://talk.openmrs.org/users/leebreisacher Tester March 15

The commit did trigger a build of uitf (of course), which then triggered a build of refapp-distro, but I’m confused about how the version dependencies work. The refapp-distro pom has:

1.7.2

And the uitf pom has:

1.8-SNAPSHOT

So that means the refapp will not pick up the latest uitf, right? How is this supposed to work?

Lee

Visit Topic https://talk.openmrs.org/t/refapp-ui-tests-and-saucelabs/5263/6 or reply to this email to respond

To stop receiving notifications for this particular topic, click here https://talk.openmrs.org/t/refapp-ui-tests-and-saucelabs/5263/unsubscribe. To unsubscribe from these emails, change your user preferences https://talk.openmrs.org/my/preferences

or, click here <reply@talk.openmrs.org?subject=unsubscribe> to unsubscribe via email.

OK, but then when/how does the refapp pom get changed to a non-SNAPSHOT version? (Just trying to understand the process).

Lee

Many modules are set up to be released from bamboo. These automatically update the refapp distro’s pom.xml with their released version. uitestframework is not released from bamboo, so I manually updated the pom.xml file.

You can see examples of both these things in the history of that file: https://github.com/openmrs/openmrs-distro-referenceapplication/commits/master/pom.xml

··· On Mon, Mar 14, 2016 at 9:17 PM, Lee Breisacher wrote:

leebreisacher https://talk.openmrs.org/users/leebreisacher Tester March 15

OK, but then when/how does the refapp pom get changed to a non-SNAPSHOT version? (Just trying to understand the process).

Lee

Visit Topic https://talk.openmrs.org/t/refapp-ui-tests-and-saucelabs/5263/8 or reply to this email to respond

In Reply To darius https://talk.openmrs.org/users/darius Darius Jazayeri https://talk.openmrs.org/users/darius /dev/5 March 15 You’re correct. Current changes to uitf are building the 1.8-snapshot version, but the distro pom declares that the distribution will use an earlier version. You should change the distro pom to depend on the latest version of uitf (1.8-SNAPSHOT), so the distro build can see your latest changes. …

Visit Topic https://talk.openmrs.org/t/refapp-ui-tests-and-saucelabs/5263/8 or reply to this email to respond

To stop receiving notifications for this particular topic, click here https://talk.openmrs.org/t/refapp-ui-tests-and-saucelabs/5263/unsubscribe. To unsubscribe from these emails, change your user preferences https://talk.openmrs.org/my/preferences

or, click here <reply@talk.openmrs.org?subject=unsubscribe> to unsubscribe via email.

@leebreisacher looks like the same is happening here: https://ci.openmrs.org/browse/REFAPP-OMODDISTRO-INTTESTS-4573/log