I d’ like to understand a bit more on the execution of the test cases and how to ignore them. Up to now, I’ve realized that a failing test should be ignored by adding @Ignore in the code. I don’t know if this assumption is true. After a while, I discovered a test RecordMergeIssueTest which fails locally. It sounds really odd, since it has no @ignore. I was Then, I became suspicious and I checked travis CI log (reference to one of the last execution) and I noticed that this test does not run. a) Do you agree on the fact that RecordMergeIssueTest has not been executed? b) if so, how do you exclude the tests from a CI job? How can see the list of excluded tests?
@domenico, see https://github.com/openmrs/openmrs-distro-referenceapplication/blob/master/ui-tests/pom.xml#L53 It says to execute tests only from the BuildTests group. It’s defined in the CI profile, which is used for running tests on CI.
We used that group when we didn’t have access to saucelabs and we couldn’t afford to run all tests, because the build was getting way too slow. Now, we should probably get rid of it since we can run all tests in acceptable time.
So to fix things…
- Remove the group from the CI profile config so that all tests are run.
- Mark failing tests with @Ignore.
- Get rid of the @Category(BuildTests.class) annotation.
- Remove annotating test with @Category(BuildTests.class) from the UI test guidelines
Just a clarification regarding the first point: Is it sufficient to update the pom.xml file (I’ll delete the row you pointed out) or we should check also CI profile, maybe a file in Travis? let me know, I’d like to take into account also the eventual CI configuration. Maybe at the beginning some of you might provide guidance.
Yes, it is sufficient to delete that line.
For reference here’s the command, which is used to run tests on Travis-CI https://github.com/openmrs/openmrs-distro-referenceapplication/blob/master/travis_script.sh#L5
I’ve been executing all the tests in order to get a big picture of failing tests. When I execute some of them locally, they fail while they do not fail on CI. Just an example: EditAllergyTest with local execution results in: Running locally… POST person failed javax.ws.rs.RedirectionException: HTTP 301 Moved Permanently at org.glassfish.jersey.client.JerseyInvocation.createExceptionForFamily(JerseyInvocation.java:945) at org.glassfish.jersey.client.JerseyInvocation.convertToException(JerseyInvocation.java:932) at org.glassfish.jersey.client.JerseyInvocation.translate(JerseyInvocation.java:770) at org.glassfish.jersey.client.JerseyInvocation.access$500(JerseyInvocation.java:90) at org.glassfish.jersey.client.JerseyInvocation$2.call(JerseyInvocation.java:671) at org.glassfish.jersey.internal.Errors.process(Errors.java:315) at org.glassfish.jersey.internal.Errors.process(Errors.java:297) at org.glassfish.jersey.internal.Errors.process(Errors.java:228) at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:422) at org.glassfish.jersey.client.JerseyInvocation.invoke(JerseyInvocation.java:667) at org.glassfish.jersey.client.JerseyInvocation$Builder.method(JerseyInvocation.java:423) at org.glassfish.jersey.client.JerseyInvocation$Builder.post(JerseyInvocation.java:326) at org.openmrs.uitestframework.test.RestClient.post(RestClient.java:80) at org.openmrs.uitestframework.test.TestData$JsonTestClass.create(TestData.java:69) at org.openmrs.uitestframework.test.TestData$JsonTestClass.create(TestData.java:65) at org.openmrs.uitestframework.test.TestData.createPerson(TestData.java:576) at org.openmrs.uitestframework.test.TestBase.createTestPatient(TestBase.java:374) at org.openmrs.uitestframework.test.TestBase.createTestPatient(TestBase.java:380) at org.openmrs.reference.EditAllergyTest.setUp(EditAllergyTest.java:36) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:24) at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27) at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55) at org.junit.rules.TestWatcher$1.evaluate(TestWatcher.java:55) at org.junit.rules.RunRules.evaluate(RunRules.java:20) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229) at org.junit.runners.ParentRunner.run(ParentRunner.java:309) at org.junit.runners.Suite.runChild(Suite.java:127) at org.junit.runners.Suite.runChild(Suite.java:26) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229) at org.junit.runners.ParentRunner.run(ParentRunner.java:309) at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:86) at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:459) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:678) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:382) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:192)
The http request is: method=POST, uri=http://qa-refapp.openmrs.org/openmrs/ws/rest/v1/person, status=301, reason=Moved Permanently with the “body” part {“names”:[{“givenName”:“User233734”,“familyName”:“Tester233734”,“middleName”:“Interface233734”}],“addresses”:[{“address1”:“Address1233734”,“address2”:“Address2233734”,“cityVillage”:“City233734”,“stateProvince”:“State233734”,“postalCode”:“33735”,“country”:“Country233734”}],“gender”:“M”,“birthdate”:“1991-10-25”}
Have you experienced any thing like that?
For the one interested in the method POST used a wrong URL, the correct one is uri=https://qa-refapp.openmrs.org/openmrs/ws/rest/v1/person. The pon.xml file must be fixed. A bug will be open