# Platform 2.0 Beta testing against RefApp on UAT server **URL:** \ **Category:** Quality Assurance\ **Tags:** release-testing, platform-2.0\ **Created:** [May 17, 2016, 5:39am UTC](https://talk.openmrs.org/t/platform-2-0-beta-testing-against-refapp-on-uat-server/6393 "2016-05-17T05:39:30Z")\ **Posts on this page:** 20\ **Page:** 1
**Author:** ![maany](https://talk.openmrs.org/user_avatar/talk.openmrs.org/maany/32/692_2.png) [@maany](https://talk.openmrs.org/u/maany)\ **Post date:** [May 23, 2016, 10:51am UTC](https://talk.openmrs.org/t/platform-2-0-beta-testing-against-refapp-on-uat-server/6393/1 "2016-05-23T10:51:52Z")
We are trying to figure out how to test Platform 2.0 Beta and based on previous discussions on talk and PM calls, we have zeroed in on the following 2 ways : 1. Deploy reference application on Platform 2.0 beta (this setup currently lives on [uat-platform.openmrs.org](http://uat01.openmrs.org:8080/)). Then run Automated and Release tests for the RefApp. 2. Test rest endpoints for rest-ws and fhir modules on the same uat server manually or by automating the process With regards to point 1, we would like to leverage some of the work you did on creating [Test Cycles on Jira for the Reference Application JIRA project](https://issues.openmrs.org/browse/RA/?selectedTab=com.thed.zephyr.je:pdb_cycle_panel_section). Specifically, we wish to clone the Automated Tests and OpenMRS 2.2 release tests and run them(manually/ automatically) against the RefApp instance on [uat-platform.openmrs.org](http://uat-platform.openmrs.org). Would you have any tips/ suggestions on how we should go about cloning and running these tests? // cc @raff@tmueller ---
**Author:** ![raff](https://talk.openmrs.org/user_avatar/talk.openmrs.org/raff/32/11894_2.png) [@raff](https://talk.openmrs.org/u/raff)\ **Post date:** [May 17, 2016, 8:59am UTC](https://talk.openmrs.org/t/platform-2-0-beta-testing-against-refapp-on-uat-server/6393/2 "2016-05-17T08:59:29Z")
@tmueller is no longer actively involved in testing. For setting up release tests please see the QA Leaders section point 4 at [https://wiki.openmrs.org/display/docs/QA+Testing+Manual](https://wiki.openmrs.org/display/docs/QA+Testing+Manual) Most tests have been automated, thus you can find them under Automated Tests Cycle, but only a few of them can be reliably run in a CI environment due to delays and timeouts handling. We are in [the process of fixing the rest](https://wiki.openmrs.org/display/docs/Automated+Testing+Guidelines). However, you should still be able to run them all locally (less prone to delays and timeouts as opposed to CI), which I would recommend for the platform release. I’ll add instructions on how to do that to the wiki by tomorrow. Some automated tests may be failing and they will need to be run manually to verify why. There’s a few people listed under the QA team at [https://wiki.openmrs.org/display/docs/QA+Testing+Manual](https://wiki.openmrs.org/display/docs/QA+Testing+Manual) whom I’d invite to a testing sprint and ask to run tests manually as described at [https://wiki.openmrs.org/display/docs/How+to+execute+a+Test+Case](https://wiki.openmrs.org/display/docs/How+to+execute+a+Test+Case) ---
**Author:** ![maany](https://talk.openmrs.org/user_avatar/talk.openmrs.org/maany/32/692_2.png) [@maany](https://talk.openmrs.org/u/maany)\ **Post date:** [May 23, 2016, 12:44pm UTC](https://talk.openmrs.org/t/platform-2-0-beta-testing-against-refapp-on-uat-server/6393/3 "2016-05-23T12:44:45Z")
Thanks for the info @raffWe have RefApp running against Platform 2.0 beta on [uat-platform.openmrs.org](http://uat-platform.openmrs.org), I think we are good to go ahead and create the **following test cycles** and also a **testing sprint**. (it would be great to have it this week) - Clone the test cycle “OpenMRS 2.2 Release Tests” to “Platform 2.0 Beta Release Tests” - Clone the test cycle “OpenMRS 2.2 Functional Tests” to “Platform 2.0 Beta Functional Tests” I was wondering who should be handling these tasks, officially. (sorry, new to QA). I would definitely love to help out if we are looking for someone! 😃 > [@raff](#): > > However, you should still be able to run them all locally (less prone to delays and timeouts as opposed to CI), which I would recommend for the platform release. I’ll add instructions on how to do that to the wiki by tomorrow. I just cloned openmrs-distro-referenceapplication and did a mvn clean install to run the tests locally. Only 8 tests ran of which 2 were skipped. This behavior was exactly similar to how these tests run on [Bamboo](https://ci.openmrs.org/browse/REFAPP-OMODDISTRO-INTTESTS-4617/test). i.e the same tests ran in both local environment and on bamboo. Could you direct me to the updated instructions to run all of the tests locally. 😃 > [@raff](#): > > here’s a few people listed under the QA team at [QA Testing Manual - Documentation - OpenMRS Wiki](https://wiki.openmrs.org/display/docs/QA+Testing+Manual) whom I’d invite to a testing sprint and ask to run tests manually That would be great!! Also, If we need more manpower here, there are **19 awesome folks from our community** who had reached out to me in order to help us test Platform 2.0 Beta. I’d be happy to connect them with sprint lead whoever he/she would officially be. ---
**Author:** ![raff](https://talk.openmrs.org/user_avatar/talk.openmrs.org/raff/32/11894_2.png) [@raff](https://talk.openmrs.org/u/raff)\ **Post date:** [May 24, 2016, 4:12pm UTC](https://talk.openmrs.org/t/platform-2-0-beta-testing-against-refapp-on-uat-server/6393/4 "2016-05-24T16:12:47Z")
Please see [https://github.com/openmrs/openmrs-distro-referenceapplication#running-ui-tests-locally](https://github.com/openmrs/openmrs-distro-referenceapplication#running-ui-tests-locally) @maany, feel empowered to create test cycles and add any failing automated tests to the Platform 2.0 Beta Functional Tests cycle so that they are executed manually to confirm, if something got broken or a test simply needs to be updated. ---
**Author:** ![burke](https://talk.openmrs.org/user_avatar/talk.openmrs.org/burke/32/17_2.png) [@burke](https://talk.openmrs.org/u/burke)\ **Post date:** [May 26, 2016, 1:41pm UTC](https://talk.openmrs.org/t/platform-2-0-beta-testing-against-refapp-on-uat-server/6393/5 "2016-05-26T13:41:08Z")
@maany, I took the list of automated tests for the reference application and marked some as high priority [here](https://docs.google.com/spreadsheets/d/1wR_tQlqOfVEemLzQnhnzYguMA9q3tKoXSdVFr09qguw/edit?usp=sharing). If folks aren’t able to get through all the tests, I would prioritize these. [https://docs.google.com/spreadsheets/d/1wR\_tQlqOfVEemLzQnhnzYguMA9q3tKoXSdVFr09qguw/pubhtml?gid=0&single=true](https://docs.google.com/spreadsheets/d/1wR_tQlqOfVEemLzQnhnzYguMA9q3tKoXSdVFr09qguw/pubhtml?gid=0&single=true) ---
**Author:** ![maany](https://talk.openmrs.org/user_avatar/talk.openmrs.org/maany/32/692_2.png) [@maany](https://talk.openmrs.org/u/maany)\ **Post date:** [May 27, 2016, 9:51am UTC](https://talk.openmrs.org/t/platform-2-0-beta-testing-against-refapp-on-uat-server/6393/6 "2016-05-27T09:51:27Z")
Thanks @raff and @burkeI’ve incorporated your advice and created the following 2 Test Cycles 1. [Platform 2.0 Release Tests](https://issues.openmrs.org/secure/enav/#?query=project%20%3D%20%22PLAT%22%20AND%20fixVersion%20%3D%20%22Unscheduled%22%20AND%20cycleName%20in%20(%22Platform%202.0%20Release%20Tests%22)) 2. [Platform 2.0 Beta and OpenMRS 2.4-SNAPSHOT Functional Tests](https://issues.openmrs.org/secure/enav/#?query=project%20%3D%20%22RA%22%20AND%20fixVersion%20%3D%20%22Unscheduled%22%20AND%20cycleName%20in%20(%22Platform%202.0%20Beta%20and%20OpenMRS%202.4-SNAPSHOT%20Functional%20Tests%22)) I have also updated the [Platform 2.0 Beta testing Wiki page](https://wiki.openmrs.org/display/docs/Platform+2.0+Beta+Testing) to include accurate instructions on executing the tests listed in the above test cycles. ---
**Author:** ![maany](https://talk.openmrs.org/user_avatar/talk.openmrs.org/maany/32/692_2.png) [@maany](https://talk.openmrs.org/u/maany)\ **Post date:** [June 13, 2016, 2:50pm UTC](https://talk.openmrs.org/t/platform-2-0-beta-testing-against-refapp-on-uat-server/6393/7 "2016-06-13T14:50:43Z")
@wyclif @dkayiwa could you look into the following failing tests whenever you get a chance this week 😃 1. [https://issues.openmrs.org/browse/RA-674](https://issues.openmrs.org/browse/RA-674) 2. [https://issues.openmrs.org/browse/RA-682](https://issues.openmrs.org/browse/RA-682) 3. [https://issues.openmrs.org/browse/RA-694](https://issues.openmrs.org/browse/RA-694) 4. [https://issues.openmrs.org/browse/RA-747](https://issues.openmrs.org/browse/RA-747) 5. [https://issues.openmrs.org/browse/RA-772](https://issues.openmrs.org/browse/RA-772) ---
**Author:** ![wyclif](https://talk.openmrs.org/user_avatar/talk.openmrs.org/wyclif/32/19607_2.png) [@wyclif](https://talk.openmrs.org/u/wyclif)\ **Post date:** [June 13, 2016, 8:15pm UTC](https://talk.openmrs.org/t/platform-2-0-beta-testing-against-refapp-on-uat-server/6393/8 "2016-06-13T20:15:04Z")
These tickets seem to belong to the reference application project, why are they considered as blockers to the platform? Are their underlying root causes coming from the platform? And why should tests hold back the release if the features are actually working? ---
**Author:** ![maany](https://talk.openmrs.org/user_avatar/talk.openmrs.org/maany/32/692_2.png) [@maany](https://talk.openmrs.org/u/maany)\ **Post date:** [June 14, 2016, 1:00am UTC](https://talk.openmrs.org/t/platform-2-0-beta-testing-against-refapp-on-uat-server/6393/9 "2016-06-14T01:00:54Z")
Hey @wyclif, [quote=“wyclif, post:8, topic:6393”] Are their underlying root causes coming from the platform? [/quote] We want to figure this part out. If the root cause is the platform, we intent to fix them before the release. 🙂 ---
**Author:** ![wyclif](https://talk.openmrs.org/user_avatar/talk.openmrs.org/wyclif/32/19607_2.png) [@wyclif](https://talk.openmrs.org/u/wyclif)\ **Post date:** [June 14, 2016, 3:13pm UTC](https://talk.openmrs.org/t/platform-2-0-beta-testing-against-refapp-on-uat-server/6393/10 "2016-06-14T15:13:26Z")
I see, ProviderTest seems to marked as ignored and doesn’t appear on [this](https://issues.openmrs.org/browse/RA/?selectedTab=com.thed.zephyr.je:pdb_cycle_panel_section) page, the instructions on this [page](https://wiki.openmrs.org/x/r4P-B) say you shouldn’t have to run the tests manually, they should be marked as fail if they fail on CI, when I look at CI everything is green, so I don’t seem to understand what needs to be fixed. ---
**Author:** ![maany](https://talk.openmrs.org/user_avatar/talk.openmrs.org/maany/32/692_2.png) [@maany](https://talk.openmrs.org/u/maany)\ **Post date:** [June 14, 2016, 7:16pm UTC](https://talk.openmrs.org/t/platform-2-0-beta-testing-against-refapp-on-uat-server/6393/11 "2016-06-14T19:16:30Z")
> [@wyclif](#): > > when I look at CI everything is green, @wyclif, the CI only runs 8 tests of which 2 are skipped. [https://ci.openmrs.org/browse/REFAPP-OMODDISTRO-INTTESTS-4659/test](https://ci.openmrs.org/browse/REFAPP-OMODDISTRO-INTTESTS-4659/test) There are about 80 tests. In order to execute the remaining tests on CI, they need to be modified as described here : [Automated Testing Guidelines - Documentation - OpenMRS Wiki](https://wiki.openmrs.org/display/docs/Automated+Testing+Guidelines) That’s why we decided to manually do the tests for now. > [@wyclif](#): > > I don’t seem to understand what needs to be fixed In case of ProviderTest, the reason causing it to fail seems to be step 2 here : [https://issues.openmrs.org/secure/enav/#/540?query=issue%3DRA-747&offset=1](https://issues.openmrs.org/secure/enav/#/540?query=issue%3DRA-747&offset=1) I understand the ambiguity surrounding the exact description of the task. It’s not a very rigid task and we just want a dev/5/ to take a look at the failing tests. The idea, like you mentioned before, is to figure out the root cause for the failing tests. If in order to fix the root cause, we need to make changes in Platform code, we intent to make those changes now itself. If the code changes are required to be made in RefApp, a ticket should do! I hope this makes sense 🙂 ---
**Author:** ![wyclif](https://talk.openmrs.org/user_avatar/talk.openmrs.org/wyclif/32/19607_2.png) [@wyclif](https://talk.openmrs.org/u/wyclif)\ **Post date:** [June 15, 2016, 12:17am UTC](https://talk.openmrs.org/t/platform-2-0-beta-testing-against-refapp-on-uat-server/6393/12 "2016-06-15T00:17:03Z")
It doesn’t make much sense to me as to why ui tests would fail just because you switched to platform 2.0, first thing that comes to my mind is that the data and metadata on the uat01 server is different from that on the int02 server the tests are scripted against, I will look into this and see if it is the cause ---
**Author:** ![wyclif](https://talk.openmrs.org/user_avatar/talk.openmrs.org/wyclif/32/19607_2.png) [@wyclif](https://talk.openmrs.org/u/wyclif)\ **Post date:** [June 15, 2016, 3:07am UTC](https://talk.openmrs.org/t/platform-2-0-beta-testing-against-refapp-on-uat-server/6393/13 "2016-06-15T03:07:16Z")
Something is still not clear e.g where does one run the tests from? According to the instructions on one of the wiki pages about these tests it seems like you can’t do it manually so how do we expect a developer to debug the tests and fix them? And if they can run them manually, where is the code they are supposed to check out to work on things? Because for sure it can’t be the master branch of the distro project. Anyways, I took some to look closely at some of the tests and noticed some glaring issues, some key ones are: - Seems that I guessed right that the metadata on different test servers makes it impossible to have the tests pass against all of them, e.g the LoginTest uses the unknown location to authenticate but on uat server unknown location doesn’t have login tag so you can’t log in anyways. - Some tests appear incorrectly written, e.g when the web driver instance is started, the code automatically authenticates the user as admin but some tests the way they are written, they attempt to re-authenticate a user may be as a different user for test purposes which means they go to the login page when user is already logged in which of course will get redirected to back to the home page so this makes some of these tests’ assertions fail. Also some tests expect a certain app count which can vary based on the modules and their versions installed on a given test server. - Some tests don’t actually exist, e.g the tests mentioned in the descriptions of [RA-772](https://issues.openmrs.org/browse/RA-772) and [RA-747](https://issues.openmrs.org/browse/RA-747), Delete Visit Note Test doesn’t exist as mentioned in [RA-682](https://issues.openmrs.org/browse/RA-682) In summary, there is nothing I found that suggests that the failures are caused by platform 2.0, the tests are just incorrect. It appears like may be we need to have a separate branch for each test server or make the metadata unique on all test servers . We need to move some hard coded things out of the tests, e.g the app count because it depends on the version of a module or the ref app distro installed on the test server you are running the tests against. ---
**Author:** ![wyclif](https://talk.openmrs.org/user_avatar/talk.openmrs.org/wyclif/32/19607_2.png) [@wyclif](https://talk.openmrs.org/u/wyclif)\ **Post date:** [June 15, 2016, 7:25pm UTC](https://talk.openmrs.org/t/platform-2-0-beta-testing-against-refapp-on-uat-server/6393/14 "2016-06-15T19:25:04Z")
Hi @maany! I want to fix the logout issue but I don’t know how you’re building the distro that you’re running on the uat server, are you doing it manually? ---
**Author:** ![dkayiwa](https://talk.openmrs.org/user_avatar/talk.openmrs.org/dkayiwa/32/179_2.png) [@dkayiwa](https://talk.openmrs.org/u/dkayiwa)\ **Post date:** [June 15, 2016, 7:34pm UTC](https://talk.openmrs.org/t/platform-2-0-beta-testing-against-refapp-on-uat-server/6393/15 "2016-06-15T19:34:38Z")
After committing and ensuring that the corresponding module CI plan has run successfully, just manually run this: [https://ci.openmrs.org/browse/REFAPP-DOM](https://ci.openmrs.org/browse/REFAPP-DOM) ---
**Author:** ![wyclif](https://talk.openmrs.org/user_avatar/talk.openmrs.org/wyclif/32/19607_2.png) [@wyclif](https://talk.openmrs.org/u/wyclif)\ **Post date:** [June 15, 2016, 10:53pm UTC](https://talk.openmrs.org/t/platform-2-0-beta-testing-against-refapp-on-uat-server/6393/16 "2016-06-15T22:53:24Z")
The server that the plan deploys to is completely different and is not running platform 2.0 and the updated modules ---
**Author:** ![dkayiwa](https://talk.openmrs.org/user_avatar/talk.openmrs.org/dkayiwa/32/179_2.png) [@dkayiwa](https://talk.openmrs.org/u/dkayiwa)\ **Post date:** [June 15, 2016, 11:05pm UTC](https://talk.openmrs.org/t/platform-2-0-beta-testing-against-refapp-on-uat-server/6393/17 "2016-06-15T23:05:33Z")
Just do the above and you will see your changes reflected on [http://uat01.openmrs.org:8080/openmrs](http://uat01.openmrs.org:8080/openmrs) ---
**Author:** ![wyclif](https://talk.openmrs.org/user_avatar/talk.openmrs.org/wyclif/32/19607_2.png) [@wyclif](https://talk.openmrs.org/u/wyclif)\ **Post date:** [June 15, 2016, 11:44pm UTC](https://talk.openmrs.org/t/platform-2-0-beta-testing-against-refapp-on-uat-server/6393/18 "2016-06-15T23:44:39Z")
My question is not really how to deploy to the uat server but rather where to get the code to setup my dev environment, that way I can set it up on my machine, reproduce the issue locally and try to fix it. ---
**Author:** ![dkayiwa](https://talk.openmrs.org/user_avatar/talk.openmrs.org/dkayiwa/32/179_2.png) [@dkayiwa](https://talk.openmrs.org/u/dkayiwa)\ **Post date:** [June 16, 2016, 9:18am UTC](https://talk.openmrs.org/t/platform-2-0-beta-testing-against-refapp-on-uat-server/6393/19 "2016-06-16T09:18:59Z")
Oh i see! Here we go: [https://github.com/dkayiwa/openmrs-distro-referenceapplication](https://github.com/dkayiwa/openmrs-distro-referenceapplication) ---
**Author:** ![wyclif](https://talk.openmrs.org/user_avatar/talk.openmrs.org/wyclif/32/19607_2.png) [@wyclif](https://talk.openmrs.org/u/wyclif)\ **Post date:** [June 16, 2016, 11:32am UTC](https://talk.openmrs.org/t/platform-2-0-beta-testing-against-refapp-on-uat-server/6393/20 "2016-06-16T11:32:33Z")
Great! This is what I’ve been looking for all along, thanks @dkayiwa [Next page](https://talk.openmrs.org/t/platform-2-0-beta-testing-against-refapp-on-uat-server/6393.md?page=2)