We are (incorrectly) using devtest01 and devtest02 for automated and manual
testing of the reference application.
Rather than being devtest these should be qa or uat or something.
Michael Downey brings this up from time to time and I agree with him every
time, but I don’t have time to work through all the details. Does anyone
out there have done spare cycles to take this on?
I presume the steps involved are:
Decide, with Michael, what the servers should be called, and get them
set up by help desk.
Make changes to CI to point to the new servers
Update wiki documentation
Turn off the old servers, see what breaks that I have missed here.
int01 – OpenMRS Platform built from latest commit to core or any modules bundled in the platform. Purpose is to prove OpenMRS Platform still runs and to serve for automated testing.
int02 – OpenMRS Reference Application built from latest commit to core or any module distributed in the ref app. Purpose is to prove OpenMRS Community Distribution still runs and for automated testing.
int03+ – Used whenever we need an additional server for automated testing (e.g., automated testing of a new module that doesn’t belong on or might unnecessarily disrupt the operation of int01 or int02).
qa01 – OpenMRS Platform built from latest commit. Used for manual testing (e.g., making manual REST calls to it). This would be similar to a headless version of demo.openmrs.org, but would be running a version built from the latest commit (to core or any module bundled in the platform.
qa02 – OpenMRS Reference Application built from latest commit. Used for manual testing. This would be similar to demo.openmrs.org, but would be running a version built from the latest commit (to core or any module distributed in the ref app).
qa03+ – Used whenever we need to perform manual testing for an application or module that doesn’t belong on or might unnecessarily disrupt the operation of qa01.
uat01 – OpenMRS Platform from manually triggered build. Purpose is to run a specific snapshot of the platform released to uat01, so specific releases could be user-tested during preparation for a new release.
uat02 – OpenMRS Reference Application from manually triggered build. Purpose is to run a specific snapshot of the reference application released to uat02, so specific releases could be user-tested during preparation for a new release.
uat03+ – Used whenever we are releasing something other than the reference application and need a server onto which trigger new builds as we work through user acceptance testing of the impending release.
If we agree, then:
CI should be deploying “nightly” (build from latest commit) of platform to int01 & qa01 and the reference app to int02 and qa02.
Automated testing by CI of platform or reference application should be performed against int01 and int02, respectively.
Any /dev/3+ should be able to manually trigger a new release (of a particular commit) of the platform to uat01 or the reference application to uat02.
I think we should also have qa and uat environments for the OpenMRS
Platform release. That needs to be tested also.
Further, is there some reason we can’t give the servers more meaningful
names in this scenario? E.g. int-platform and int-refapp instead of int01
and int02.
It would nice to stay a little consistent with the numbering where the ones that have a 1 e.g int01, qa01 etc are for the platform deployments while those with the 2 e.g int02, qa02 are for ref app deployments
The rest look okay (platform deploying automatically to int01.openmrs.org, UAT01/UAT02 manually deploying to corresponding servers, and demo manually deployed). I assume there’s a separate, automated process to reset the demo hourly.
When we get done shifting things around, there shouldn’t be any builds deploying to devtest servers.
My modification to the proposal was to specifically name the servers for
platform and refapp as:
int-platform
int-refapp
uat-platform
uat-refapp
Distro builds should be deployed to int-refapp automatically, and to
uat-refapp iff the UI tests pass. (Though I suppose it doesn’t truly need
to be conditional.) CI already does this but we’d need to tweak the plans
with renamed/switched servers.
Optionally the plans could be refactored, but the minimal amount of work
really is just assigning the server names and making trivial mods to our
build plans.
If they pass, deploy automatically to qa02/qa-refapp
Allow manual deployment to uat02/uat-refapp
Questions:
Are we going as int01/int02/int0X or int-platform/int-refapp/int-extras-01? Both are good for me. I suppose we can start with the numbers and move it later.
Why do we need both uat01/uat-platform and demo? Aren’t they the same thing? I’m not sure there’s a reset process on it. Maybe a cron process inside the machine, I don’t have easy access to it.
If dev machines are not going to be deployed from CI, how developers are planning on using them? Do they need a key, or what?
@cintiadr the Platform should not be quite as you describe it.
The platform should be driven by this github repo, and it should get its own plan(s) in the CI pipeline, similar to the Reference Application one. (Basically it’s openmrs-core + webservices.rest module. Eventually we may add a couple more modules.)
I can only answer a couple of your questions at the bottom:
Where are the end-to-end tests of the platform?
We don’t have any. We should have some REST-based tests, I guess, since the platform eventually will not include a UI.
Why do we need both uat01/uat-platform and demo? Aren’t they the same thing? I’m not sure there’s a reset process on it. Maybe a cron process inside the machine, I don’t have easy access to it.
demo should have the latest released version of the reference application (and its database should be regularly reset, but its code only updated on a release). uat would have the development version of code, and would be updated on every build.
Which servers should Ebola be deployed to?
Ebola should just stay exactly as it is on devtest03.openmrs.org. No work is happening on that module at this point, so we don’t need to do any more deployments (and this should be disabled already) but we do want to keep that around for a while longer as a demonstration.
@cintiadr aologies, just woke up and saw this thread. So we’re currently using qa01.openmrs.org for hosting our FHIR work. We initially requested a QA server because the FHIR work was intended to be tested during a hackathon.
Problem is, we configured apache to bypass security on this machine, and we need these configs to make the FHIR work run against the SMART Platform.
Do you intend to destroy this setup? if so, i’d really love some grace period to copy over the apache configs so that I can replicate the current setup elsewhere I’d hate to have to figure this out all over again…
If they pass, deploy automatically to qa02/qa-refapp
Allow manual deployment to uat02/uat-refapp
devtest03 show not be touched.
Questions pending:
Are we going as int01/int02/int0X or int-platform/int-refapp/int-extras-01? Both are good for me. I suppose we can start with the numbers and move it later. @ryan has probably some opinion how hard it would be.
If dev machines are not going to be deployed from CI, how developers are planning on using them? Do they need a key, or what?
@darius If the platform won’t have a UI, would it make sense to deploy to int01/int-platform to run the REST-based test? Do we have those tests now?
uat should be deployed manually or automatically? I believe @darius and @burke disagree here.
Can I make deployment to demo automatic after a release of the platform? If the platform won’t have an UI, will it still be useful for a demo server? @ryan, do you know if the demo machine has a scheduled process for reset?
To address @surangak comment - @ryan, should we rename the qa01 machine to qa03 instead? Or do you have a better idea?
Also @ryan, do you think we should have the new machines as ubuntu 14? Or would it be too hard?
(Aside: I think it is a bit better if we always deploy to both int and qa, regardless of whether the int tests pass. But this is minor, and tangential to the main part of the work, so I’m fine leaving things as-is, i.e. “if they pass, …”)
Are we going as int01/int02/int0X or int-platform/int-refapp/int-extras-01? Both are good for me. I suppose we can start with the numbers and move it later. @ryan has probably some opinion how hard it would be.
My personal vote is for int-platform/int-refapp. I don’t see why we would force people to learn “01” and “02” for fixed-purpose machines. But I’ll defer to @ryan’s opinion on this.
If dev machines are not going to be deployed from CI, how developers are planning on using them? Do they need a key, or what?
I presume they will be given to devs on an as-requested basis, with some key-based process that exists, but is independent of the renaming we’re discussing here.
@darius If the platform won’t have a UI, would it make sense to deploy to int01/int-platform to run the REST-based test? Do we have those tests now?
There is little real benefit to having all of int/qa/uat for the platform, but I think we should have a parallel structure for each distro anyway. We should write a couple of sentinel black-box tests of the REST functionality, but these tests don’t exist yet.
uat should be deployed manually or automatically? I believe @darius and @burke disagree here.
UAT should be manual. I’m pretty sure that @burke and I agree on this.
Can I make deployment to demo automatic after a release of the platform? If the platform won’t have an UI, will it still be useful for a demo server? @ryan, do you know if the demo machine has a scheduled process for reset?
Demo should have the reference application not the platform. Having a release of the refapp distro automatically deploy to the demo server is a nice-to-have, but since it only happens once every 6 months, I consider it a very low priority to automate.
This is correct. All non-production test machines are created upon request and assigned to the requester for the purpose stated in the request. They do not “exist” in our standard production server maintenance & change management processes. Details at: Jira Service Management
The system at demo.openmrs.org is an exception because it is maintained as a public/advertising type of resource and as such it follows our standard change management processes, access control, monitoring, etc.
I agree that “int-platform/qa-platform/uat-platform” and “int-refapp/qa-refapp/uat-refapp” are preferable since it makes these longstanding resources more intuitive (easier to guess & remember).
Happy for infrastructure (@ryan) to paint this shed. … but, if we make these aliases now, then @cintiadr could script against them now and migrating the actual targets (e.g., moving the int-platform alias from devtest01 to int01) could happen independently.
I’d hate to have to figure this out all over again…
… but, if we make these aliases now, then