I’m curious about to know more about OpenMRS Infrastructure and Helpdesk. Maybe @elliott, @michael, @ryan, … , can help
Some questions I have in my mind :
Which tools and environment do you use ?
What/where are physically server - as daschang : cloud, vm, …
Do you use vagrant, or devops tools ?
What is the wok-flow between stagged/prod ? What is done for deploying or updating ID dashboard ?
Anyone can help helpdesk ?
…
Sorry if this topic is duplicated. I read some wiki about OpenMRS Infra, but I don’t find any answer, and it’s probably better to chat !
Moreover, If it hasn’t been done yet, It would be an interesting University subject.
As you’ve probably noticed, the OpenMRS community infrastructure team hosts a large number of hosted applications and resources for use of the community. These run in several different hosting environments for stability, and are located in a combination of several locations In the United States, Bloomington, Indiana; Indianapolis, Indiana; El Segundo, California; and Corvallis, Oregon. We also have resources hosted in Amsterdam, Netherlands.
Most of our servers run Ubuntu Linux server distributions and we always prefer to run the LTS (long term support) versions. There are a few exceptions where necessary. Our team regularly applies security patches to servers as they are released every week, and all of our servers are behind firewalls for security reasons. Additionally, for the most part, all of our servers have at least 1 set of “staging” servers to test major changes before they are rolled out. This helps prevent unexpected downtime our outages.
In the last year or so, we’ve begun to use several dev ops tools to help manage deployments of software, and I’ll let @ryan reply about some of those tools and how we’re planning to use them in our long-term strategy.
The OpenMRS Community Help Desk at https://help.openmrs.org/ is powered by Desk.com (again, for redundancy in case all of our other resources are unavailable) and can be accessed by anyone with an OpenMRS ID through the web, or by anyone via e-mail at helpdesk at openmrs dot org. Desk.com can also be used to manage our social media channels and interactions.
The Community Infrastructure Team has 24x7 on-call support in case of urgent issues, and is always looking for infrastructure volunteers! For more information about the team and how to help, people should head to: https://wiki.openmrs.org/display/RES/Community+Infrastructure+Team
Thanks for giving lots of details, and taking time to write this answer
It’s interesting, and reading your message, I’m a bit impressed how OpenMRS takes care of quality, availability and reliability of its infra and services.
(I remember how quickly OpenMRS reacted few months ago, about Heartbleed case)
I should also add that we monitor our services 24x7 with tools like New Relic, Pingdom, and PagerDuty, all which integrate with our help desk software.
We currently use Ansible and Puppet for software deployment and configuration management. Ansible is used for more ad-hoc stuff such as installing a security patch across all prod and staging servers. It came in handy when we reissued our SSL certs after the heartbleed vulnerability, This allowed us to deploy our new certs and restart the affected services, for example.
Puppet is currently being used by our bamboo build servers. We have a set of puppet modules that will configure a server to create a set number of bamboo agents, connect back to our main bamboo server and be ready to build any software on ci.openmrs.org. We plan on using puppet for more tasks like these and also consolidating our configuration management across servers.
I’m glad you are interested and If you have any more questions feel free to ask!
Your answer is really , and well detailed.
I don’t really know Ansible, but as you gave pointers, I will be able to quickly learn more about it.
What do you think of providing vagrant file (+ puppet for provisioning) for developers working on OpenMRS applications (as ID, Modulus, Atlas) ?
So it will be easier to test locally with the same environment than production, and have a database, server, well configured ?
We have a vagrant + puppet provisioner for the openmrs reference application which helps set up a dev environment for working with the reference application. I also made one for testing out how openmrs behaves with multiple instances pointing to the same database here. I think it is a fine way to help developers set up an environment quickly without having to worry about what platform/OS version they are on. As for deploying applications we are looking more heavily into Docker since it has hit 1.0 recently. We currently are using docker for talk.openmrs.org.
If you’re interested in reading more about our systems hosted at Indiana University, check out https://www.xsede.org/web/guest/iu-quarry (includes photos!) to read more about Quarry which runs our resources there. Of note:
Total available storage: 880 TB
Total nodes available: 140 x IBM HS21 Blade servers, 230 x IBM iDataPlex dx340 rack-mounted servers = 370 nodes
Total available memory: 370 nodes x 16 GB RAM = 5.8 TB
Of course these are specs for the entire system which others use as well. But that’s how much is available in total.
Details on Data Capacitor 2, our storage system there, is available at: https://kb.iu.edu/d/avvh
…
