OpenMRS infrastructure is back!

[burke (Burke Mamlin)]

As you might have noticed, the OpenMRS community infrastructure (including OpenMRS Talk, dev sites, website, shortlinks ,etc.) were offline this week.

All OpenMRS community systems should now be back up and working.

@Background

The OpenMRS infrastructure is the supported by a generous donation of resources from Indiana University’s High-Performance Computing Jetstream2 project. This is an academic-led cloud environment that leverages open source solutions like OpenStack to provide programmable cloud resources that allow infrastructure as code.

What started as a planned maintenance window turned into a system administrator’s nightmare when network issues occurred and an undocumented bug was encountered in the underlying distributed file storage solution. The network issues were quickly addressed; however, the file storage bug required the Jetstream2 team to submit a pull request to patch the bug, build a version of the file system with the patch applied, and then restart the cloud environment. After two sleepless nights and working closely with their vendor, the Jetstream2 team got us back and going today.

On the bright side, not only was the storage systsem bug documented & fixed, the Jetstream2 cloud received multiple maintenance fixes, and critical hardware (CPUs and cooling systems) were migrated to an uninterruptible power supply (UPS)/generator to reduce the risk of future downtime.

[ball (Ellen Ball)]

My appreciation to the OpenMRS Infrastructure team. They are unsung heroines and heroes of this community effort!

Ellen Partners In Health

[wodpachua (Olara Moses)]

Indeed! @ball couldn’t agree less…