Jira Email Notifications should be Fixed

Development
1

Many of us were increasingly noticing that tagging someone in Jira / OpenMRS Issues was not sending automated email notifications. Thank you @mseaton @samuel34 @dkigen @vasharma05 for escalating this!

This should be fixed now.

Today we discovered that the reason members had stopped receiving Jira notifications was that our SSO email domain service went down (for id.openmrs.org), and no one noticed. Restarting the service seems to have fixed this. (Thank you @ibacher!)

Preventing this in future

We’ve asked @scott to set up an automated alert for this service, that feeds into the #infra-incidents channel, and ideally also status.openmrs.org.

What if it’s not fixed for me?

If you notice that you are still not seeing Jira notifications at your expected email, please let us know in this thread! That could mean there are other problems involved.

2

@ibacher was it postfix that went down? All services have health checks so should be auto-restarted when having issues… unless the health check is not right.

3

It wasn’t so much that postfix went down as that the container had somehow been removed altogether, even though the Keycloak and Postgres containers remained running.

4

Interesting… it may have had something to do with running out of disk space on the machine.