Metadata sharing nolonger on the mdsbuilder

Tags: #<Tag:0x00007f0773c4efd8> #<Tag:0x00007f0773c4ed80> #<Tag:0x00007f0773c4ea60> #<Tag:0x00007f0773c4e290>

@akanter super use knowledge is all you need … :relaxed:

@taremwatadeo, I’ve updated concepts in referencemetadata to the latest CIEL and released the referencemetadata module. You can proceed with RA 2.5 release.

1 Like

Thank you @raff

Per this post: Clarification regarding update Metadata Sharing packages to the latest versions of CIEL concepts

… it looks like the MDS packages may have been lost again on this server?

@mogoodrich , it seems like this is indeed the case.

Since I have ssh access to this server, if anyone ( @cintiadr maybe?) could point me to the instructions on restoring the packages, I can go ahead and do the needful(especially since I am currently blocked due to this and we are already out of time with reference to the Ref App release) :slight_smile:

cc : @dkayiwa @darius @raff @burke

Not that I have any idea what is those packages you are talking about, or how they are included.

That said, you shouldn’t run anything other than SQL imports on that machine. It will be lost and rebuilt.

If you give me the instructions (when you find it), send me the link so we can make sure that’s not lost when I restart the containers, run ansible or recreate the machine.

I’m not 100% up to speed on what is happening here, so I’m hesitant to weigh in, but I’ll try… :slight_smile:

The MDS packages are bundled groups of concept and other OpenMRS metadata created by the Metadata Sharing module.

These would be stored in the metadatasharing_exported_package table in the OpenMRS database.

From reading earlier in the thread I believe mdsbuilder is meant to be “place of truth” where we store these packages, which are then bundled with the reference application.

Not sure how the mdsbuilder server is managed, or if this was ever properly documented/conveyed to the infrastructure team, but because of this, when we recreate the machine it’s important to not lose the existing database. Not sure if this is what happened?

Take care, Mark

Don’t blame you for not knowing @cintiadr and @mogoodrich :slight_smile: . This server is like a black box and I haven’t found much documentation about it on wiki either. I just know that the current output is not what is expected but I have no clue on how to fix it.

I know a lot more about the server (not really the OpenMRS service).

So, here you can find the docker compose and its initial database. You can see the only things we are keeping as ‘data’/volumes are the database folder and .OpenMRS folder. There are backups for those folders.

I documented this image in wiki

The ticket that did the migration to the new server is ITSM-4054.

I don’t really understand what was migrated (database-wise), because I don’t really understand how the service is being used at all.

I’ve moved all packages to the new mdsbuilder server when migrating the server… I even recall us creating packages for a Reference Application release from that server a few months back. I don’t know when/how those packages were deleted. We should have backups of the database taken daily so it’s a matter of tracking down, which one to recover. I can look into that before the end of this week.

thanks @raff!

Unfortunately the oldest backup we have from May 31, 2018 does not contain any packages… We must have lost packages prior to that.

The one way to recover I can think of right now is import RefApp metadata from the latest release of Reference Metadata module into mdsbuilder… I can try to do that tomorrow. @burke or @dkayiwa could you please create an account for me on https://mdsbuilder.openmrs.org/openmrs/ ? I don’t have one anymore.

It’s the second time we lost data on mdsbuilder. This time backups didn’t help as it was spotted too late (we keep backups only for a certain amount of time). It would be good to have some checks in place to discover such a loss earlier. At least we should keep a backup taken with the last release of RA and not expire it.

1 Like

Meet me on skype, Rafal. Happy to set up a user account for you.

That’s unfortunate. Nonetheless, I have messaged you with the admin details which I got from @cintiadr for the server.

If you add to the manual backups S3 bucket, it’s not going to be deleted.

But I domino understand how that happened.

Great news! I found a backup on my machine from 20171222 (when I did the migration). All packages have been restored. I also upgraded mdsbuilder to the latest CIEL (20181012). @reubenv, please proceed with the release from the 2nd point of Release Steps.

@cintiadr, are you saying that S3 backups shouldn’t have been deleted? The last backup I see for mdsbuilder on S3 is from May…

Thanks a ton, @raff ! This made my day(technically night here :stuck_out_tongue:), I can now move ahead with the release!

Sort of. Bucket openmrs-manual-backup doesn’t have a lifecycle. Whatever you put in there, should be there forever (or until someones deletes it).

Bucket openmrs-backups is for automated backups, and indeed, it’s kept for 6 months.

All clear, thanks! Let me push at least one backup to openmrs-manual-backup then!

In case you are wondering how you could have known that (without digging AWS console), they are created via terraform: https://github.com/openmrs/openmrs-contrib-itsm-terraform/blob/master/base-network/backup.tf

It’s also described in https://github.com/openmrs/openmrs-contrib-itsmresources/wiki/Backups-Strategy