Ethiohri: Data migration plan

Hi @ICAPEthiopia & @OHRI, how are things going w.r.t. the Smartcare ET → OpenMRS DB data migration planning? @burke and I were trying to remember if you needed any additional resources or support from us on this topic at this moment.

@burke & @dkayiwa are there any data migration resources we should share here? (For a non-OMRS system to OMRS) @jennifer are you able to share the migration resources you collected when we were supporting the Rwanda platform upgrade?

(ICYMI: We are no longer planning to use the FHIR API for initial data migration, which was our original idea as mentioned here.)

CC @dagimm @eudson @mwaririm

2 Likes

https://wiki.openmrs.org/display/docs/Migrating+Data+to+OpenMRS

https://wiki.openmrs.org/display/docs/Data+Migration

1 Like

Hi @grace we decided that data migration should be out of scope for the July MVP. But it is something that must be done in the near future.

Here’s the link to the RwandaEMR Upgrade Toolkit. It includes a section listing out various resources: tools, guides, etc that other OpenMRS implementations have used.

1 Like

@grace, perhaps we could arrange a hands-on meeting with devs from @ICAPEthiopia & @OHRI along with @dkayiwa (and/or @ibacher if avail) to work through the data mapping for migration? @paul mentioned that he would be eager to help and to attend such a meeting.

I’m imagining we could help match existing data to the OpenMRS data model similar to what we started in the “OpenMRS Model” worksheet of the gap analysis spreadsheet or, if the Ethiopia team has already begun that mapping process (which ideally would map each object & its properties to OpenMRS equivalents).

1 Like

@dagimm what do you think? Do you want to get started on this work now? i.e. Are you interested in a hands-on meeting like Burke describes, and if so, should we set up a hands-on meeting now or later, like in Aug?

IIRC I think there was a team member you were considering allocating to this work in general … or I might be getting mixed up w/ reporting.

CCing @eudson as an FYI.

1 Like

@grace we can start working on this starting next week. I will be working on it.

2 Likes

Sounds great @dagimm. Turns out there’s a lot of people who are offering to help by sharing their own similar experiences as well (w/ non-OMRS to OMRS system migration). But the first session can be a focused hands-on technical one with just a few people.

Do any of these 3 times on Thursday work for you? (And also @dkayiwa and @eudson if you could please fill this out too. @ibacher if you could possibly join that would be cool too.)

3 time options that work for Burke here: :point_down: :point_down: :point_down:

Later, we could have a second session, and invite additional engineers who have gone through similar experiences before. Some examples brought to my attention today are:

  • Haiti: ITECH’s iSanteEMR change to an OpenMRS-based EMR, and that was even a free text system
  • Kenya: Palladium’s massive migration of tons of data from IQ-CARE (non-OMRS) to KenyaEMR (OMRS) and they even published about it - they are a good resource being a recent experience at national level scale
  • Botswana: Intellisoft is actively migrating data in Botswana from a proprietary system called PIMS to OpenMRS-database BotswanaEMR

How does this general plan sound?

1 Like

@grace This sounds great! The proposed times for the discussion works for me. Excited to kick start this activity.

1 Like

Wonderful thank you Dagim!! I’ve sent an invite to you, Burke, Daniel, Paul, Eudson & Ian for Thursday at 6pm EAT.

1 Like

Today we met to start working on this. Attendees: ICAP-ET: @dagimm, Abdi UCSF: @eudson OMRS: @dkayiwa, @grace, @paul, @burke

We confirmed that we are not using the FHIR API for this data migration, we will pursue traditional data migration approach. Though for reference Burke shared this general mapping list of FHIR Resources to the OpenMRS Data Model.

First/Next Steps:

1. First Goal: Clear mappings of datastructures in SmartCare to data structures in OpenMRS.

Link that shows the data we have in SmartCare: Detailed Smart care model to FHIR resource mapping Detailed Smart care model to FHIR resource mapping - Google Sheets

So next, we need something like this spreadsheet that maps the Smartcare model to OpenMRS resources. @burke and @paul to spend 1-2 hrs together and see how much mapping they can get through. (e.g. they estimate the patient resource section will take ~1hr to get through.)

:point_down:

Spreadsheet we will use for this mapping work: Ethiopia Smartcare model to OpenMRS mapping: Ethiopia Smartcare model to OpenMRS mapping - Google Sheets

:point_up_2:

Later, could build an abstract that mirrors the platform we’ll be sending data into, and a module that will send the data there. (Also here are some reference concepts Dagim shared.)

2. Data Moving Strategy

Eg will we use REST API? Or…? @dkayiwa are there any particular tools you recommend to import into OpenMRS? However we did agree that we can decide this better once we see how the mappings are working.

3. Learning from Others

After we’ve given Paul & Burke a chance to get back to us, we will look into setting up a wider community call to learn from other orgs who have done a Non-OMRS to OMRS data migration (as mentioned here).

Update: Paul and Burke are meeting this week to work on this:

Hi there!

Took a little longer than a week :sweat_smile:, but we just met and reviewed the entire data model and am at the point where we should reconvene and talk through some specific questions we have.

We did a fair bit of research, and we think we understand the structure of how SmartCare is capturing data… Maybe sometime in the next week we can reconvene? :slight_smile:

@grace : can you help us find a time?

1 Like

@dagimm so Burke and Paul have reached a point where they have some questions about the SmartCare datamodel - I reached out to their EA and they only have 3 slots available next week. Could any of these work for you? Doodle

Can someone summarize the content migration status for me? What is the current status in mapping to CIEL or other standards? I see from this thread that the current work is mostly focused on the data model, but wanted to be sure I was not missing something… thanks.

Good questions Andy. So back in ~June at the Ethiohri hackathon, @wamz spent the 2 weeks reviewing the concepts needed for Ethiopia’s main 3 HIV forms with ICAP-Ethiopia BA leads @desalegnl & @asresut. He onboarded them to OCL online and CIEL, and encouraged them to map to the OHRI-recommended CIEL concepts as much as possible (i.e. the CIEL concepts included in OHRI’s collections). Where a relevant CIEL concept wasn’t available they either used a custom OHRI-provided concept or made their own. (@wamz is this all correct?)

Relevant Collections/Sources online in OCL:

@wamz @desalegnl @asresut where are things at now - I imagine there’s quite a bit of concept work still to do?

FYI @paul - Dagim and I have been following up with Morgan to find a time that will work for you, Burke, and Dagim together. So expect a calendar invite for this within the next ~3 weeks (we tried for this week but didn’t work for a number of reasons).

Thanks, @grace that is really helpful. I will see if I can create some CIEL concepts where there are custom ones… but for example, for BMI, why was the CIEL concept not used?

@dagim,

Great talking to you today. Here are links to items we discussed today:

Next steps:

  • You & your team to try creating person & patients in OpenMRS via REST
  • Bring any questions here, on Slack, or via email (whichever works best for you)
  • We’ll plan on meeting again with Paul once schedules permit (e.g., the week after next)
1 Like

@dagimm, I understand you’ve had a meeting or two with @stans. How are things progressing? Have you been able to successfully create persons & patients via REST?