Best Practices/Experiences for Data Import - Mapping of CSV columns to concepts

We are working on importing data from the CSV file below into OpenMRS where each line is an encounter:

Phone Follow up date Type of Care Follow Up Type ART Number EID Number
25677000000 2018-03-14T21:59:21.050552+03:00 EID SMS Message EXP-10008
25677000001 2018-03-14T21:59:21.050552+03:00 ART & ANC SMS Message ART-FEMALE-1

I am looking for for guidance on the following:

  1. Mapping of the data in columns to concepts.
    • Phone and Followup date are okay since each has one value per obs
    • Type of Care - may have one or two or more coded values. Is there any other choice other than IF-Else statements
  2. The last 2 columns are identifiers - ART Number, EID Number is it possible to search across 3 different identifiers at the same time?

This file may contain anywhere from one to 100 encounter records.

1 Like

@dkayiwa the code is currently in this file https://github.com/ssmusoke/openmrs-module-aijar/blob/emtct_module/omod/src/main/java/org/openmrs/module/aijar/page/controller/EmtctDataImportPageController.java#L74-L89

My understanding is that you would need to create the appropriate concepts for (1)

Then (2) would be stored as patient identifiers whose identifier types you need to first create.

@dkayiwa I am looking for an example where an Obs is created and the appropriate value added as part of manual building of the encounters.

Do I save all the completed encounters at once, or one at a time? How do I ensure that the transaction rolls back in case of any errors/failures?

To reduce the complexity, for the first pass, i would not worry about transactions. I would just write code that saves data assuming it is correctly formatted and hence passes all validations. After that is done and confirmed to work, i would then look into the transactional issues.

@dkayiwa noted how about this question below

@ssmusoke- It would probably be easier if you could find a way to create encounters first then add observations later. You can add another column to the file, with generated UUIDs and each encounter would be saved with the corresponding UUID. The added column would also be a way of tagging this encounters so you can recognize them easily. It will also help you to map observations to their corresponding encounters later when you get to creating observations. …and because you will have that mapping, you can decide to create observations one column at a time.

I assume the following would work:

 Encounter existingEncounter = Context.getEncounterService().getEncounterByUuid("some uuid");
			    Obs o = new Obs();
				o.setConcept(Context.getConceptService().getConcept("The concept in the current column"));
				o.setValueText("valueText"); //or o.setValueCoded(Context.getConceptService().getConcept(someConcept)); or o.setValueDate(valueDate); or o.setValueText(valueText);
				o.setEncounter(existingEncounter);
				existingEncounter.addObs(o);
				Context.getEncounterService().saveEncounter(existingEncounter);

On saving encounters, I would prefer to save them all at once. …having @Transactional annotation on the saving method in your service implementation would let Spring start and commit/rollback transactions for you.

1 Like