Import of concepts from CIEL dictionary fails with Duplicate name error

Software Bahmni
, , ,
1

We are trying to setup the metadata for Bahmni using the latest CIEL dictionary with version v2024-06-27. When we try to initialize the CIEL zip with initializer module, there are about 60 concepts for which import error is thrown and the concepts are not imported. The error says org.openmrs.api.DuplicateConceptNameException: 'Toxemia con convulsiones' is a duplicate name in locale 'es' for the same concept. Attached a detailed error log. ciel-import-error.txt (5.9 KB)

The list of concepts which fail:

83724AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
79911AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
159968AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
159104AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
158756AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
158686AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
158615AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
158413AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
158676AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
158464AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
159352AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
158652AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
158650AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
158449AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
158362AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
158357AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
158380AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
158077AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
157535AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
158367AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
157387AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
157991AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
157781AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
157837AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
158025AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
157477AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
157404AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
156551AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
156479AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
156816AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
156595AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
156733AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
156247AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
156519AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
156244AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
156790AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
156721AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
156510AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
156783AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
156709AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
117939AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
155654AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
156114AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
156111AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
155213AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
155556AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
155210AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
156095AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
155198AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
155876AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
155941AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
155110AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
155666AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
110818AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
156168AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
156167AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
156166AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
156110AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA
154934AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA

@akanter Can you please assist on this ?

cc. @mksd @dkayiwa @akhilmalhotra @angshuonline

Thanks

2

You must be loading this onto a database that somehow already holds those concepts.

It’s not really solving the actual issue but a workaround would be to retire them first through a CSV file that is loaded before the CIEL bunch. This can be achieved by setting the _order: column’s value to something lower than that of the CIEL CSV file.

Assuming 10 is lower than the CIEL CSV file’s order, something like this:

Uuid Void/Retire _order:10
83724AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA TRUE
79911AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA TRUE
159968AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA TRUE
159104AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA TRUE
158756AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA TRUE
158686AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA TRUE
158615AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA TRUE
158413AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA TRUE
158676AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA TRUE
158464AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA TRUE
159352AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA TRUE
158652AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA TRUE
158650AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA TRUE
158449AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA TRUE
158362AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA TRUE
158357AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA TRUE
158380AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA TRUE
158077AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA TRUE
157535AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA TRUE
158367AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA TRUE
157387AAAAAAAAAAAAAAAAAAAAAAAAAAAAAA TRUE
3

I am just starting with a fresh MySQL 8.0 database and these concepts are not added without CIEL zip.

4

What are you actually doing there? How are you doing this?

5

We have added the CIEL zip file inside configuration/ocl folder and letting the initializer import the dictionary

6

Did you validate that the trick mentioned above doesn’t help? I.e. retiring those concepts beforehand.

7

Nope, that doesn’t help because after CIEL import only, these concepts and UUIDs become available

8

There was a different validation requirement associated with the unique names within a locale which was discussed recently (I think). Which version of CIEL are you using? I thought that the last couple of releases resolved this issue based on the validation schema. I could be wrong, though… @burke can you confirm?

9

I am using the latest released version v2024-06-27 .

10

FYI, the CIEL concept 83724 is retired, so something might be wrong that the validation is including retired concepts when it should not.

11

Can you check for other concepts please, there are unretired ones which causes the issue as well. Example: 158413 fails for pĂłlipo de la cavidad nasal

image
image1330Ă—1000 94.5 KB

12

Whoa, that’s a duplicate within the same concept which is case-dependent! I’ll check where else there are only case differences as this might be the issue rather than shared names ACROSS different concepts

13

The list i had shared above have same issue, where there is a duplicate name within the same concept and are mostly on the es locale.

14

Not sure why 159968 is failing as there are no concept names in spanish.

15

I think I understand why this happened. They are concepts which were duplicated SNOMED SAME-AS concepts. I am cleaning them up and they will be resolved in the next release. Since they are almost all concepts which already exist with a different name, the fact that they weren’t imported is probably OK. However, the dupes will be retired and the duplicate names removed.

16

Work done. Will be in next release this week.

1 Like
17

Thanks @akanter.