Introducing Open Concept Lab for distributions

Projects Open Concept Lab
1

We’d like to migrate Reference Application (Ref App) to use openconceptlab.com (OCL) as the source of concepts. At the same time we’d like to set the path for managing concepts by other distributions.

Currently, the OCL has the latest CIEL dictionary loaded. It is possible to import all CIEL concepts into Platform using the openconceptlab module and get incremental updates whenever new versions of the dictionary are available.

The OCL groups concepts in SOURCES or COLLECTIONS. Every concept in OCL must be created under a SOURCE. For example CIEL concepts are created under the CIEL SOURCE.

COLLECTIONS, on the other hand, group references to concepts, which exist under different sources. It is currently possible to create a collection with references to concepts without specifying concept versions (the concept will be updated whenever it is updated in the source). In the future OCL plans to add support to include in collections:

  1. references to exact concept versions
  2. references to all concepts from a given source / given version of a source
  3. references to all concepts from a collection / given version of a collection

For Ref App we do not need the entire CIEL dictionary, rather a small subset of it and we do not want to introduce our own concepts. We can rely solely on using a COLLECTION, which has been created for that purpose at http://api.openconceptlab.com/orgs/openmrs/collections/reference-application/ (currently empty due to a bug in OCL preventing us from adding anything)

We will be able to subscribe to that collection the same way as we subscribe to the CIEL source using the openconceptlab module. In Ref App we will subscribe to the exact version of the collection. The export will be triggered before releasing Ref App and it will contain concepts in versions available at the time of export. The initial export file will be distributed with the referenceapplication-metadata module so that no connection to openconceptlab.com will be required.

When we release a new version of Ref App, we will export a new version of the collection from OCL with any new concepts and updates and replace the file in the referenceapplication-metadata module. The openconceptlab module will identify and import all concepts with newer versions.

At any given time you will be able to subscribe to one SOURCE or COLLECTION of concepts using the openconceptlab module. Basically if your distribution wants to rely entirely on the CIEL dictionary, you will subscribe to the CIEL SOURCE. If your distribution wants to add concepts from other sources or needs just a subset of the CIEL SOURCE, then you will subscribe to a COLLECTION created for your distribution.

That said if you start from Ref App, you can change to subscribe to the full CIEL dictionary or any other collection, which contains a subset of concepts required by Ref App. It is particularly important when you try to upgrade to a newer version of Ref App, at which point we will verify that you have all required concepts or the missing concepts can be installed without validation errors.

The workflow implies that the concept management will eventually be done exclusively on the OCL server. Currently, the OCL UI does not let you do much and for Reference Application we will focus on using the REST API. Also the OCL server doesn’t have any validation yet so you may create sources or collections of concepts that include duplicate concepts and they may not import properly into Platform.

There is still a lot of pending work on the OCL server before concept dictionaries can be easily managed from there.

Right now we have a smaller goal for supporting Ref App. Over the next weeks I will work with Jonathan Payne to identify anything missing in OCL for that particular use case.

As always I’m looking for feedback and flaws in the workflow as described.

5 Likes
2

@raff, thanks for this detailed update. I LOVE that this is moving ahead and that we are pushing our concept management for the reference application into OCL and building tools for subscribing and packaging that content. I am extremely eager for us to start doing similar things at PIH.

I had a look also at the openconceptlab module that has been developed and it looks super well done at first glance, so I’m also eager for PIH to start looking into how we can use that as well.

Main questions to follow-up from your comments:

  • Is there any work underway (or projected plan) to address the Collection features you highlight are missing? I’m thinking in particular of Collections that enable us to subscribe to a set of Collections or sources. Also to fix any bugs (like the one you highlighted) that are preventing us from demonstrating a working end-to-end demonstration of this.

  • Although Concepts is clearly the biggest, most valuable, and most painful driver, discussions with Jonathan indicated to me that OCL might also serve as a platform for general metadata management and distribution. For example, having collections of Drugs, Locations, etc. Is there anything fundamental about the design of the openconceptlab module that limits us to using this for Concepts, or is there room here for growth in this arena?

Thanks! Mike

1 Like
3

The OCL server is not actively developed right now. It is waiting for some volunteer or funded devs.

The good news is that 2 devs from Soldevelo are joining us next week and I hope they can help us with switching to use the OCL for the Ref App. Once we reach that first goal we will see, if we can assign them to do more work on OCL.

Regarding your second bullet the openconceptlab module has no such limits and support for other types can be added.

4

is there any update on this work?

5

Hi Terry, ThoughtWorks is working with Vital Wave (and including Rafal, Andy and others) to implement this and other functionality. We should have this done by the end of October, 2016.

1 Like
6

great… that is awesome. thanks for that update.

7

Can you give us a bit more description about the project, so that we know what will be the result of the work being done?

8

The short project description is that we’ll be doing enhancements of OCL so that it can be used to manage the data dictionary for an African country. (@paynejd can maybe say more).

The biggest benefit for the OpenMRS community is that we’ll be implementing the collections feature.

9

Hi all,

I’m excited to announce that Vital Wave has contracted with Thought Works to implement OCL for the Ethiopia Ministry of Health as part of the Gates-funded Data Use Partnership. Ethiopia MOH will use portions of the CIEL dictionary and ICD-10 to represent its national HMIS data and will also use OCL to host its national reporting indicators and data definitions for several health programs.

While OCL has already been used in a limited way to allow OpenMRS to subscribe to the CIEL dictionary, Thought Works will expand on OCL to support what we call collaborative terminology management. As Darius said, the key feature will be to implement collections, which are how OCL handles subsets and values sets. Thought Works is also expanding OCL’s support for indicators and concept authoring, adding better user/group/permissions management, improving performance, and generally preparing the platform for broader, reliable use. Thought Works is actively working on OCL now and we are planning to implement in October 2016. We will simultaneously be planning how to best support users of OpenMRS, which likely means making a subset of functionality freely available to the OpenMRS community (e.g. subscribing to the full CIEL dictionary).

I want to say a big thanks to Rafal and the developers at Soldevelo for their recent work on OCL, who developed the OCL Subscription module, integrated docker, and helped inform our current development effort. Also want to thank the OpenHIE/OpenMRS communities (esp. PIH) who informed the API specifications and use cases.

Finally, I’ll be reaching out to this group shortly to get feedback on the draft collections specification before Thought Works implements. Will look forward to hearing from many of you!

Cheers, Jon

2 Likes
10

@paykasa - is there any update as to how this is progressing and if the timelines remain the same? Thanks for all the great work on this critical task. Steve

11

Hi @sletchford, the work is progressing as planned and we aim to launch publicly in November after the bulk of the development work is completed. I will post an update here in a couple of weeks with more info. Thank you!

12

@paynejd - thanks. Wonderful to hear. Will be a great asset. Steve

13

@paynejd - checking in to see how this key initiative is progressing and what the updated target dates are. Thanks, Steve

14

@paynejd - checking to see what the update is on this key work. Thanks!

15

@paynejd and @darius - great work to you and your teams on the progress on OCL as a tool for managing terminology. Looking forward to the coming upgrades on the OCL subscription module for syncing with released CIEL updates.

@akanter - could you comment on if / how OCL may (or may not) one day play a role for users to propose new concepts / synonyms for future CIEL releases?

Thanks all for making the benefits of clean clinical data at the grassroots an achievable dream for many. Steve

16

That is certainly the idea, Steve. First, we need to monitor and manage the CIEL content on OCL, then we would allow for proposals to be viewed and adjudicated and assigned CIEL concept IDs. Right now, we need to get more people over to using OCL to obtain the content while emailing for new concept proposals and questions.

1 Like
17

@akanterhttps://talk.openmrs.org/users/akanter- thanks! Very helpful update.

18

@raff / @darius / @paynejd,

I was looking for information on status of this project and came across this thread. I see that the Reference Application collection appears to exist in a live version of OCL, and that there is a tagged 2.6 version, which maps to the most recent major release of the refapp. Can you confirm that this was used to package up concepts within the latest refapp release?

What is the general status of OCL as something that organizations could consider trying to migrate to as a means for supporting overall concept management and creation, for packaging up collections within their own distributions, and for subscribing to changes and receiving updates into live systems? Is there significant work remaining for this to be possible? Is it possible today?

Thanks very much, Mike

1 Like
19

@raff / @darius / @paynejd / @akanter - pinging on this. Can you point me in the direction of where to find more information on the status of this work?

Thanks, MIke

20

Hi Mike,

I can give you an update on this – and sorry for the slow reply

We have just finished moving OCL over to OpenMRS servers, which significantly improved stability, and @raff, Maurya, and engineers from Apelon have made some improvements so that organizations like PIH can use it for metadata management. For the specific features you mentioned:

  • OCL is ready to go for management of concepts and mappings – back end functionality is there, though the UI still needs work
  • The repository versioning and export API can be used to package up specific versions of collections for distributions and can be fully automated through the API
  • Subscribing to changes is only somewhat supported – there is no atom feed, but you can use a lastUpdated flag to filter – if you’re wanting to subscribe to an OCL repository using OpenMRS, you may be able to use the OCL Subscription Module, which periodically grabs a full export (not the diff) and keeps your local OMRS up-to-date without overwriting local concepts – will need to follow up with @raff on the status of the subscription module

So, you could start using OCL today if it meets your requirements. The primary use case we are working towards with PEPFAR is actually using OCL as more of a back-end with scripts in OpenHIM automating synchronization of many sets of metadata between DHIS2 and OCL and publishing the metadata on OCL for distribution.

back-end functionality would meet all or most of your current requirements, but that there would be some User Interface work that you’d like to see completed (or that you could contribute to) to improve the concept management experience.

Let me know if you’d like to jump on a call to discuss further or if other questions

Finally, I do not think that OCL was used to package up the concepts for the refapp, but would need to ask @raff

Thanks, Jon

1 Like