Understanding OCL-API and getting it work
As per openconcept lab web site ‘OCL provides access to major terminologies, such as the Columbia University eHealth Laboratory’s interface terminology’. As students of PES University studying IAD’s (iad.org.in) integrated approach to health care, we wanted to understand what OCL provided and what we could use and collaborate. In this we explored the OCL (Open Concept Lab) oclapi project which can be found on github at https://github.com/OpenConceptLab/oclapi.
Oclapi is basically a django rest framework api which can be used to access information about various organization for health like CIEL, WHO etc and many other sources under these organization. It also gives information about the various concepts, which can be understood as coded terms which are used for taking electronic medical data.
We document below the steps we followed, some issues we faced and how we resolved them. We followed the steps given in the README from – Manual Environment Setup (on a Mac) – repeated here with the changes that we incorporated along the way.
Step 1 ( Installation ) -
install solr using brew installer (if you are unfamiliar with brew please google it) i.e. brew install solr install mongodb with brew installer as well Create virtual environment using python virtualenv ( recommended version - 2.7) virtualenv -p Inside the virtual environment - install django nonrel latest version (can be found using a simple google search) cd into django/ocl folder (This is in the project you would have cloned in your system) Run pip install -r requirements.txt The installation will not run correctly the first time because the first line in the requirements.txt file is basically to install the django nonrel project but the link provided in the text file is pointing to 404 error. So you can comment that line using # and then download the django-nonrel project from github and then install it under this virtualenv.
Step 2 ( Executions / Edits ) -
Activate your virtualenv you have created in step 1. cd into django/ocl folder and run - ./manage.py syncdb ( create a superuser if not prompted use - ./manage.py createsuperuser ) ./manage.py runserver ( to see if everything is working fine ) Next to make sure the api works fine we have to make changes to the django oclapi project also required for import Change models.py in oclapi and create the customform.py and custommodels.py in the same folder Note - Overriding the ListFields etc from djangotoolbox because there is no widget to view ListFields and DictField by default for the django admin site.
———————————————————————————— In file models.py …. make following changes ———————————————————————————— … … … from django.core.validators import RegexValidator from django.db import models from django.db.models.signals import post_save, pre_save from django.dispatch import receiver from djangotoolbox.fields import DictField, ListField
The line below is the edit
from custommodel import DictField, ListField from rest_framework.authtoken.models import Token rom oclapi.utils import reverse_resource, reverse_resource_version from oclapi.settings.common import Common from django.db.models import get_model … … …
———————————————————————————— Code for customform.py to be placed in oclapi folder ———————————————————————————— import ast
from django import forms from djangotoolbox.fields import DictField
class StringListField(forms.CharField): def prepare_value(self, value): if(value==None): value="" return ', '.join(value)
def to_python(self, value):
if not value:
return []
return [item.strip() for item in value.split(',')]
class DictionaryField(forms.CharField): def generate_dictfield(self, field_name, field, **kwargs): defaults = { ‘required’: field.required, ‘initial’: field.default, ‘label’: self.get_field_label(field_name, field), ‘help_text’: self.get_field_help_text(field), } defaults.update(kwargs) return DictField(**defaults)
def to_python(self, value):
if isinstance(value, basestring):
value = ast.literal_eval(value)
return value
if isinstance(value, dict):
return value
raise ValueError('"%s" could not be converted to a dict.' % value)
———————————————————————————— Code for customodel.py to be placed in oclapi folder ———————————————————————————— from djangotoolbox.fields import ListField, DictField from django.db import models from .customform import StringListField, DictionaryField
class ListField(ListField): def formfield(self, **kwargs): return models.Field.formfield(self, StringListField, **kwargs)
class DictField(DictField): def formfield(self, **kwargs): return models.Field.formfield(self, DictionaryField, **kwargs)
————————————————————————————— Edit the file sources/models.py ( add at the end so that we can edit using the api available after this edit when running project i.e. localhost:8080/admin sources and source versions will now be an available option)
Code for edits to source/models.py file add at end of file
—————————————————————————————— admin.site.register(Source) admin.site.register(SourceVersion) ——————————————————————————————
Edit the file mappings/models.py file If the error is for importing the module then we can use imp module of python for giving absolute path of the module. make sure to change path correctly to your file in the edits given below
——————————————————————————————
edited lines below
import imp from django.contrib.contenttypes.models import ContentType from django.core.exceptions import ValidationError from django.db import models from django.db.models import get_model from django.db.models.signals import post_save from django.dispatch import receiver from mappings.mixins import MappingValidationMixin from oclapi.models import BaseModel, ACCESS_TYPE_EDIT, ACCESS_TYPE_VIEW, ResourceVersionModel from sources.models import Source, SourceVersion
MAPPING_RESOURCE_TYPE = ‘Mapping’ MAPPING_VERSION_RESOURCE_TYPE = ‘MappingVersion’
modify the path below different from user to user
concept = imp.load_source(‘concepts.models’, ‘’)
class Mapping(MappingValidationMixin, BaseModel): parent = models.ForeignKey(Source, related_name=‘mappings_from’) map_type = models.TextField() from_concept = models.ForeignKey(concept.Concept, related_name=‘mappings_from’) to_concept = models.ForeignKey(concept.Concept, null=True, blank=True, related_name=‘mappings_to’, db_index=False) to_source = models.ForeignKey(Source, null=True, blank=True, … … …
—————————————————————————————— Edit the lines in mapping/importer.py This is because if it finds an exception while importing mapping then it basically stops importing other mappings. So this will ensure that even if there is error while importing mapping that is concept doesnot exist or mapping to that particular source doesnot exist. In this case it will still continue importing other mappings from file and will write the mappings which failed that can be used later after creating that particular concepts or creating that particular source if it doesnot exist. —————————————————————————————— …… …… …… except InvalidStateException as exc: str_log = ‘Source is in an invalid state!\n%s\n%s\n’ % (exc.args[0], data) self.stderr.write(str_log) logger.warning(str_log) self.count_action(ImportActionHelper.IMPORT_ACTION_SKIP) # lines below edited except Exception as e: str_log = ‘Mapping does not Exist !\n%s\n%s\n’ % (e, data) # Enter your own path here will vary from user to user with open(‘path to any file to track the log’) as fp: fp.write(str_log) self.stderr.write(str_log) logger.warning(str_log) self.count_action(ImportActionHelper.IMPORT_ACTION_SKIP)
# Simple progress bars
if (self.count % 10) == 0: str_log = ImportActionHelper.get_progress_descriptor( ‘mappings’, self.count, total, self.action_count) self.stdout.write(str_log, ending=’\r’) self.stdout.flush() if (self.count % 1000) == 0: logger.info(str_log) … … …
——————————————————————————————
Step 3 ( Sources Edit ) - Create sources for the imports from the json file in ocl/demo-data/ folder -
Now we enter the api localhost:8080/admin we had registered sources and source versions earlier now we can just create a new source version called HEAD. required for import to work.
Start with - create a source CIEL remember (source-id can be to be used later)
Create organizations list like image given below required for import -
run the below command to start import - ./manage.py import_concepts_to_source --source --token <source_id> <path_to_concepts.json_file> Token is the user token which is using the api. For more information refer the documentation for importing concepts given in the link mentioned at the top. all the concepts should now be stored in mongo database can be searched using the api import complete.
Credits - (CCBD Team) Ashok N Raman Rishabh Dutt Reuben Cherian