I’d like to start a discussion on the OCL tech stack. I’m writing it from a perspective of a person who stays in the JVM world for most of my development, so I may be wrong in many places and I hope to be corrected.
First off all OCL API uses django-nonrel-1.5. It’s a fork of django 1.5 with an added support for NoSQL DBs including MongoDB. That version of django is quite outdated (released 3 years ago) and is no longer supported by django. It is even hard to find documentation for django 1.5. The oldest version they provide docs for seems to be 1.7 and even for that version they put a red warning that it is an unsecure and unsupported version. There is a branch of django-nonrel-1.7, but I couldn’t find any info, if it is stable and the last commit was in 2014, which would suggest not to use it.
Django seems to be best suited to work with relational databases, thus even in django 1.9 there’s no built-in support for MongoDB. People seem to use external libraries like pymongo to connect to MongoDB when working with django, but continue to use a relational database as the main data source.
I’m not really sure what to do with that knowledge at the moment. I’m hoping I’m missing something and there’s a way out. I’m looking for some advice from people who have experience with Python on how to best deal with that. @kkaczmarczyk, what would you recommend?
Another thing is that we have a separate project (in django 1.6.2), which is purely for providing UI. It doesn’t seem to be very complete (many pages do not work) so I think that it may be best going forward to drop it and develop UI, which communicates with our REST API using some front-end js frameworks like Angular or React.
I agree with this ↑. I’m not really sure what to suggest for the API, but I strongly suggest going through the pain of moving off a dying platform sooner rather than later . I’m not sure if this means upgrading to the latest Django or doing something more extreme though.
I also think it would make sense for the UI to be done in ReactJS, AngularJS, or similar, without any python.
I don’t know anything about Django, but some quick googling brings up MongoEngine (last commit 18 days ago) and this presentation about it.
In the context of a planned significant investment in OCL, one should consider all options (e.g. porting it to a different framework, or language, even), if there are some advantages to it. But…is there a driver for being concerned about Django at this point? (Getting onto a currently-supported non-forked version of Django makes sense, for sure.)
Thanks for this, and I am sure @paynejd had some rationale for doing so. Remember that OCL needs to support not only OpenMRS, but anyone, so although we want to have a close integration, we can’t be OpenMRS-specific.
When OCL development started, Django 1.5 was the latest and greatest and unfortunately the REST API modules did not natively support hierarchical resources, hence the requirement to fork it. Basically, the module natively support /orgs/org_id but not /orgs/org_id/sources/source_id/. Based on the API specifications that we all helped develop, the hierarchical resources was important.
Django has native support for relational databases, but it was thought that a document-based data model (i.e. NoSql database) would better suit the project. Given the learning curve required for MongoDB, I’m not sure we made the right decision on this one.
Haystack is the module used to synchronize MongoDB with Solr. My original developer determined that the link Haystack connector for MongoDB and Solr was mature, but my experience is that it has had extremely low performance, which is why importing the CIEL dictionary takes nearly a week, when it should take far less.
The consensus among this group has been that we wanted to make small improvements with the existing code base to get Collections working so that we could support the basic OpenMRS use case, while considering what the most appropriate long-term architecture and tech stack are. If it turns out that adding collections support is a big investment, then we should prioritize having the architecture discussion soon.
Thanks again for your work on this, Rafal. Let me know if there are other questions.
Hey Rafal,
I have some previous experience with CouchDB Django project, where just
with usage of CouchDBkit framework application is production-ready. As a
main data source (django core, authentication, etc.) it is still using
usual rdbms suspect. If document-oriented approach is needed, I think
that should be considered as best approach in Django. That said, CouchDB
could have better external support than other non-relational DBs.
I am not familiar with django-nonrel fork, and am not sure which parts
of the fork is needed/used in OCL project (apart from ORM/db engine,
potentially some django admin view capabilities?), but usually with help
of external frameworks/libraries mainstream Django can be used. There is
something called MongoEngine http://mongoengine.org/, unrelated with
django-nonrel, which might be worth a look. Also, Django 1.5 seems quite
dated by now, and using the latest supported Django version is (usually)
a good decision, since you could be missing new features.
I would agree with sentence that Django is best suited with relational
databases, though you can have Django working with anything with some
effort involved. The question is how much time/cost-efficient effort you
are able to put into it.
. I’m not sure if this means upgrading to the latest Django or doing something