@darius , in a previous thread (see below, pre-openmrs.talk), you had discussed throwing rest representations of encounters + obs into elasticsearch. We are in the design process of moving data into elasticsearch at this point primarily for analytics purposes. However, in the future we’ll be working on an offline version that I’d like to utilize pouchdb to support. Ideally, we’d come up with a single nosql representation that we could use across nosql dbs (perhaps with some minor configuration changes).
Presently, as you mention, the representation conflicts with how elasticsearch wants one to do things. My preference to solving the “value” problem mentioned below would be to have obs represented identical to the mysql table. That is, we do not reduce the value to a single “value” field, but keep all the “valueNumeric, valueBoolean” etc as fields in the document. On the elasticsearch side, we could then define our mapping properly to handle the different data types and thus conduct queries properly.
@darius, did you make any decision on how to do this? Ideally we would change the rest ws module to reflect these changes. I suspect that anyone using rest may have already build models using the current form. It would be import to find a way of supporting both. I’m think, keep the current representation (and deprecate later) and add in this new representation.
Would love to hear the community’s thoughts/experiences with this.
From Feb 20, 2015
Related to this thread, I’ve been playing around with putting OpenMRS REST representations verbatim in ElasticSearch (ES). The hope is to be able to simultaneously store OpenMRS data so that it might be imported into another OpenMRS server (e.g. for patient transfers), while also taking advantage of some of the cool search features that ES can do.
ES is built on Lucene, and they pitch the idea that “Toss it a JSON document and it will try to detect the data structure, index the data and make it searchable.”
However I hit a problem related to the way we do our REST representation of Obs: because we have a “value” property whose datatype may vary, ES chokes (basically it infers the datatype from the first obs you put in, and fails when you try to put in an obs of a different datatype). I assume the underlying issue would hold for anything based on Lucene, but I haven’t verified that.
This is easy enough to work around, but I’m wondering if it’s telling us something, and that we’d be better off with our Obs representation having specific fields like valueNumeric, valueText, etc.
Anyone have thoughts on this?