No AFAIK, the query must be intelligent enough to detect the newly updated rows since the last execution. All our tables are fortunately audited, we just need to keep track of the last execution.
I don’t know, probably immediate (near real time, because EL indexes asynchronously).
You can specify a cron expression in the CONF file. If you don’t like polling, another option is whenever an entity is inserted/updated/deleted write the event in a JMS queue and implement the synchronization with EL there. But in this case LogStash would be useless. In the end if you want higher control (Logstash has many limitations) you’ll end up with a custom synchronization process.
But it’s going to be slower than you’d like to get actual deidentified data from an implementation that has expressed interest. I recommend that @raff follows up on that lead, and @lluismf goes ahead and generates some random diagnoses.
I would not approach it this way.
So, the demo data that you’re using is based off an old OpenMRS approach and concept dictionary, and it’s not what we current recommend to people. (This could also explain the concepts without a preferred name situation.)
It would be better to use the reference application + the CIEL dictionary (or just the subset that’s packaged with the refapp) + reference demo data. This will auto-generate diagnoses, with the right structure. In that structure you’d want to look for obs where concept_id = 1284. (technically it should really be obs where obs.concept has a SAME-AS mapping to “org.openmrs.module.emrapi:Coded Diagnosis” but I would ignore this during the spike).
In order to use Kibana’s Tile Maps I need to index person addresses converting latitude + longitude to geolocation points.
It’s easy to do in Logstash, using a transformation filter, and we already have these fields in person_address, but they are NULL and the cities are fake.