While I would love to see a robust and scalable data warehousing solution delivered out of the box with OpenMR, I realistic initial step in that direction would be to build toward alignment on the first piece needed: getting data out of OpenMRS.
Leveraging a tool like debezium has a couple of benefits: (1) more robust than Hibernate interceptors as @wyclif pointed out and (2) the OpenMRS community can focus our efforts on adapting widely-used tools rather than building, owning, and having to sustain more bespoke tooling (i.e., leverage the work of other communities already addressing the low-level issues of connections, pooling, queuing, etc.). The challenge with debezium is doing the work to provide hooks for clients to listen not only for database-level events, but also domain-level events (e.g., patient changed). We also need an approach that doesn’t require clients to call the OpenMRS API to fetch every object that has changed (either by streaming out the data – not just references – or coming up with more efficient ways to access resources in bulk), since asking the API to marshal all observations for even a hundred patients can produce a denial of service attack on our FHIR API module.
We’re getting closer to being able to deliver the OpenMRS Platform as a stack rather than a war file, which will make it easier to share functionality that goes beyond a single java virtual machine. It would be awesome if we could create a proof of concept where a container or two brought up alongside OpenMRS could provide secure endpoints to listen for high value data- or domain-level events and a demonstration of the potential showing how it can be used to generate & maintain near real-time flattened tables (e.g., hello world of flattened patient & flattened encounter tables) that don’t have to live in the same database as OpenMRS.
It sounds like this will the topic of today’s TAC call – with Andy trying to fit in another agenda item. Unfortunately, I have a conflict and will miss at least the first half of the call… but I’ll be looking forward to reviewing the recording.