Do you have any preference for choosing one over the other?
Obviously, if your reports are Excel or PDF or some other similar format, then it would make sense to use a library that works within those particular formats.
How about tooling, what open-source licensed products are available that are easy to use both for techies and non-techies, probably one that does not need to go to Google’s servers to bring back a chart. Interactive charts would be fun, though static ones are good to start
Assuming this is related to https://wiki.openmrs.org/display/projects/Built-In+Reports+for+Reference+Application, my gut is that we might want to provide some configurable widgets/controls that can be added to the reference application, where one of the configuration options is the dataset/report to run, and other configuration options are the type of chart, and dataset elements to use, etc. But this would be something different from the typical report rendering architecture. Other widgets would just allow for configuring one or more reports to expose (eg. display name and url), which would then either run the report synchronously and download the rendered report, or redirect the user to another page…
@mseaton, yes, it’s the related project, which I should have mentioned.
@lluismf, I haven’t seen Kibana before, thanks! It’s not the best fit for the project I was thinking about, because we want something simple running within OpenMRS, but it’s a great suggestion for OpenMRS data mining. It seems like a decent alternative to Pentaho. Do you have any experience with that tool? Would you be interested in setting up some basic demo for OpenMRS? I’m very interested in pairing on that.
No, I don’t have any experience but a colleague from work is evaluating it and so far he’s very happy. In a couple of days he’s loaded milions of rows in ElasticSearch (from Oracle) using LogStash with no coding (just defining SQL queries to extract the info). Then he created several graphics with Kibana (maps included because the original data has geolocation). On top of that, searching is ultra-fast thanks of Lucene magic.
I don’t have much time but yes, I would be interested but I will need some assistance. As a minimum, an OpenMRS big enough, the data to be extracted (observations?) and graphs that make most sense.
So I guess you would first provide a new REST API, some sort of ChartResource? … that would hold series of plottable data or tables of non-plottable data that could easily be consumed by front-end libraries?
@solemabrothers, many sites have moved from this approach to an approach of flattening data within a secondary datastore and performing analytics there. Some have created bespoke SQL queries to do the flattening (can require some large & hard to maintain queries) and, more recently, people have been looking to existing open source tooling (e.g., debezium and Apache tools) to flatten data. There have been successes, but many are implementation-specific solutions.
PIH created petl as an extensible approach to defining flattening & reporting queries in SQL in a standard way that can be run by an engine.
The Analytics Engine Squad was created by groups trying to solve this problem in a way that could benefit others and be extended by the community. AMPATH and Google built openmrs-fhir-analytics to extract data into a FHIR schema and use Apache parquet (columnar datastore) to tackle the calculation of indicators.
We’d love to get folks from UgandaEMR, NigeriaMRS, KenyaEMR, PIH, AMPATH, Mekom, etc. (all of whom are trying to solve this problem in different ways) to share approaches & coordinate efforts on reporting & analytics to reduce unnecessary duplication of efforts, increase awareness & adoption of new tools (debezium, Apache tools, etc.), and (hopefully) begin discovering the parts of reporting that can benefit from shared tooling in place of bespoke solutions.