Gsoc 2020: Expose System Metrics For Monitoring.

ayesh · March 2, 2020, 9:47am

Continuing the discussion from Exposing system metrics for monitoring OpenMRS servers:

ayesh · March 10, 2020, 9:40am

I would like to continue talk thread for this project since I saw that you guys will be my mentors for the project.

Btw I got some great input from @cintiadr @ibacher. From what they said I felt like it’s much broad scope for the project. Will it be possible to narrow down the scope by identifying which are the metrics that will be exposing outside through this project.

Infrastructure based metrics
Data analytics points (Ex :- No of users used the patient registration forms)

cintiadr · March 10, 2020, 11:45am

For the infrastructure/JVM, I’d just try and get what https://metrics.dropwizard.io/4.1.2/manual/servlets.html#manual-servlets offers by default. Ensure that show-jvm-metrics is true

I could see https://micrometer.io/docs/concepts#_supported_monitoring_systems as well, which could be useful later.

ayesh · March 10, 2020, 9:00pm

Hi @cintiadr @judeniroshan @suthagar23

I think it’s good to have a module that aggregates and exposes metric data.

For now it contains endpoints which exposes

JVM based metrics collected from the openMRS core. With the help of metrics.dropwizard.io
It should have support to micrometer plugin which has the capability to publish data to different monitoring systems based on the data structure monitoring systems required the data points to be (example dimensional and hierarchical)
Creating an observer class that triggers for specific events that we required as data points in the end. (for an example how long it took for some one to register a patient) and exposes data when requested by an admin from an http call or when requested by the montoring platform

ayesh · March 23, 2020, 12:58pm

Hi @cintiadr @judeniroshan @suthagar23

I thought of an architecture like this with an hibernate interceptor which sits in core since all queries will be going through core I think it’s the best place for it to be.

ibacher · March 23, 2020, 1:54pm

@ayesh If you’re going to be intercepting Hibernate events, it may be better handle that via the Event Module or even the Atom Feed Module rather than having to re-invent the wheel. Or maybe just break my suggestion of UI based reporting out from the main project of system monitoring data.

ayesh · March 23, 2020, 2:06pm

Hi @ibacher

I think that is a great idea.Btw this is the first time am hearing about event module @ibacher. Thanks alot for suggesting this way.

Btw I hope openmrs modules are subscribed as listners.

ayesh · March 23, 2020, 8:23pm

Hi @ibacher

I see the atom feed module is based on event module which basically an active_mq publisher. Which I think we can use in the new module we will be creating

I see the atom module already logs few of the main components data points what about we extend it to get the data points which we are looking for ?

ibacher · March 23, 2020, 9:28pm

That seems to me like it could be a reasonable solution. There are two things about it that makes me a little hesitant in fully endorsing the idea. One is that I’m not sure that Atom (the format or the feed technology) is the future. The other is that I’m not sure how widely deployed the Atom feed module is outside of Bahmni. Maybe the best immediate target is just something built on top of the Events module.

I’d love to hear insight from others in the community on this point, however.

ayesh · March 23, 2020, 9:47pm

Hi @ibacher

Agreed. And we can use atom feed as a starting point since it already has the functionality to extract data from events.

ayesh · March 24, 2020, 9:12am

Hi @ibacher

I see a PIH module which was not actively in development has a monitoring solution .

I can see it record java run time metrics. But there seems to be lot more in that module. Any idea about this module ?

github.com

PIH/openmrs-module-emrmonitor/blob/master/api/src/main/java/org/openmrs/module/emrmonitor/metric/JavaRuntimeMetricProducer.java#L51




/**
 * @return a list of produced metrics
 */
@Override
public Map<String, String> produceMetrics() {


    Map<String, String> metrics = new LinkedHashMap<String, String>();


    // Memory
    Runtime runtime = Runtime.getRuntime();
    metrics.put("memory.total", FormatUtil.formatBytes(runtime.totalMemory()));
    metrics.put("memory.total.bytes", Long.toString(runtime.totalMemory()));
    metrics.put("memory.free", FormatUtil.formatBytes(runtime.freeMemory()));
    metrics.put("memory.free.bytes", Long.toString(runtime.freeMemory()));
    metrics.put("memory.maximum", FormatUtil.formatBytes(runtime.maxMemory()));
    metrics.put("memory.maximum.bytes", Long.toString(runtime.maxMemory()));


    // Date/time
    Calendar cal = Calendar.getInstance();
    metrics.put("datetime.display", DateFormat.getDateTimeInstance().format(cal.getTime()));

I see @mseaton as a core developer in this module.

mseaton · March 24, 2020, 12:29pm

@ayesh thanks for noticing! I’m happy to connect over this module. I am definitely interested in this project and would be interested to see this existing module evolve to meet these needs rather than starting from scratch. This was not a throw-away module, and did have some design put into it, but never took off, and I’d love to see it evolve to where it was useful. @ibacher also, please feel free to connect with me over Slack or we could set up a time to talk.

ibacher · March 24, 2020, 12:45pm

@ayesh Thanks for digging that out. I knew something like that existed but I wasn’t sure entirely where.

ayesh · March 24, 2020, 1:13pm

@ibacher @mseaton

Agreed with you guys

ayesh · March 24, 2020, 5:01pm

@mseaton yeah having a small session will be great to have an idea.

cc :- @ibacher @judeniroshan

ibacher · March 24, 2020, 5:37pm

That sounds good to me. We should engage @judeniroshan and @suthagar23 on such a session, however!

suthagar23 · March 24, 2020, 7:09pm

@mseaton, Please let me know, if you would like to be one of the mentor for this project?

ayesh · March 26, 2020, 9:38am

Hi All

I am not sure weather we have a doodle account for openmrs.

Btw I created a doodle to get an idea about when we can have a meeting

cc :- @mseaton @ibacher @cintiadr @suthagar23 @judeniroshan

cintiadr · March 26, 2020, 10:54am

Screen Shot 2020-03-26 at 9.53.41 pm

I will pass this time! Probably it’s better for me to continue with the async comms.

dkayiwa · March 26, 2020, 11:43am

Loool, that is too terrible for you!!!