Gsoc 2020: Expose System Metrics For Monitoring.

Tags: #<Tag:0x00007fa3ef6f50f0>

@mseaton yeah having a small session will be great to have an idea.

cc :- @ibacher @judeniroshan

That sounds good to me. We should engage @judeniroshan and @suthagar23 on such a session, however!

1 Like

@mseaton, Please let me know, if you would like to be one of the mentor for this project?

Hi All

I am not sure weather we have a doodle account for openmrs.

Btw I created a doodle to get an idea about when we can have a meeting

cc :- @mseaton @ibacher @cintiadr @suthagar23 @judeniroshan

Screen Shot 2020-03-26 at 9.53.41 pm

:smiley: I will pass this time! Probably it’s better for me to continue with the async comms.

Loool, that is too terrible for you!!! :smile:

Hi @ibacher @mseaton @judeniroshan

I found that it’s possible for all of us to have the call today 18:30 - 19:30pm CET (Central Europe Time). I hope all of you will join the call. But am not quite sure will it be possible for us to connect from slack ?.

Screenshot from 2020-03-27 10-23-26

Time differences might be the constraint here ! but it sounds cool. :slightly_smiling_face:

@joachimjunior the original doodle is set on cet time :slightly_smiling_face:

Oh :smile: got you @ayesh

I can setup a Zoom for this if that would be more efficient. I’ve never done a multi-person call via Slack.

Sure that will be great @ibacher

Here’s the details for the Zoom meeting:

Ian Bacher is inviting you to a scheduled Zoom meeting.

Topic: Expose System Metrics Time: Mar 27, 2020 01:30 PM Eastern Time (US and Canada)

Join Zoom Meeting

Meeting ID: 383 583 838

One tap mobile +13126266799,383583838# US (Chicago) +16465588656,383583838# US (New York)

Dial by your location +1 312 626 6799 US (Chicago) +1 646 558 8656 US (New York) +1 346 248 7799 US (Houston) +1 669 900 6833 US (San Jose) +1 253 215 8782 US +1 301 715 8592 US 877 369 0926 US Toll-free 877 853 5247 US Toll-free Meeting ID: 383 583 838 Find your local number: https://brown.zoom.us/u/ahaTawOf8

Join by SIP 383583838@zoomcrc.com

Join by H.323 162.255.37.11 (US West) 162.255.36.11 (US East) 221.122.88.195 (China) 115.114.131.7 (India Mumbai) 115.114.115.7 (India Hyderabad) 213.19.144.110 (EMEA) 103.122.166.55 (Australia) 209.9.211.110 (Hong Kong) 64.211.144.160 (Brazil) 69.174.57.160 (Canada) 207.226.132.110 (Japan) Meeting ID: 383 583 838

1 Like

Hi @ibacher @judeniroshan will u be joining the call ?

@mseaton @ibacher @cintiadr @judeniroshan

What do you think about these two tools that @cintiadr mentioned I think both are general-purpose metric instrumentation libraries

I can see we can expose metric reports as JMX, CSV and JSON objects in Dropwizard I know these ways of exposing already enough for most of the monitoring tools which are currently in use. But it seems Micrometer has more support to monitoring tools which is a plus point for it?

  1. Drop Wizard :- https://metrics.dropwizard.io/4.1.2/manual/servlets.html#manual-servlets
  2. Micrometer :- https://micrometer.io/docs/concepts#_purpose

Also Micrometer does seem to be being adopted by Spring (in the form of Spring Boot) which might be a good argument for it. I’m pretty agnostic about the framework chosen, though; just that we aren’t reinventing the wheel.

3 Likes

+1 to what @ian said. I’d probably lean towards micrometer for the reasons above, but I have done no actual analysis on either.

1 Like

Thanks all for the feedback :slightly_smiling_face:

Hi @ibacher @mseaton @mozzy @judeniroshan

I was looking at the atom feed module to understand how it captures events related to object types.

I actually found a HibernateInterceptor which capture the db events and serveEvents for the atomfeed module.Which actually amaze me a little. Because I thought the events were served based on the Events module activemq message broker.

Few questions I have

  1. Is Events module actually engaged in the atomfeed module if so where is it ?
  2. I think up to saving to the event to the db we can use the same flow for our module. Because it has a mechanism to queue the events and do the db save transactions one by one. I think that is a good advantage for us since for a module that captures data points It’s a must to make sure not to lose any data points at any point. What do you think the are we going to use the same flow till saving the event details in the db. The events table looks like below.

For the data points, that we are going to add additional to what already captured by atomfeed can be extended easily with the way they have developed it.

         1 Include the openmrs class of the object type in the atomfeed config.
         2 If there is no subresource builder make sure to add it. 

Then it will start capturing the events of the new class that we included.

But one thing I noticed it’s using transaction classes in ICT4H commons module. I am not sure whether this is good or we have to migrate them to openmrs organization.

  1. What are the data points we can capture other than what atomfeed does?

Here is the list of classes that are included for now.

I see some of them as enabled : false any paticular reason ?

  1. Why do we have api , api 1.9 , api 2.0 separations ?

Hi @ayesh!

Thanks for your digging on this.

Yes, the Wiki says. that atomfeed runs on the Events module, but actually it looks like that was never implemented. The Hibernate interceptor is basically identical across the two except that: in Events it uses the afterTransactionCompletion() event and checks that the transaction was completed before firing notifications, where in Atom-Feed, it uses the beforeTransationCompletion() event and seems to generate events regardless of if the transaction was committed or not (the change was made in this commit). This seems to have been related to some funkiness with accidentally creating a sub-transaction to store data in a the feed, but there’s a lot more in this Talk post. It appears that the future state Darius alluded to was never implemented. That might be worth reviving.

Questions:

  1. Apparently, no. I still think having a single solution here is worthwhile, but apparently the event module could use some attention.

  2. One of the advantages of using the events module and ActiveMQ is that we don’t have to implement custom queuing. I would think either migrating ActiveMQ in the events module to use the database for storage or building off the AtomFeed module are the right ways to address this.

  3. AtomFeed’s main use-case (as I understand it) is for sharing data among the various components of Bahmni and as part of Sync 2.0. The classes excluded are probably not commonly shared via those interfaces (admittedly Order is a bit of a weird one).

  4. Backwards compatibility? api-1.9 is only applicable when run in versions < 2.0.0; api-2.0 is applicable in versions > 2.0.0.

1 Like