Community Health Metrics/Dashboard

It would be great to start adding this knowledge to the Wiki somewhere - how to get stats from our code repo to monitor the health of the community…

1 Like

Hmmmm…Is this a good time to revisit this Community KPI’s and Activity Analytics project? It would be great to be able to go to one place for our community data needs. :slight_smile:

1 Like

Since that project was originally proposed, there has been further discussion here around working on a custom solution, using something like Bitergia, or exploring how we can use GoogleBigQuery. @suthagar also shared this resource: https://www.gitmemory.com/openmrs

Here’s what we have been providing in our annual report and/or our quarterly BoD reports since 2015:

  • Commits (quarter/annual)
  • Distinct committers (excludes known bots, but not fully cleaned) (quarter)
  • Min # of committers having 50% of commits (excluding commits by bots) (quarter)
  • Number of downloads (quarter/annual)
  • Number of Talk posts (quarter/annual)
  • Number of new Talk members (annual)
  • Total Talk Visits (annual)
  • Number of Talk Topics Created (annual)
  • Number of Talk Posts Written (annual)
  • Number of implementations by country/implementation (annual)
  • Number of active patients managed by country/implementation (annual)

As we’ve been collecting much of that data since 2015, we are now able to look at emerging trends. We probably want to continue to collect a lot of this data with the same frequency and could make it publicly available quarterly or annually on a “Community Health Dashboard,” in addition to the Atlas (which needs to be fixed).

Two questions:

  1. How did we originally envision using this data? What did we want to know?

  2. Are there examples of how we’ve used this data to inform our direction/activities?

@janflowers @darius @dkayiwa @burke @terry @irenyak1 @c.antwi

1 Like

Thanks @jennifer for this information. And surely good questions.
What is the importance of this data, how is it put to use beyond the quarterly and annual reports? Could we be losing anything in case we do not collect it? For the past 4 years, how has this data informed our decision as Management and the big OpenMRS community?

From another thread, @janflowers talked of an option of having this process automated, How do you see that? Is it something we can easily implement or we can continue with our usual manual process?

Thank you all.

Adding to this from an email offline with @darius:

Value is in -

  1. publish numbers on a discoverable "community health" page
  2. define real-time (e.g. weekly) metrics that we can automate.

For context, @darius original vision was to:

  • automatically pull activity data from github, from Talk, etc
  • store this in elasticsearch "data warehouse" (supported by the infra team)
  • build various tools for measuring community and individual activity against this database
  • …these current analyses are just my first spike down this path
  • Detailed description of this at https://wiki.openmrs.org/display/projects/Community+KPIs+and+Activity+Analytics

See: Prioritizing community metrics for OpenMRS

My opinion is that (sadly) we have never used this info to inform decisions or set direction.

Personally I haven’t ever been in a situation where the aggregate-level metrics were actionable for me. E.g. imagine we see an upwards or downwards trend in # of commits over time. So what?

My original idea was to start calculating an activity score for each community member (see https://wiki.openmrs.org/display/projects/Community+KPIs+and+Activity+Analytics). This would be fun for community members, but my actual goal was to:

  1. notice when a community member who has been “very active” for a while becomes less active, and personally reach out on a case by case basis to see if there’s a way to keep them engaged
  2. recognize new people as they become “very active” and make sure the community and the community management team are aware of those people

In the end I never had time to work on this. Partly I was dissuaded by not wanting to start having significant new infrastructure we need to run and manage.

If someone actually plans to use the answers to these kinds of metrics, I’d be interested to help spec out a design.

1 Like