How to help with O3 performance?

@dkayiwa was there maybe some work done around Get Lab Results of Patient, or is it still something SolDevelo can look into?

1 Like

@pwargulak no work has been done around Get Lab Results of Patient. So it would be great for SolDevelo to look into it! :slight_smile:

2 Likes

Hi @dkayiwa @ibacher

we’ve looked into FHIR Get Lab Results endpoint and implemented some solution that improves the endpoint by around 70% according to our tests.

Notes on the results

  • it was measured using the openmrs-contrib-performance-test project, export SIMULATION_PRESET=dev TIER_COUNT=2 TIER_DURATION_MINUTES=1 && ./mvnw gatling:test
  • it was executed on a Local as-is
  • and it was executed with an artificial network Delay of 5ms, added at docker container level, to all containers, mainly to highlight an impact of DB queries and simulate deployment in a production environment (which is usually slower than local instance)

The results

To see full results you can download ‘gatling-results.zip’ attachment from related ticket

Here’s the PR with fixes where I’ve included some technical details.

Would be great if you could take a look when you have a free moment.

CC: @pwargulak

2 Likes

Around 70%? This is very exciting!!! Let me head straight to reviewing the code and reading through the details that you have shared. :smiley:

1 Like

@dkayiwa @ibacher any feedback for the recent PR from @druchniewicz ? Have you confirmed the 70% performance improvement?

3 Likes

We have provided some feedback on the pull request.

2 Likes

Yes, thanks for feedback, we’re up to date with your all suggestions and comments, currently we applied draft changes suggested by @raff and left some questions for him. Work is still in progress to get the best solution. Kind regards

Thank you so much for the awesome progress! :+1:

I’ve seen in PR comments that the cacheable solution sparked some changes in other places, but ultimately was implemented.

@druchniewicz Can you share some status? I’ve been wondering how this work fits in the bigger picture of OpenMRS - our 1st involvement was much more self-contained. :slight_smile:

I created a new PR - O3-4765: Improved FHIR Get Lab Results endpoint performance by druchniewicz · Pull Request #568 · openmrs/openmrs-module-fhir2 · GitHub as discussed in the comments in the original PR.

It contains different approach of solution. @raff can you take a look?

I performed some tests with new solution on my local machine and it turns out that results are even better:

Notes on the results

  • it was measured using the openmrs-contrib-performance-test project, export SIMULATION_PRESET=dev TIER_COUNT=2 TIER_DURATION_MINUTES=1 && ./mvnw gatling:test
  • it was executed on a Local as-is
  • and it was executed with an artificial network Delay of 5ms, added at docker container level, to all containers, mainly to highlight an impact of DB queries and simulate deployment in a production environment (which is usually slower than local instance)

The results

1 Like

@druchniewicz @raff is this solution something that can be easily replicated in other endpoints/places in OpenMRS?

@druchniewicz thanks for being open to completely change the solution!

The new solution is based on @Cacheable for caching results, which is much simpler, easier to monitor and data is kept between requests, which explains better results. In openmrs-core 2.8.x the implementation of @Cacheable has moved from ehcache to Infinispan, which can run in clustered environments with cache invalidation between nodes. The whole configuration is provided by openmrs-core.

@pwargulak yes, I’m hoping that @Cacheable becomes the preferred pattern for caching in OpenMRS as it is easily applicable. It’s clean, easy to monitor with e.g. jconsole, grafana, etc. and it is supported in clustered environments with many nodes and cache invalidation between nodes with just a config change. For more details see here.

PR has been finally merged and changes are available on master branch of fhir2 module.

Thank you @raff reliable review and suggestions on what path would be best to follow.

I believe that this can be first step in propagating this solution in other modules. It seems that using the OpenMRS caching mechanism in other modules can be applied quite easily and the results in improving performance can be really significant.

1 Like

Swinging by here to (1) say a HUGE THANK-YOU to the team at @SolDevelo who have been so intentional about helping with O3 performance. Thank you, thank you, thank you!! This was a big gap in community help and you stepped in at the perfect time. I keep hearing for example how pleased Daniel is with your help and findings :slight_smile: :tada: :folded_hands: :clap:

(2) In case anyone hasn’t seen it yet: GSOC student @jayg created this explainer video explaining how to set up and configure our Performance Test dashboard on Gatling. Check it out, and thanks to @jayasanka for encouraging this video too :slight_smile:

3 Likes

@grace Thank you so much for your kind words and appreciation! :pray: :tada:

We could not have made it without Community involvement, quick reviews from @dkayiwa and @ibacher, and great ideas from @raff!

That Gatling is a blast! Must have for any project.

2 Likes