New guidelines for module releases

Development
1

We are finalizing the process of migrating from module distribution channels managed by our infra team to Bintray and cloud Artifactory. It is possible thanks to the generosity of the JFrog company, who is hosting us for free!

The Add-ons Index is hosted on OpenMRS maintained servers. However, it is a lightweight application using Bintray as storage backend.

As a reminder the migration entailed:

  1. Moving away from Nexus to cloud Artifactory for mavenrepo.openmrs.org, where we store jars and omods for development.
  2. Moving away from Modulus to Add-ons Index backed by Bintray’s CDN.

Modulus will continue to operate for 2 weeks and is periodically synced to Bintray and available under https://bintray.com/openmrs/omod, which is also indexed by the Add-ons Index. After 2 weeks we will point modules.openmrs.org to the Add-ons Index and take Modulus down.

If you release your module to mavenrepo.openmrs.org, it is automatically published to Bintray and visible in Add-ons Index so there’s nothing you need to do other than forgetting about manually releasing omods to Modulus.

For any new modules, which are not community maintained (not hosted at https://github.com/openmrs), we kindly ask you to publish them under your Bintray account (both maven and omod artifacts). It will help us to scale and take off the maintenance burden from the OpenMRS community. Please see updated instructions on the Module Release page.

Please let me know, if anything needs to be better documented or you need individual assistance.

We are also looking for volunteers, who will go through our wiki pages and look for places, which mention releasing module to Modulus and update them accordingly preferably by pointing them to the Module Release page. Let me know, if you are interested and we can work together to help you identify such pages!

4 Likes
2

Thanks @raff!

Just to confirm, at this point going foward there is no manual step required when releasing a core OpenMRS module to publish it to Add-ons? It automatically gets deployed to Bintray from Artifactory?

I released a new version of the Provider Management module on Friday and manually uploaded it to Bintray, which may not have been required? (And which I potentially put in the wrong place?)

I tried to confirm that it’s in Add -Ons, but I can’t get to addons.openmrs.org right now… is that still the correct URL?

Thanks! Mark

3

Correct, no need to push it to Bintray, nor Add-ons.

It’s all good OpenMRS Add Ons

4

Perfect, thanks @raff!

5

To point out: we have added (almost?) everything that was in Modulus into the Add On Index, and this definitely includes all the modules that were in the reference application.

For new modules, you need to explicitly add it, according to the instructions at https://github.com/openmrs/openmrs-contrib-addonindex/blob/master/PUBLISHING-AN-ADD-ON.md

(And as a module author you might want to check on all your active modules just to make sure they’re in the index.)

6

\o/ \o/ \o/ http://gifimage.net/wp-content/uploads/2017/02/Happy-GIF-Image-Download-26.gif

I’d love to have modules.openmrs.org DNS entry managed by terraform (just like https://github.com/openmrs/openmrs-contrib-itsm-terraform/blob/master/watamu/main.tf#L41), if that’s not too much to ask :slight_smile:

If needs to be manually deleted from our DNS server before re-creating it via terraform, but it shouldn’t be a big deal.

(I can do that, but I have the feeling @burke would be interested in doing this one himself :smiley: )

7

@raff, can I power off modulus?

8

@raff Thanks alot for the Information. i think we also need to ripple these changes on the concerned wikis to smoothen the work for future Release Managers.Beginning with RefApp 2.7 upwards.

9

@cintiadr, yes, let’s do it! modules.openmrs.org should redirect to addons.openmrs.org. We need to schedule it. Before we shut if off I need to run a script to sync Bintray and modules one last time. Let me know when you have some time to do it.

@tendomart, would you be interested in searching around the wiki to find pages mentioning old instructions and updating them? If yes, then please claim https://issues.openmrs.org/browse/AO-12.

1 Like
10

@raff am willing.However got a question is it a manual search or there is a tool to accomplish just that?However i may take it on after am done with releasing RefApp 2.7,if no other folk has shown interest…

11

Any weekend for me is good :slight_smile: During week days, it will be a little complicated this week, but we could schedule next week.

If you have a script I’d need to run, I could do that.

12

We need to test and verify that the OpenMRS API’s ModuleFactory can correctly get updates and new versions of modules from add-ons. (I who wrote a unit test for it, but someone should really test this in real life before we make the final switch.)

-Darius (by phone)

13

I can do that.

14

Happy to hear! Unfortunatley, it is a manual search.

1 Like
15

@raff, did you get to do this?

(Assuming not, we need to assign someone to do it before the end of the calendar year when modulus is turned off!)

2 Likes
16

Oops, there’s definitely at least one bug in the current addons code: I’m looking into this: https://issues.openmrs.org/browse/AO-13

17

I have found a big problem. Now that we’re hosting our modules on bintray, the download URLs are like Service End for Bintray, JCenter, GoCenter, and ChartCenter | JFrog (i.e. the filename is in the query string, not the path).

However when openmrs-core finds a module update with that filename and downloads the file, it ends up naming it “download_file”, which fails to load (since it doesn’t have an omod extension), and OpenMRS then deletes both the old and new files from its modules folder.

[Debugging] The url itself does a 302 redirect to https://dl.bintray.com/openmrs/omod/htmlformentry-3.4.0.omod which does a 302 redirect to a long akamai url. The problem is at https://github.com/openmrs/openmrs-core/blob/2.1.1/api/src/main/java/org/openmrs/module/ModuleFactory.java#L1564 [/Debugging]

I’m not sure how to approach this. The hacky approach is to change addons.openmrs.org so that it indexes the first redirected url instead of the one we’re currently indexing. (Or maybe we can just expose that url for this legacy behavior.)

The right way would be to fix openmrs-core so that it handles the “response-content-disposition” header (and ideally uses an existing library instead of custom url-redirect-following code that we’ve written in OpenMRS).

The problem is that implementations running existing releases would have a pretty annoying bug if they try to update modules via Manage Modules in the legacy UI (or presumably in the new OWA UIs for this, because it’s a core bug) where trying to update a module actually deletes it.

Thoughts?

18

We do have a nginx box to do http hackery magic:

https://github.com/openmrs/openmrs-contrib-itsmresources/blob/master/ansible/host_vars/campo.openmrs.org/vars#L12 (accessible only for /dev/4 and /dev/5).

I doubt I’ll have the time to play with that (and it’s not even my expertise :D) , but if anyone wants to go for http requests manipulations of any sort, that’s the place and the right box.

1 Like
19

I agree with @cintiadr, the solution for this is to use nginx to rewrite the url/headers as needed to make the legacy code happy. Would something like this help?

server_name: "modules.openmrs.org"
location ~ ^.*/download/(.*)$ {
  return 302 https://dl.bintray.com/openmrs/omod/($1);
}
location ~ ^.*$ {
  return 302 https://addons.openmrs.org/;
}

That is, we point modules.openmrs.org to our redirect server (instead of addons) and it redirects any download requests to https://dl.bintray.com/openmrs/omod/{filename}. All other requests to modules are redirected to addons.openmrs.org.

Would that work?

20

After my previous post I realized that I can work around this issue with a very localized hack in the Add On Index code. (Here is the commit, if anyone cares.) I prefer to do this versus having an nginx rewrite because it keeps the hacky business login in the server that’s actually serving these requests.

So, I no longer have any objections about turning modulus off and pointing its hostname to addons.