Organizing modules on Bintray

I’m in the process of setting up publishing omods from our maven repo to Bintray automatically.

Currently on Bintray there are 3 modules in a dedicated omod repo at https://bintray.com/openmrs/omod

They were deployed there manually for testing and use openmrs-module- prefix for package names.

I propose we use module id without any prefix instead.

Personally, I think this makes sense within an “omod” repository on bintray (i.e. the “openmrs-module-” is already implied by the fact that they are in the omod repository). The “openmrs-module-” convention is really just something we adopted for github to enable easy identification and searching of OpenMRS module code across disparate repositories, from what I recall.

Mike

I fully agree! :slight_smile:

+1 (I agree)

It’s fine to drop “openmrs-module-” from the front if it’s there, but don’t assume a module will have this prefix. Also, we should make sure fully qualified module ids won’t break it (e.g., com.example.mymodule). Assuming webservices.rest works, then fully qualified ids will likely already work too, since it’s just a matter of not breaking if the id contains a period.

@raff, are you only talking about newly published artifacts to maven, or does this cover migration from modulus?

I think we need to consider them together, because of we are going to be migrating modules published on modulus into bintray we need their uids to be consistent with future releases via maven.

(Also I don’t think we should blindly migrate every module from modulus, but that may be another conversation.)

-Darius (by phone)

@darius, the way it works is that I can configure Artifactory to publish all artifacts meeting certain criteria to Bintray (including released in the past). I came up with a regex to find all omod jars and put it on Bintray in the correct package and with the right file extension.

I can run it for the whole maven repo or just a defined list of omods (though the regex may get long in such a case).

I’m talking about modulus not maven repo. Our primary source of truth so far has been modulus even though we also duplicate some releases to maven.

We have 200+ modules on modulus, but many fewer in maven. So I think we should make sure to consider the modulus ones in migrating.

-Darius (by phone)

I’m only talking about modules published to maven, but module ids are used as identifiers on modulus as well, aren’t they?

We do need to migrate modules from modulus to Bintray. I don’t know how many modules we have in maven repo in comparison to modulus. If it is many more it may be worth to script that process. If it is just a dozen or so we may migrate them manually.

I’ve just setup the link between Artifactory and Bintray so we now have all modules that have ever been deployed to our maven repo under https://bintray.com/openmrs/omod

I think the next step is to write a script, which will update all modules on Bintray with attributes from Modulus:

  1. Description
  2. Website
  3. Required OpenMRS versions
  4. Owner
  5. Maintainers

The last 3 will be added as custom package attributes on Bintray.

The script should also migrate any missing modules which are in Modulus, but not in Bintray (about 100 of them).

I’d like to pause for a one discuss point: do we really want to put everything under OpenMRS’s bintray account? There’s some value in only having things that are or have been “officially supported” be on OpenMRS’s bintray account, and to encourage that the default behavior is for people to first publish in their own space (which is free for open source thanks to bintray and github).

@burke, do you care to tackle this now? Because the easiest approach is to just to ignore it and migrate everything…

I know it would add complexity if we liberally allow publishing of OMOD snapshots to OpenMRS’s community-shared artifactory, but don’t allow everything to go to OpenMRS’s bintray account. So maybe that’s misguided.

On the other hand, I’ve been looking through all the modules currently in Modulus, and with a very strict definition >40 of them are deprecated/inactive. (That’s an underestimate.) Perhaps we should avoid moving those to bintray?

One more thing to consider is that Bintray has a feature of including (linking) packages published under other accounts in the “canonical” OpenMRS repo as a way to gather all packages in one place.

I’d vote to be very open and let people publish under OpenMRS module repo either directly from Artifactory or by linking from their accounts. I wouldn’t use Bintray account to distinguish “officially supported” releases from those supported by individual members of the community. In fact I’m in favor of promoting and helping with modules developed “unofficially” to help the ecosystem grow.

There are other ways like avatars or tags, which can be used to highlight modules included in RA or developed by teams like Bahmni, PIH, etc.

I’m still unable to login into bintray or artifactory, I thought we have to use our old credentials from nexus but none seems to recognize my username.

@wyclif, please follow on e-mail I originally sent you regarding Artifactory. Bintray has a separate authentication and you can link it with your github account for example. You do not need access to Bintray yet unless you want to evaluate it.

Providing mavenrepo.openmrs.org to others in the OpenMRS ecosystem has been very valuable, and as we’re retiring Nexus it does make sense that we continue providing this service via Artifactory.

If Bintray promotes this “linking” workflow it makes sense to do that too.

@raff, I guess I’m convinced that this is the way to go; we should move over all the maven artifacts to artifactory and bintray. (I guess you did this already.)

Then we need to migrate everything else from modulus over to bintray.

(I would lean towards creating a bintray account like “openmrs-community.” Anything that has ever been a bundled or core-supported module would go to the openmrs account, and other things would go to openmrs-community.)

I’m going to be on vacation for the next two weeks, but when I get back I should be able to finish off the Add-On Index work to replace Modulus, assuming we’ve gotten everything migrated from there to Bintray.

Does that timeline work for you @raff?

Yes, I already created a link between Artifactory and Bintray so all modules deployed to Artifactory are published under the openmrs account on Bintray as well. I should be able to migrate Modulus to Bintray before you are back.

Managing another account on Bintray isn’t appealing to me. Maybe it’s enough to have omod-community repo under the openmrs account? Anyway, I would make things simple and put all modules together in one repo as we do in Artifactory.

Let me rephrase my thinking.

Going forwards, I would think that OpenMRS publishes some modules to bintray/openmrs, but the default behavior for someone creating a new module is that they publish it to their personal bintray space (or, eventually, github releases if we figure out that workflow).

So, when we migrate things from modulus, as a one-time effort, things that are “supported by OpenMRS” in some way go into bintray/openmrs, and everything else goes to bintray/openmrs-community. We’d never really have to manage that account again after this point. Going forwards those modules should get released in a different space.

(Anyway, you all will ultimately decide what to do while I’m on vacation. This was just my thinking. And having a separate repo is also a decent option.)

What was our final decision on this? It looks like some module releases have been published to bintray, but the last release to bintray was back in April (so assumedly all the modules released as part of the Ref App release aren’t here).

Is modulus still currently our source of record?

(No hurry if it hasn’t been completed yet–we are just planning on doing a release of the endTB distro of Bahmni soon, which includes at least one omod, and I want to make sure I deploy it to the current “proper” location).

I understand that Rafal has written migration scripts in such a way that we can run them more than once, and it doesn’t need to be a single big-bang-migrate-and-retire-modulus like I was thinking.

But we haven’t retired modulus yet. It’s still our source of record.

Correct, Artifactory is setup to publish modules from maven to Bintray automatically, though the syncing is disabled at the moment thus RA 2.6 modules didn’t end up being synced to Bintray. I’m still experimenting with the sync feature.

In addition we’ll have an automated migration from modulus to Bintray, which we will be able to trigger more than once to make the migration seamless.

Once all the pieces are done, we’ll retire modulus. Until then it is still the proper location for modules.