Who should have write access to deploy to our maven repository?

We currently have a request from @makombe about writing access to our jfrom/maven repository, and I’d like us to define the rules.

From our wiki, we have:

Community maintained modules

Community maintained modules are available at OpenMRS · GitHub. For those modules developer artifacts are published to the dedicated OpenMRS maven repository at https://mavenrepo.openmrs.org. Compiled OMODs are published to the Bintray repository at Service End for Bintray, JCenter, GoCenter, and ChartCenter | JFrog.

[…] every new module adds some maintenance on us, thus modules are approved on case-by-case basis. All modules published to the OpenMRS maven repository are automatically published to the Bintray repository without any additional steps. […]

Most releases are done through CI at https://ci.openmrs.org/.

We also recommend that everyone else should create their own bintray accounts.

My questions would be:

  • Do we have modules currently deployed to our maven repo which are maintained by implementers? On that case, should they continue to be published there?
  • Do we provide exceptions for certain/biggest implementers?
  • As we have a redirect from mavenrepo.o.o, we could potentially add more aliases to other bintrays accounts. Is that a reasonable idea at all?

(Follow up from [Urgent] Maven repo credentials - #37 by ssmusoke)

cc @dkayiwa @darius @burke @raff @ssmusoke @mogoodrich

I generally approach conventions like this considering autonomy, sustainability, risk, and scale.

  • autonomy: people should be able to do what they want/need to do and this should only be limited to the extent needed to address the other factors
  • sustainability: how much time & resources does it take to maintain the convention? even small amounts of effort can add up. lists need someone who tends to the list; accounts needs someone to create them, clean out old ones, deal with spam, etc.; and, a “maintainer” can be a single point of failure. entropy always wins.[1]
  • risk: could someone easily break things if they made a mistake and, if so, would it be hard to recover?
  • scale: what if 1000 people wanted to do the same thing tomorrow?
  • Do we have modules currently deployed to our maven repo which are maintained by implementers? On that case, should they continue to be published there?

Scanning omods in mavenrepo.openmrs.org, it looks like the majority are modules used in the Platform and Reference Application with some implementation-specific entries (e.g., for KenyaEMR and UgandaEMR).

I think the implementation-specific modules were likely added (or at least established in mavenrepo) before we moved to JFrog.

  • Do we provide exceptions for certain/biggest implementers?

I don’t think we should, since those “certain” or “biggest” are probably the implementations that drive the conventions & culture for the community.

  • As we have a redirect from mavenrepo.o.o, we could potentially add more aliases to other bintrays accounts. Is that a reasonable idea at all?

It depends on what problem we’re trying to solve. The addons index was designed to allow distributed management of the list and where omods could be found.

Have we designed things to assume a bintray.com account, which is not free for everyone? If someone (person or org) does not qualify for an open source bintray account, do they have a free alternative (e.g., can they upload artifacts to Maven Central)?

I think these are questions @dkayiwa, @darius, @raff, and perhaps @mogoodrich might be better to answer.


[1] Second law of thermodynamics - Wikipedia

Currently, Addons index allows one to add files published to Bintray and OpenMRS maven repo, see here.

The OpenMRS Maven repo proxies Maven Central so anything pushed to Maven Central should be also seen by the Addons index.

If there’s a person/org, who does not want to use Bintray or Maven Central, an extension to the Addons index can be contributed to support another way of hosting files (e.g. http, ftp).

We (PIH) publish several of our modules to OpenMRS Maven Repo… in particular, our Bamboo pipeline publishes snapshot versions of many of our modules during our build pipeline. This definitely pre-dates JFrog and may even pre-date the setup of the OpenMRS Bamboo pipeline, so we were kind of “grandfathered” in.

I understand that we can’t allow everyone to do this, so if the decision is made that we have to move away from this, I’m fine with that, as long as we come up with a reasonable timeframe, etc. Also, I haven’t researched, but I’d want to take into consideration if we could get a JFrog Maven repo free for non-profits/open source, or would we have to pay for this?

I will add that as a /dev/5 it’s nice to have my own privileges to read/write to the OpenMRS JFrog repo in the case that there are any problems with an automated deployment that I need to clean up, but if the majority feels we want to limit this, I’m fine with that as well.

fyi @mseaton

Take care, Mark

Does it automatically proxy everything on Maven Central? Or just things with some naming convention?

@darius my understanding is that we proxy everything in maven central (usually that doesn’t mean it’s a full copy, it’s either a pass through or lazy download).

@mogoodrich I absolutely don’t want to revoke PIH write access out of the blue. Even if you had PIH-specific modules deployed somewhere else, you can still retain the user account on our jfrog account to aid with core releases and so on.

Not that I understand exactly what are all the problems we are solving with our maven repo, but addons helps to make a module available for others.

But during development they might have a module that depends on other module/api. That’s a problem that only a maven repository can help.

We also have distributions wars in maven repo, which I don’t know if they are useful during development.

@burke, I believe that only open source apps can be deployed to maven central (same as bintray). So open sources could have free maven storage, but not proprietary.

I suppose we can investigate how hard it would be for an implementation to start releasing their things to bintray (including if development gets more complicated). @makombe, would it be a viable solution for you?

For the modules that are already there, I’m thinking of creating a new repository in mavenrep.o.o, one per group. Let’s say, modules-pih, modules-emruganda, and so one. So the CI user (only) for those groups would be limited to that specific repo. Human users would continue to be able to modify everything.

With a little bit of coordination, I think this would be achievable by the end of the year. It would also pave the way for when/if it gets migrated somewhere else.

And improves security overall as a bonus.

I think that should work for us, thanks @cintiadr!

I think that would be nice. In case a module depends on other module/api it is easily available for development(for collaboration purposes) as well as to those who would like to use them, thanks @cintiadr

Hi @makombe,

Sorry, I didn’t really understand your answer.

So, is bintray an acceptable tool for your team for new modules? If not, what’s missing there?

Also, do you know which modules or artefacts that belong to you are already deployed to jfrog?

@cintiadr Yes, bintray is ok for our team.

Awesome, I raised ITSM-4204 for the work with the modules already there.

I don’t believe I will start working on it for 2 or 3 months :D, but I will try to keep you all informed when that happens.

@cintiadr Kindly update us on the progress of @makombe request. Let us know in-case anything is required from us for fast track this request.

Hi @andrine,

The decision was for @makombe team to create and maintain their own bintray, so there’s no request pending.

Kindly clarify since the request from @makombe was to allow us to publish our modules and artifacts onto the openmrs repository. Kindly let us know if this can be done.copying @aojwang

Let’s try to undo the misunderstanding.

@makombe requested to allow you to publish your modules (not community modules owned by our github organisation) to our maven repository.

That’s not how our maven repository is supposed to work. We should only have community owned modules, the ones that are in our github repository. The suggestion we give for other teams is to setup their own bintray. So I asked if that solution would work for your case, and the answer was ‘yes’.

If it’s not on github/openmrs , it shouldn’t be in mavenrepo.openmrs . The exception is the teams which were already there due to historical reasons, I won’t just go and delete them.

Okay @cintiadr .This is well understood. Thanks.

Hi @mogoodrich,

As I’m handling JFrog for other reasons, I created now ‘modules-pih’ and ‘modules-pih-snapshots’ in JFrog.

https://mavenrepo.openmrs.org/nexus/content/repositories/modules-pih https://mavenrepo.openmrs.org/nexus/content/repositories/modules-pih-snapshots

I granted access to write to these repositories for only PIH users (including your CI user). No other system or person has permission to deploy there. It’s part of the ‘public’ repo, so it should work as before.

So all you’d have to do is change your pom files to use the new URLs. Everything would still be exactly the same.

In a couple of months, I’d like to change your CI user to only have access to deploy to those repositories (to avoid any confusion). All ‘human’ users should continue with the same access as now.

Great, thanks @cintiadr, I will test it out!

@mogoodrich while the integration between jfrog and bintray is not working, I do expect that jfrog support will eventually help us there.

Which bintray repository (if any) would you like your modules to be automatically deployed to? Would you be ok to have an PIH bintray repository?

Same for @ssmusoke and UgandaEMR modules.

@cintiadr re: bintray, I’m not sure… what are we thinking the potential options are? How does/will the automatic deployment work? (if there’s a thread describing everything feel free to just point me to that link :slight_smile: )

When you say “PIH bintray repo” are you talking about having a PIH branch within the OpenMRS organization, or PIH requesting it’s own bintray account?

Thanks!

Take care, Mark