Strategy for maven repo and software distribution in general

There were a couple of challenges (see: 1 and 2) we faced recently around our maven repo, which forced us to think about moving away from a self maintained repo to an externally managed service.

In addition to the maven repo I suggested to reconsider how we distribute software. Right now it is divided across sourceforge (wars and standalones), modules.openmrs.org and a missing distribution channel for Open Web Apps.

Even though Sourceforge served us well hosting wars and standalones since the beginning of OpenMRS, their reputation isn’t the best these days (see e.g. this). They have no proper REST API so any integration is hard.

Let’s consider alternatives:

  1. Bintray seems like a good fit for all our binaries. It can support maven artifacts (backed by http://jcenter.bintray.com/ and https://oss.jfrog.org/repo/), wars, standalones, omods, owas and more. It has a fully featured REST API so integration is easy. I’ve reached out to the JFrog company, which stands behind Bintray and learnt the following:
  • Free Bintray for open-source projects is limited to 10GBs of storage and 1TBs of downloads a month
  • Once the project exceeds the limit we’ll be asked to remove storage to clear up space or in the case of downloads, no further downloads will be allowed or else we’d have to move to a paid plan

We would possibly fit in the free plan when using Bintray for just modules and owas.

If we migrate our maven repo (uses 10-12GBs of storage, download stats unknown) to Bintray we’ll instantly have to move to the paid plan. Same applies for wars and standalones, which looking at sourceforge stats easily use up 2TBs of downloads a week and 10GBs of storage. Hosting all binaries on Bintray would cost easily ~$6000/month for estimated 13 TBs downloads a month and 20GBs of storage.

  1. There’s also a possibility of using Artifactory. Artifactory is a repository manager for plenty of package types. On this thread we’ve been offered by @jbaruch a free Artifactory Cloud instance for OpenMRS! If we get that officially, we could stay on the free side by using Artifactory for all our binaries (inc. maven artifacts, standalones, owas and modules). It has a robust REST API, which we could use to create a thin UI client for presenting modules and owas. It also integrates with Bintray so we could easily push modules and owas to Bintray from Artifactory. I’ve contacted JFrog directly and they are yet to confirm the sponsorship offer.

  2. Another option for maven repo alone is https://oss.sonatype.org/, which is a gate to Maven Central. I confirmed with Sonatype, they do not have free usage limits for open-source. One issue is that Maven Central has strict requirements regarding published artifacts and if we want to migrate old artifacts we would have to make sure they all comply with the rules described here. It may require from us writing a small tool, which will run through all our artifacts and fix issues. Once we do that they can do a bulk migration for us. The big benefit is that OpenMRS artifacts would be officially in the Maven Central repo along other major open-source projects (only OpenMRS SDK is currently there).

  3. There’s also GitHub releases (for all our binaries excluding maven artifacts), which do not have free usage limits for open-source (except a single file must be under 2GBs). They expose a fully featured REST API, which we could also utilise for building a thin UI client for modules and owas.

That’s all what I was able to find. If you know of any other possibilities please share them here!

2 Likes

Thanks @raff! I’m certainly not up-to-date on what is available, so I defer to those that know more. Would be interested to hear what you think the best option is. Artifactory seems like a good option, assuming we get confirmation of the offer from JFrog?

Take care, Mark

If we actually have that offer up, Artifactory looks like for me the simplest and easiest one. I’d go with it, and I’m not even sure that pushing to bintray is the best for downloadables. Maybe we just search/rest api from artifactory and be done?

Creating artefacts for maven central can be pretty interesting, I had to do that and getting GPG to work on the bamboo agents as I needed took some time. But as you mentioned, comes with the advantage of well, being maven central. I believe it’s not possible to delete anything after uploading, even if a release was made by mistake.

Heads up, we added the following URLS on previous pom files, hence we kinda of have to keep them working ‘forever’:

We can do that with magic apache redirects, but we cannot forget this.

I agree that if we are able to get Artifactory for free, it seems the best and easiest. Should we ping them again? (And…how long would we wait to hear back before moving on to some other option?)

I’m also for Artifactory. I’ve pinged both @jbaruch and JForg’s support. I’ll let you know as soon as I hear anything back from either.

Hi guys, sorry for the delay, I’ll get back to you soon with the decision.