[RESOLVED] Nexus and Confluence: DISK SPACE IS LOW!

Can’t all SNAPSHOTS until end of life releases be removed after performing an offline backup?

And where do you propose we make that backup to? We don’t even have space to tarball it.

I can maintain them on a Digital Ocean server Until something is ready.

diving deeper, it appears all thst space is metadata – I’m gonna delete those --it’s all trash :slight_smile:

1 Like

I nuked the trash directory which frees up a lot of space. We’re fine for now. We need to periodically actually empty the trash folder for older snapshots.

You may now resume making releases. We’re not however going to maintain a mirror of Maven Central anymore. Going to nuke those directories to free up some space.

@r0bby, does /opt/sonatype-work/nexus/storage still take ~70GB or was that reduced by clearing trash? How much space is now free on that server?

Have we confirmed that this change will not affect developers?

I’m not entirely sure – but the proxy repo has been disabled anyways. People should configure maven central in their poms.

@darius / @mogoodrich / @maurya / @raff – please check if development cycles are affected and report your findings back here (e.g., building something that needs one or more artifacts that were previously proxied and are not locally cached). Hopefully, maven will just silently use a central repo. If not, then we need to get the word out about any changes people need to make + ensure CI is adapted as well if needed.

I have it re-caching central – but if we can save disk space – this would keep nexus down. I’d like to delete those proxy repos to be honest – we don’t need to be caching them…If Maven can silently fall back let me know – I’m going disable the proxy repo – don’t worry data is still there. I wanna see if it breaks things.

@burke from the other thread sounds like we are down to 16gb now… so like 40gb+ was trash. I’ll try the PIH builds and see what happens…

Mark

@mogoodrich, actually it’s 10GB now. Like 51GB + was trash from snapshots. I’ll keep monitoring this – but the immediate danger is over. We can migrate in a way thats not disruptive now.

So long as Nexus keeps under control and doesn’t grow, I am happy. I’ve disabled the proxying of maven central – all proxy repo actually. Once I am certain this won’t break things, I’m going to outright remove the proxy entirely.

I guess something got wrong, because multiple builds on Travis are failing(e.g. distro-referenceapplication, html-formentry, PR at metadatamapping), all of them due to the same error:

Failure to find org.azeckoski:reflectutils:jar:0.9.14 in http://repo.maven.apache.org/maven2 was cached in the local repository, resolution will not be reattempted until the update interval of central has elapsed or updates are forced → [Help 1]

This issue was encountered before: [TRUNK-4898] - OpenMRS Issues and was linked with issues in openmrs maven repo: Module releases missing from maven repo

2 Likes

Created an issue at https://issues.openmrs.org/browse/TRUNK-4921

I’ll fix it asap.

@burke, Maven does check maven central by default. Only if you specify a mirror in your settings.xml you can prevent Maven from accessing maven central, but we don’t do that and never recommended that.

@r0bby, it’s not only SNAPSHOTs that you deleted, but all artifacts, which were copied from the Maven central repo, thus we experienced TRUNK-4921. I’ve uploaded 4 missing artifacts, which are no longer available in Maven central (we should probably upgrade to newer versions soon) to our 3rd party repo and we should be good now.

Thanks everybody! PIH build cycle looks fine (well, except for a couple failing smoke tests, but that’s on our end).

Take care, Mark

Please upgrade your poms to add that mirror. I am not adding back maven central. There is no reason to be mirroring it.

No one needs to add any mirrors to poms. You read my post wrong.

Ah nevermind