Add "Maven Deploy" Task to end of "Release" Stage of Ref App modules

mogoodrich · October 7, 2016, 7:52pm

For those modules that have a release stage configured in Bamboo, I propose adding a “Maven Deploy” task as the last step of the process.

Reasoning: after doing the release, the module is upgraded to the next snapshot version, and the change is pushed to git, but I don’t think that snapshot version is deployed to the SNAPSHOT repo until the build runs again (ie after the next change to the module).

In the PIH build pipeline, we are automatically upgrading version numbers, and so for modules in which we are relying on SNAPSHOT version numbers, our version numbers get upgraded and then the builds fail because the snapshots don’t (yet) exist in the Maven Repo. (I’m 100% sure why our version numbers get upgraded if the versions don’t yet exist, but, regardless, building the new version as soon as we upgrade the version number seems the right think to do.)

Thoughts? Mark

wyclif · October 7, 2016, 8:35pm

Makes sense

cintiadr · October 8, 2016, 12:16am

Hum. This should be already happening for every module which belongs to the refapp. This is what is called after the release:prepare/perform ( e.g. https://ci.openmrs.org/build/admin/edit/editBuildTasks.action?buildKey=AU-AUL-RTM)

github.com

openmrs/openmrs-contrib-bamboo/blob/master/distro-update.sh#L95


      
          #!/bin/bash
          # Script to checkout the distro and update version a certain module. 
          # It can save the released version or next snapshot. 
          
          set -e
          
          PROPERTY=""
          NEXT_DEV_VERSION=""
          RELEASE_VERSION=""
          PREPARING_DISTRO=""
          SCM=""
          BRANCH=""
          
          CLONE_FOLDER="target/distribution"
          CURRENT_DIR=$(pwd)
          
          echoerr() { echo "$@" 1>&2; }
          
          help(){
              echo -e "\n[HELP]"
              echo "Script to update version in refapp distro"
              echo "Usage: `basename $0` -r release-version -d development-version -p pom-property -s scm -b branch -n update-next-snapshot (true/false) [-h]"
              echo -e "\t-h: print this help message"
              echo -e "\t-r release-version: version to be released"
          }
          
          
          test_environment(){
          
          	if [[ "$RELEASE_VERSION" == "" || "$PROPERTY" == "" || "$NEXT_DEV_VERSION" = "" || "$PREPARING_DISTRO" == "" || "$SCM" == "" || "$BRANCH" == "" ]]; then 
          		echoerr "RELEASE_VERSION = $RELEASE_VERSION \tPROPERTY = $PROPERTY \tNEXT_DEV_VERSION = $NEXT_DEV_VERSION \t \
          		PREPARING_DISTRO = $PREPARING_DISTRO\t SCM=$SCM\t BRANCH=$BRANCH"
          	    echoerr "[ERROR] At least one command line argument is missing. See the list above for reference. " 
          	    help 
          	    exit 1
          	fi 
          
          	if [[ "$$MAVEN_HOME" == "" ]]; then
          	    echoerr "[ERROR] MAVEN_HOME variable is unset. Make sure to set it using 'export MAVEN_HOME=/path/to/your/maven3/directory/'"
          	    exit 1
          	fi
          }
          
          ARGUMENTS_OPTS="r:d:p:s:b:n:h"
          
          while getopts "$ARGUMENTS_OPTS" opt; do
               case $opt in
                  r  ) RELEASE_VERSION=$OPTARG;;
                  d  ) NEXT_DEV_VERSION=$OPTARG;;
                  p  ) PROPERTY=$OPTARG;;
          		s  ) SCM=$OPTARG;;
          		b  ) BRANCH=$OPTARG;;
                  n  ) PREPARING_DISTRO=$OPTARG;;
                  h  ) help; exit;;
                  \? ) echoerr "Unknown option: -$OPTARG"; help; exit 1;;
                  :  ) echoerr "Missing option argument for -$OPTARG"; help; exit 1;;
                  *  ) echoerr "Unimplemented option: -$OPTARG"; help; exit 1;;
               esac
          done
          	
          test_environment
          
          # If preparing a refapp distro release, commit the released version. Otherwise, next snapshot
          UPDATE_RELEASE=""
          if [[ "$PREPARING_DISTRO" == "true" ]]; then
          	UPDATE_RELEASE="$RELEASE_VERSION"
          else
          	UPDATE_RELEASE="$NEXT_DEV_VERSION"
          fi
          
          
          mkdir -p $CLONE_FOLDER
          git clone $SCM $CLONE_FOLDER
          cd $CLONE_FOLDER
          git config push.default current
          git checkout $BRANCH
          
          if ! egrep -q "<$PROPERTY>[^<]+</$PROPERTY>" pom.xml; then
          	echoerr "[ERROR] Property $PROPERTY does not exist in the distro pom $SCM. It cannot be updated"
          	exit 1
          fi
          
          sed -i'' -r "s|<$PROPERTY>[^<]+</$PROPERTY>|<$PROPERTY>$UPDATE_RELEASE</$PROPERTY>|" pom.xml
          
          # git diff-tree is always returning 1 in the Bamboo agents :/ Using git diff instead
          if [[ -z $(git diff) ]]; then
          	echoerr "[WARN] Property $PROPERTY was already set to $UPDATE_RELEASE. Skipping commit."
          	exit 0
          fi
          
          git add pom.xml
          git commit -m "[Maven Release] Increasing version of $PROPERTY to $UPDATE_RELEASE"
          
          git push

Do you have any occurrence of that? What happened is that apparently some modules were not released from Bamboo due to some unexpected errors (which I don’t yet have any data to investigate).

The $PREPARING_DISTRO is a Bamboo variable (https://ci.openmrs.org/admin/configureGlobalVariables!default.action | Releasing a Module from Bamboo - Documentation - OpenMRS Wiki ), and the default is true. So I’d expect the SNAPSHOT to always be there just after the release if the released module is part of refapp.

darius · October 8, 2016, 4:42pm

Shouldn’t the git push trigger another build to be run, and thus deploy the next development snapshot?

dkayiwa · October 8, 2016, 6:34pm

I have always noticed what Mark reported, even when releasing from bamboo worked flawlessly. As a result i got into a habit of always manually running the bamboo plan immediately after releasing such that the snapshot for the next version gets deployed.

cintiadr · October 8, 2016, 11:44pm

In fact, it doesn’t. Bamboo will ignore any commit with start with the ‘[maven-release-plugin]’, to make sure you are not building a stable version (doing a ‘mvn install’ in a non-snapshot version will cause the .m2 cache to have invalid artefacts). I do remember seen that in Bamboo code, but I’ve never tried in other CIs - but it wouldn’t surprise me if they have the same behaviour.

Daniel, even for modules which are in refapp distro? If so, something went wrong.

dkayiwa · October 9, 2016, 8:04am

@cintiadr yes. For instance, i have just released version 1.2 of the adminui module. Any reference to 1.3-SNAPSHOT result into this: Failure to find org.openmrs.module:adminui-api:jar:1.3-SNAPSHOT in http://mavenrepo.openmrs.org/nexus/content/repositories/public was cached in the local repository, resolution will not be reattempted until the update interval of openmrs-repo has elapsed or updates are forced -> [Help 1]

Until when a commit is made to the module or when i manually run its bamboo plan.

A second example is the coreapps module where i have just released version 1.11.1 and registrationapp 1.6

cintiadr · October 10, 2016, 10:38pm

Alright.

So, I can see you released that module in Build 135

As the preparing.refapp.distro.release was not changed, it used the default true, which committed the released 1.2 version to the distro.

The case would be different if that variable was set to ‘false’. The next snapshot would be upload to nexus, and the distro would receive the next SNAPSHOT on the commit instead of the the released version. So you wouldn’t need to add new commits to the distro like a96cd3 or 55371d.

So, automation appears to be in place, but it’s possibly not very discoverable. We are only uploading the next snapshots to nexus if the snapshot was automatically added to refapp distro by the release build in Bamboo.

Should we add the SNAPSHOT to the distro as the default option?
Should we always upload the snapshot, regardless if that module is in a distro or not?

mogoodrich · October 11, 2016, 2:47pm

I think we should keep the default option of not adding the SNAPSHOT to the distro, but we should always upload the new snapshot, regardless if the module is in a distro or not.

Mark

raff · October 12, 2016, 9:31am

I agree. Thanks @cintiadr for looking into that!

darius · October 12, 2016, 4:39pm

I don’t see this documented in the Reference Application Release Process, which would explain why it wasn’t correctly set this time.

@cintiadr, can you coordinate with @maurya to ensure this is added in the right place?

cintiadr · October 14, 2016, 8:41am

The change to deploy snapshots will be merged soon (PR-2), I kinda lost some git access . I will merge and test it properly then

I updated Reference Application Release Process to change the preparing.refapp.distro.release bamboo variable. Feel free to reword it.

I also added two tasks in the distro release task to reset the preparing.refapp.distro.release to false after a green refapp distro release. That one is pretty hard to test, so I wouldn’t be surprised if it doesn’t work on the first time. @taremwatadeo is the new release manager, right? So please let me know if I broke it

dkayiwa · October 14, 2016, 9:03am

Thanks a lot @cintiadr for making this happen!

About git, i have just given you admin access to the openmrs-contrib-bamboo project. I wonder how i forgot you when working on dev stages. You are now a /dev/3

cintiadr · October 14, 2016, 10:24am

Yeah, I’m not exactly a dev, so the /dev/* is sort of weird for me

dkayiwa · October 14, 2016, 1:08pm

@cintiadr a spill over effect of your great work to support devs. By the way, i did not know that you are a non dev. This reminds me of the fact that we need to come up with some sort of badge or grouping for non devs.