Continuing the discussion from CI Build is unable to connect to host:
I’m trying to get Reference Application UAT for 2.4 released onto devtest05.openmrs.org, but the build fails with “permission denied”. I’ve verified that the public key at /home/bamboo/.ssh/authorized_keys matches the one on int01.openmrs.org.
Is there anyone who might be able to point me toward a solution? What am I doing wrong?
Thanks.
-Burke 
Took a look. Really new to this, and kinda clueless. Does it help if I try to trigger it for you? not sure how to even trigger the build, to be honest. It just shows 'deploy to" buttons everywhere…
It seems the problem is we are unable to get ssh setup for bamboo on devtest05.openmrs.org. The problem is where to find the right ssh public key for bamboo.
It seems that under CI administration we really should have a shared credential for “bamboo” setup to control ssh keys. Since for each environment it is possible to specify ssh keys, it seems that this is what is being done instead.
I can’t find out any other place in bamboo to specify ssh credentials in the documentation except under shared credentials.
It may be a good idea when we have a solution for this problem to document the steps needed to setup up a server for CI deployment.
It would be in the bamboo user’s home directory under ~/.ssh/authorized_keys, where ~ stands for the user’s home directory. We really should start using containers for our CI environment.
I can see that the build https://ci.openmrs.org/deploy/viewDeploymentResult.action?deploymentResultId=81133694 was running on machine gw108.iu.xsede.org. It was failing on executing the ssh to devtest05.openmrs.org from the command line.
When using the scp/ssh command, the keys would be indeed used from ~/.ssh/config| ~/.ssh/id_rsa The public key is in https://github.com/openmrs/openmrs-contrib-bambooagent/blob/master/files/ssh/id_rsa.pub
Those machines are initially set with ansible (I believe) and then we run puppet (https://github.com/openmrs/openmrs-contrib-bambooagent for a readme). There are two users running bamboo, bamboo-agent-1 and bamboo-agent-2, per machine. The ssh config is replicated in both users by puppet.
When using SSH and SCP commands, using regular bash, those things are configured directly on the agent (the actual machine running the builds), not on the bamboo UI or any place on bamboo server.
Why would that help with SSH keys?
Probably wouldn’t – was likely OT – sorry.
Thanks for pointing this out. I confirmed that the public keys match and, just to be certain, I installed the private key on a local docker container and confirmed I could successfully ssh to devtest05,
When I scan failed logins on devtest05 with lastb -w | grep bamboo I get this:
bamboo-agent-2 ssh:notty 149.165.228.106 Thu May 19 12:58 - 12:58 (00:00)
That explains the problem – i.e., the agent is trying to ssh with the agent-specific username “bamboo-agent-2”. Shouldn’t these agents be using “bamboo” as the username when accessing target machines?
So, when you run a command like ssh server, it will use the current running user (on this case, either bamboo-agent-1 or bamboo-agent-2). You can run for example whoami to find out the user if desired.
If you want to use a different user when ssh’ing to the remote host, you can either add to the command line ssh user@server, or you can configure that in the ~/.ssh/config file (i.e., for that specific host, use this specific user).
(Other configuration we could add to the ssh/config is the -o StrictHostKeyChecking=no part, if desired. So you wouldn’t need to add on every single SSH command).
Note that the ssh/config file is configured by puppet (https://github.com/openmrs/openmrs-contrib-bambooagent/blob/master/files/ssh/config), so it shouldn’t be changed manually.
One tip I’m going to give is to not mix the SCP/SSH Bamboo tasks with SCP/SSH command line. SCP/SSH bamboo task will require you to include the key on Bamboo UI; the SCP/SSH command line will require the key to be present on the agent. I think mixing both leads to a lot of complications, as suddenly you have multiple places to update a key.
Thanks so much for your help @cintiadr. I’m stumbling my way through CI 101. I’m not sure how the Deployment Reference Application Deployment Project UAT 2.4 was created, but I used your advice to add the “bamboo” username to inline scripts and convert the SCP Task to an inline script and got the build to succeed. 
I took an extra step of moving the target username & server into variables set at the deployment project level. Not sure what conventions have been used for this (so, might be breaking them), but it seemed like a better approach than hardcoding user & server names in each step.
Thanks again!
-Burke
That’s a fair solution, so far every other deployment I saw was hardcoding it on the script, so I think it’s a good idea to use variables for that.