CI Builds Failing

(I just send the following message to the helpdesk, but for reference am posting here as well)

CI buildings don’t appear to currently be working.

I made some commits over the past 24 hours but they appear to not have triggered builds.

I tried manually triggering a job and it failed with the following error:

System Error Details

https://pastebin.com/qtKF3HNp

Since @cintiadr is travelling, @whiscard are you able to help with this? I see alerts for a number of build agents that went offline.

@burke, @pascal or @whiscard, can you restart bamboo server (and then the agents)?

It’s on the wiki. It should fix the problem

On it… 

Bamboo and four agents restarted.

Looks like some one needs to again restart bamboo and or its agents.

You are correct, bamboo agents need to be restarted, none of them are available.

There was a provider outage yesterday and I believe at least network connectivity to all machines was cut for almost 2 hours yesterday.

That cause a tsunami of alerts as you can imagine, and agents didn’t reconnect.

I should be able to look into it in a hour or so. I wouldn’t mind if someone is willing to automate that!

The 4 agents are back online.

Thanks @cintiadr for looking into this. :slight_smile: