@cintiadr, would it be possible to create a dedicated bamboo agent, which could be used exclusively by a new performance test task? The agent would need to run on a dedicated machine without any other heavy processes so we get reliable results.
Should be possible, I believe. I’d ask @burke about allocation in Jetstream, but from my point of view it should be alright.
I’d argue this machine could potentially run other builds too (unless you have reason to be strongly against it). While one a single build should run on that machine, while it’s idle I don’t see why other builds shouldn’t run there.
But reality is that, while the work is not so hard (it would be go to terraform, setup a new machine that now bootstraps ansible AND puppet), I will not have time to do that any time soon.
On the other hand, I’m happy to help someone willing to go that path.
hi @raff, great initiative! Could you share a bit on what tools are you considering (maybe http://gatling.io/ ?) and what things you would like to measure?
@cintiadr, I guess it’s okay to run other builds while the agent is idle. I’m not sure how Jetstream allocation works, but the main point here is to set it up in a way, which guarantees consecutive builds of the same commit will run in same conditions (server and tests will be run on the same agent to ignore network) and tests complete in similar time with ~5% difference at most.
I guess we could start from running them on existing agents and see if they are reliable enough. I’ll reach out to you for guidance regarding setting up a dedicated agent if needed.
@teleivo, I wanted to focus on testing REST endpoints so that we can cover platform, RA and any other distribution with a single test suite, which could be then extended. I haven’t decided on which framework to use. I’ll be looking at https://www.joecolantonio.com/2017/07/18/open-source-performance-testing-tools/ to start with. Gatling is on that list. I appreciate any suggestions.