Last year, a team of QA community members; @christine, myself, @ivanc, @jennifer, @janflowers invested in directing our Quality assurance community efforts, the team created a testing prioritisation criteria, a POC etc.
There wasn’t lots of community activity on this project as we had initially thought but there is a rising need of supporting the release process with more automation coverage.
The developed POC added cucumber onto our radar; this was to introduce behavior driven development (BDD) in our QA picture, there was need to work with business people and users in end to end testing and the team advised that adopting a BDD framework was a way forward considering it’s adoptability by others like OpenHIE.
This year our community is preparing a fellowship program to boost our QA for the reference application and we would still want to rightly support everyone’s needs while not using our efforts for what won’t be well adopted.
Considering our community comprising of more technical persons who have been picking interest in QA, do you still see the need to proceed with cucumber which adds another layer of complexity to our automation framework in order to involve business people, properly document test cases and provider better reporting?
Do you feel like writing both the BDD phrasings (feature descriptions) and automated selenium test cases is a lot more work we are taking up for less benefits or you would find use of this work?
I welcome your thoughts on this as this is guiding our next QA fellowship direction.
CC: @dkayiwa, @burke, @rafal, @wyclif…
I read a blog of some developer who put it this way:
I’ve done several projects that were using cucumber, yet never have I seen it used beneficially. The business people have never asked me about my step definitions in cucumber or anything like that. The only question that business people ask is: “Are you ready for production?”.
Then another blog put it as below:
Keep in mind, however, that in some cases the business side of the organization may not have input or opinions as to how the application tests behave. It may also be true, depending on the size of your QA team or who is doing the automation work, that an extra layer of tooling to make tests more humanly readable (as with Cucumber) will not add value. In both cases, using Cucumber at the test creation phase may not be necessary.
Am not undermining the value of Cucumber, but simply putting it in the context of our community. I like the way Martin Fowler starts answering when asked for advice in regards to a tool , technology, pattern, or anything. He starts with, It Depends. LimitationsOfGeneralAdvice
Therefore, given the fact that it takes an effort to not only create but also maintain Cucumber features, i would not divert our limited and scarce resources, for what our business people are not interested in. If they are, is it worth the effort? Let us instead use these resources to catch real bugs before they are released in production.
@k.joseph, I am interested in learning more about automation testing so that i actively support during the release period. Could you kindly support me in this process?
@dkayiwa, do you see any technical need for test case documentation through tools like Zephyr in our infrastructure and publishing clearer results e.g. Cucumber Reports. I believe even the technical person needs a way to track the test cases.
In a meantime we have a growing number of contributors picking interest in the new qaframework with cucumber and i have noticed cucumber helping them start easily.
To others here (@janflowers@jennifer@christine etc) we had one call where we had some of the key community persons recommending building off the existing selenium tests vs the new cucumber framework. Any thoughts on this, we would have to update the old code base to fix all the tests and build more tests during our upcoming QA fellowship and the fellowship would only offer selenium skills for the ref app and cypress for ocl in addition to expanding on fellow’s ref app and ocl skills.
This is the best place we’ve documented what these report screens could look like. Super helpful. Thank you!!
I especially adore the idea that we could see in the QA Framework repo the simple “everything looks good” message - imagine if we could have that for all 3 major User-facing apps: the 2.x RefApp, 3.x RefApp, and OCLOMRS, e.g.:
I’ve used the Report screenshots you added in our QA E2E tests process slide here: QA Test Processes at OpenMRS - Google Slides Hopefully this helps make the end-to-end process more clear.
I removed CucumberStudio for now though since I don’t think we’re using it, at least not for now. Can always add it back in.
Just to clarify: A user can still dig into which specific BDD step is failing using Cucumber Reports, right? That seems like the main benefit that Cucumber Studio would give (showing exactly which steps have failed); if Cucumber Reports enables that, then I think we’re well covered for the business-person-persona for now. This demo site at https://reports.cucumber.io/
makes me think it’s covered but wanted to check with you.
