Streamlining the test automation using Jira

Development Quality Assurance
, ,
4

Hello all, I have discovered that the current Zephyr version installed on the OpenMRS Jira instance is Zephyr for JIRA 5.6.4, which is a discontinued edition of Zephyr. Moreover, Zephyr no longer provides support for that version.

Therefore, currently, there are two editions of Zephyr available:

  1. Zephyr Squad: This test management solution works within Jira and allows teams to plan, execute, and report on their tests.
  2. Zephyr Scale: This standalone test management solution helps teams create, manage, and automate their testing process.

I have briefly reviewed the documentation of both solutions and also tried installing them on a Jira instance. The main difference between the two is that Zephyr Squad enables tests and executions to work as Jira tickets, while Zephyr Scale is an isolated plugin that doesn’t directly work with Jira tickets.

As for my comparison of Xray and Zephyr, here are my findings:

Similarities between Zephyr and Xray:

  • Both offer basic features such as creating, editing, and cloning tests.
  • Both have a visual step editor.
  • Both natively support Gherkin.
  • Both have hierarchical test organization.
  • Both integrate with Jenkins and Bamboo.
  • Both offer licenses for open-source organizations.
  • Both have test plans (Zephyr Squad doesn’t have this).

Advantages of Xray over Zephyr:

  • Xray can use test sets for organizing, whereas Zephyr does not have this feature.
  • Xray uses different Jira ticket types (Pre-Condition, Test, Test Set, Test Execution, Test Plan) for managing tests, while Zephyr only uses the Test ticket type.
  • Xray has test plans, which Zephyr Squad does not have.
  • With Xray, we can report issues and bugs in a very detailed manner while executing tests.

Advantages of Zephyr over Xray:

  • The test step UI is simpler and easier to use than in Xray. The Xray UI is more advanced since it has many features.

Based on the comparison between Xray and Zephyr, it seems that Xray has several advantages over Zephyr, including the ability to use test sets for organizing, the use of different Jira ticket types for managing tests, the presence of test plans, and the detailed issue and bug reporting. Therefore, my recommendation would be to use Xray for test management in OpenMRS Jira.

References: