@dkayiwa I was not careful while moving the tickets from
In Progress to
Ready For Review.
I think that tells me what I was supposed to do while changing the tickets state. I am now a consultant and have already become smart in that angle
Here is the research on the tools above.
All the tools mentioned are open source frameworks though they differ in the features they offer in respect to API. Below is the summary of what each framework offers!
Swagger is a tool which supports a functional, performance, and security testing of API.
- API design and development
- API documentation
- API Testing
- API Mocking and Virtualizations
- API Governance and monitoring
Best For: API designing.
Karate is based on the cucumber library and supports a web service test written in a domain-specific language(DSL).
- Multi-threaded parallel execution is supported.
- Allows configuration switching.
- Reports generation.
- Supports reusing Payload-data for API testing.
Best For: Supports any language which can deal with HTTP, JSON, or XML
Jmeter supports cross-platform and works at a protocol layer.
- Allows use of various programming languages.
- Load and performance testing of many different applications, servers, and protocols.
- Allows to replay test results.
- Provides support for variable parameterization and assertions.
- Supports per-thread cookies.
- Configuration variables and a variety of reports are also supported by Jmeter.
Best For: load and performance testing of web applications.
Postman is an API development environment however, its not free. It has three pricing plans which I won’t go into details for now.
- Helps in automated testing.
- Assists in exploratory testing.
- Supports Swagger and RAML (RESTful API Modeling Language) formats.
- Supports knowledge sharing within the team
Best For: The tool is best for API testing
After looking into the existing tests in openmrs-module-webservice.rest, I noticed they are using REST-Assured framework and below are some of its features
- Seamless integration with the Serenity automation framework.
- Provides some baked-in functionalities.
- Supports the syntax of BDD Given/When/Then.
Thoughts are welcome on this as this is guiding our move into automated tests for REST API direction.
So, do you have a conclusion on what would be the best way forward?
From today’s QA call, two of the framework have been selected(Karate and REST-Assured) however, one of them will be opted for after more research is done finding out which of the two is better in terms of sustainability and can support multiple languages(basically Java and Js).
Hopefully the decision will be taken in the next QA Call.
We may harness this as well
One thing I didn’t see much of in your review above was on the test reporting from these tools, i.e., can they generate results in some kind of standard format that can be easily used by other tooling to indicate how many tests passed or failed and which tests? (Most helpful would probably be something like the JUnit format).
thanks, @tendomart this is really helpful
`<dependency>` ` ` `<groupId>net.masterthought</groupId>` ` ` `<artifactId>cucumber-reporting</artifactId>` ` ` `<version>` `3.8` `.` `0` `</version>` ` ` `<scope>test</scope>` `</dependency>`
We can generate reports in HTML format, json format ,XML format as well, though the configuration especially in test runner class seems to be little deeply than selenium
The way forward on the REST API automation is follows:
REST-Assured will be used to complete the automation of the pending REST API tests. The team agreed there was no need to change the framework as we already have a large number of the API tests automated using the framework. Additionally, the tool is working well for the community. @kdaud has already completed 7 test and we are pending 6 tests to complete the automation.
The review of the other suggested tools will be completed later on and the information kept in file for future reference or when a need arises .
All the REST API tests that were identified missing by the Platform team have been automated and now running on every PR triggered on the repo and on every merge. We added more few tests that were identified during the development.
Checkout the updated Spreadsheet to see these automated tests marked in green as
Done(second column) and the links to their implementation are provided in the first column. All the tests status are tracked on our QA-Dashboard under the REST API budge which tracks the current Bamboo build of the module.
Thanks to the QA Team that has worked around ensuring the project succeeds!