Test Coverage for our REST API

grace · September 16, 2021, 1:06am

This week, the topic of REST API Test Coverage was raised in both the PM Team and QA Support Team calls.

Key Concerns:

Historically, doing manual tests of our REST API delays our Platform releases (example of manual tests here).
There are some existing REST WS tests, like these (which are buried and hard to find), but it’s not clear how much coverage our REST API has.
Given that our REST API is something ALL OpenMRS system users leverage, we should feel confident in our coverage of REST WS.

So, we talked about the need for more REST API test coverage in the Platform Team call today. Here are the key takeaways and next steps:

Key Points:
- Agreed that the spreadsheet with manual API test calls can/should be automated
Action Items:
- First: Platform Team Members (e.g. @dkayiwa, others?) to take a dozen of these tests, i.d. most critical ones, cross-check that these exist in the existing RESTWS tests. Then, create tickets for those that are left out and still need to be added in to the RESTWS module.
- Then: QA Support Team Members can write tests for those that are missing.

Here’s the 15 min discussion of interest (I cropped this link so you don’t have to listen to the full hour unless you want to): 15 minute recording of REST API Test Coverage

@dkayiwa @ibacher @burke @tendomart did I cover that alright? Anything to add?

CC @christine @kdaud @sharif @pmanko @mozzy

kdaud · September 16, 2021, 6:29am

I haven’t studied this in too much detail when it comes to REST API tests, one thing I would like to know is whether there is any automated tests(testing) for the module ?

This suggests to me setting up automated configuration within the module and have automated tests written especially for critical functionalities. @ibacher @dkayiwa @sharif @bistenes @mksd does Selenium or Cypress apply in this case? If yes, which one would be more useful to achieve the goal in a more light-weight way?

@grace @dkayiwa would be a good idea to have these tickets on our QA-Kanban-Board distinctively labelled for members to pick them up and work on them.

dkayiwa · September 16, 2021, 10:20am

Fork and clone the module. Then run mvn clean install, and tell us if this answers your question.

No, because it is not user interface functionality.

sharif · September 16, 2021, 1:23pm

Hello @dkayiwa what would be the best approach in server url while carryout testing these rest Api end points, Using local instance or using cloud server like qa or demo since will be using postman client for querying Rest calls, GET, POST

Have also seen this rest client class that was already implemented acting as util class for rest calls.However am still skeptical whether we might need some more dependencies to cater for rest utils. If we are to write automated rest calls which module among qaframework, referenceapplication -distro or uitestframwork would be better to have these functionalities?

tendomart · September 16, 2021, 2:13pm

You’re absolutely right.

dkayiwa · September 16, 2021, 2:22pm

@sharif here is an example of tests in that spreadsheet that already exist in the module:

patient_search

github.com

openmrs/openmrs-module-webservices.rest/blob/2.31.0/omod-1.9/src/test/java/org/openmrs/module/webservices/rest/web/v1_0/controller/openmrs1_9/PatientController1_9Test.java#L68-L78

    
      
          	@Test
          	public void shouldGetAPatientByUuid() throws Exception {
          		MockHttpServletRequest req = request(RequestMethod.GET, getURI() + "/" + getUuid());
          		SimpleObject result = deserialize(handle(req));
          		
          		Patient patient = service.getPatientByUuid(getUuid());
          		assertEquals(patient.getUuid(), PropertyUtils.getProperty(result, "uuid"));
          		assertNotNull(PropertyUtils.getProperty(result, "identifiers"));
          		assertNotNull(PropertyUtils.getProperty(result, "person"));
          		assertNull(PropertyUtils.getProperty(result, "auditInfo"));
          	}

patient_create

github.com

openmrs/openmrs-module-webservices.rest/blob/2.31.0/omod-1.9/src/test/java/org/openmrs/module/webservices/rest/web/v1_0/controller/openmrs1_9/PatientController1_9Test.java#L89-L101

    
      
          	@Test
          	public void shouldCreateAPatient() throws Exception {
          		long originalCount = service.getAllPatients().size();
          		String json = "{ \"person\": \"ba1b19c2-3ed6-4f63-b8c0-f762dc8d7562\", "
          		        + "\"identifiers\": [{ \"identifier\":\"abc123ez\", "
          		        + "\"identifierType\":\"2f470aa8-1d73-43b7-81b5-01f0c0dfa53c\", "
          		        + "\"location\":\"9356400c-a5a2-4532-8f2b-2361b3446eb8\", " + "\"preferred\": true }] }";
          		
          		SimpleObject newPatient = deserialize(handle(newPostRequest(getURI(), json)));
          		
          		assertNotNull(PropertyUtils.getProperty(newPatient, "uuid"));
          		assertEquals(originalCount + 1, service.getAllPatients().size());
          	}

And those tests run automatically for each pull request.

It is basing on the above that i conclude that our efforts are better spent on ensuring that each of the tests in that spreadsheet has a corresponding test in the module.

sharif · September 16, 2021, 2:35pm

Ohh we simply mock a desired get/post request

Thanks alot for your clarification , we have a clear direction now

tendomart · September 16, 2021, 2:51pm

@ibacher since the future for Platform is fhir first does this apply for fhir as well ?

ibacher · September 16, 2021, 4:38pm

Yes, though we have a pretty exhaustive suite of automated tests for the FHIR2 module already. See here for example.

cioan · September 17, 2021, 12:32pm

We have used cypress in a few instance to test the REST API. Here are some examples:

Login at location:

github.com

PIH/openmrs-pwa-apzu-ic3/blob/master/cypress/support/commands.js#L236

    
      
          
          
});
          
          
Cypress.Commands.add('openmrsapi', () => {
          
          
  cy.visit(Cypress.env('apiServer'));
            cy.get('#login-button').should('exist');
          
          
});
          
          
Cypress.Commands.add('loginwithrestapi', () => {
          
          
  cy.request({
              method: 'POST',
              url: Cypress.env('apiServer') + '/ws/rest/v1/appui/session?v=ref',
              options: {
                headers: {
                  'Accept': 'application/json, text/plain, */*',
                  'Content-Type': "application/json"
                }
              },

Retrieve login locations:

github.com

openmrs/openmrs-owa-labworkflow/blob/master/cypress/support/commands.js#L36

    
      
          
          
import { MOCKS } from "./constants";
          
          
Cypress.Commands.add(
            "login",
            (
              username = Cypress.env("username"),
              password = Cypress.env("password"),
            ) => {
              const token = window.btoa(`${username}:${password}`);
              cy.request({
                method: "GET",
                url: "/openmrs/ws/rest/v1/session",
                headers: {
                  Authorization: `Basic ${token}`
                },
              });
            }
          );
          
          
Cypress.Commands.add("logout", () => {

sharif · September 17, 2021, 12:52pm

Ohh Awesome ,Thanks @cioan , and well be back its been quite some time seeing your response

tendomart · September 20, 2021, 9:18am

Wow this is good to see and emulate

grace · September 21, 2021, 5:09pm

Update: our QA Support Team is considering Swagger, Karate, Jmeter, or Postman to support our API Test approach.

@christine @kdaud @sharif can you provide further info about this, and then perhaps interested parties in this thread can also provide feedback?

grace · September 21, 2021, 5:13pm

Also: @dkayiwa @burke @ibacher any updates on this action item from last week’s Platform Team call?

Platform Team Members (e.g. @dkayiwa, others?) to take a dozen of these tests , i.d. most critical ones, cross-check that these exist in the existing RESTWS tests. Then, create tickets for those that are left out and still need to be added in to the RESTWS module.

The QA support team lead @christine is keen to see these tickets so we can delegate to support team contributors.

tendomart · September 21, 2021, 9:25pm

@grace also inquiring about timelines , would love to see this coincide with alpha and beta testing

bistenes · September 21, 2021, 11:38pm

@sharif @tendomart Cypress is for testing user interfaces. @dkayiwa is right. I imagine that @cioan wrote that test in Cypress only because we were already using Cypress to do E2E testing of something with an interface, and the REST API test was tacked on. Don’t use Cypress for this.

grace · September 22, 2021, 10:30pm

Great update from @dkayiwa on the Platform Team call today:

The Platform Team folks have gone through all the Platform 2.4.0 REST-WS endpoint tests in this spreadsheet, and they have identified all the ones that already have test coverage. This uncovered that 14 of 79 API use cases still need automated test coverage.

You can see how in the spreadsheet, they have linked the API cases to the specific area with the test coverage; these examples can be used to assist contributors who want to work on the remaining coverage gaps:

@christine @kdaud @sharif - what do you think, should we create Jira Issues for each of the 14 remaining things that still need coverage?

sharif · September 23, 2021, 3:32am

Oh sure, i think its worth creating jira issues for the remaining un covered tests.

kdaud · September 24, 2021, 4:46am

While creating tickets to cover the remaining REST API cases, I notice that I can not move any ticket to READY FOR WORK on QA Kanban Board. Most probably I am less privileged on the RESTWS module tracked by Jira and guessing @grace is the best person to help here!

grace · September 25, 2021, 12:44am

@kdaud I’ve updated your privileges, you should now be unblocked