Can't create Obs through REST with many levels of groupMembers

Tags: #<Tag:0x00007f01bc45b590> #<Tag:0x00007f01bc45b4c8> #<Tag:0x00007f01bc45b400>

Hello, I’m not able to create Obs with groupMembers during creating Encounter. Firstly I was trying to create an encounter (/encounter endpoint) using following JSON in request:

{
   "encounterType":"72607bf4-28f6-4866-91d4-6798c0ea0ae1",
   "encounterDatetime":"2017-01-09T00:00:00.000+0100",
   "patient":"f045c657-ab4b-4309-b560-18f7d616e9f4",
   "provider":"f045c657-ab4b-4309-b560-18f7d616e9f4",
   "obs":[
      {
         "concept":"54915241-0aaa-4c33-b3bd-9213815a2454",
         "obsDatetime":"2017-01-09T00:00:00.000+0100",
         "groupMembers":[
            {
               "concept":"19d83cea-66a5-4bbf-a5a3-737b1be686f6",
               "obsDatetime":"2017-01-09T00:00:00.000+0100",
               "groupMembers":[
                  {
                     "concept":"b7d4fa65-2cc4-47c6-8196-2f3ffcdff318",
                     "value":"No",
                     "obsDatetime":"2017-01-09T00:00:00.000+0100"
                  },
                  {
                     "concept":"96bd2056-d77a-4438-ae10-92b872b0255d",
                     "value":"Yes",
                     "obsDatetime":"2017-01-09T00:00:00.000+0100"
                  }
               ]
            },
            {
               "concept":"f1b54f69-7169-4369-8f7a-722e96916abc",
               "value":"2nd",
               "obsDatetime":"2017-01-09T00:00:00.000+0100"
            },
            {
               "concept":"973fb9f2-114a-47c4-abe2-f2d2d9dcd799",
               "value":"Maybe",
               "obsDatetime":"2017-01-09T00:00:00.000+0100"
            }
         ]
      }
   ]
}

When I was using this method, Concepts with uuid’s b7d4fa65-2cc4-47c6-8196-2f3ffcdff318 and 96bd2056-d77a-4438-ae10-92b872b0255d aren’t created, and the both parents of these Concepts has a null value.

The next attempt was using a /obs endpoint. Firstly I was creating an encounter without observations, then the observations was assigning to this encounter by the /obs endpoint using following JSON:

{
   "concept":"54915241-0aaa-4c33-b3bd-9213815a2454",
   "encounter":"e42a4933-1cc6-4c76-9e91-24c4cfc53c8e",
   "obsDatetime":"2017-01-09T00:00:00.000+0100",
   "person":"f045c657-ab4b-4309-b560-18f7d616e9f4",
   "groupMembers":[
      {
         "concept":"19d83cea-66a5-4bbf-a5a3-737b1be686f6",
         "encounter":"e42a4933-1cc6-4c76-9e91-24c4cfc53c8e",
         "obsDatetime":"2017-01-09T00:00:00.000+0100",
         "person":"f045c657-ab4b-4309-b560-18f7d616e9f4",
         "groupMembers":[
            {
               "concept":"b7d4fa65-2cc4-47c6-8196-2f3ffcdff318",
               "encounter":"e42a4933-1cc6-4c76-9e91-24c4cfc53c8e",
               "value":"No",
               "obsDatetime":"2017-01-09T00:00:00.000+0100",
               "person":"f045c657-ab4b-4309-b560-18f7d616e9f4",
               "groupMembers":[]
            },
            {
               "concept":"96bd2056-d77a-4438-ae10-92b872b0255d",
               "encounter":"e42a4933-1cc6-4c76-9e91-24c4cfc53c8e",
               "value":"Yes",
               "obsDatetime":"2017-01-09T00:00:00.000+0100",
               "person":"f045c657-ab4b-4309-b560-18f7d616e9f4",
               "groupMembers":[]
            }
         ]
      },
      {
         "concept":"f1b54f69-7169-4369-8f7a-722e96916abc",
         "encounter":"e42a4933-1cc6-4c76-9e91-24c4cfc53c8e",
         "value":"2nd",
         "obsDatetime":"2017-01-09T00:00:00.000+0100",
         "person":"f045c657-ab4b-4309-b560-18f7d616e9f4",
         "groupMembers":[]
      },
      {
         "concept":"973fb9f2-114a-47c4-abe2-f2d2d9dcd799",
         "encounter":"e42a4933-1cc6-4c76-9e91-24c4cfc53c8e",
         "value":"Maybe",
         "obsDatetime":"2017-01-09T00:00:00.000+0100",
         "person":"f045c657-ab4b-4309-b560-18f7d616e9f4",
         "groupMembers":[]
      }
   ]
}

In response I’m getting 400 code and message:

Error while creating observation. Response body:
{
   "error":{
      "message":"Invalid Submission",
      "code":"webservices.rest.error.invalid.submission",
      "globalErrors":[
         {
            "code":"error.noValue",
            "message":"error.noValue"
         }
      ],
      "fieldErrors":{
         "groupMembers":[
            {
               "code":"Obs.error.inGroupMember",
               "message":"A member of this obs group has an error"
            }
         ]
      }
   }
}

After the addition of the required value to the observation, the most nested Observation are not created again.

There isn’t any stack traces. I’m working with version 1.12.0 Build 8f283e.

Is there some way to create the observations with more than one level of groupMembers? Can you help me to figure out where is the problem?

Regards!

Are you able to reproduce that problem on this server int-refapp.openmrs.org which runs a newer version of the rest module than yours?

I can’t reproduce this issue on this server, because when I’m sending following JSON to /encounter endpoint:

{
   "encounterType":"51617df0-d203-48c7-a8b2-a080c7809c99",
   "encounterDatetime":"2017-01-10T00:00:00.000+0100",
   "patient":"1e070c06-589a-40ed-8f79-168e011c607a",
   "provider":"af7c3340-0503-11e3-8ffd-0800200c9a66",
   "obs":[
      {
         "concept":"6e437431-f5eb-43b2-bb49-550a2a554aa9",
         "obsDatetime":"2017-01-10T00:00:00.000+0100",
         "groupMembers":[
            {
               "concept":"1d79a15e-0d4a-47f2-bc4a-9f98abec0977",
               "obsDatetime":"2017-01-10T00:00:00.000+0100",
               "groupMembers":[
                  {
                     "concept":"c4bbc28a-c692-452f-b361-41c653a67fb4",
                     "value":"Yes",
                     "obsDatetime":"2017-01-10T00:00:00.000+0100"
                  },
                  {
                     "concept":"c204edc8-ebb8-4b02-91ea-cac95f0cf71b",
                     "value":"No",
                     "obsDatetime":"2017-01-10T00:00:00.000+0100"
                  }
               ]
            },
            {
               "concept":"7437bfbe-80d1-4085-a573-f8c56a9400fa",
               "value":"Maybe",
               "obsDatetime":"2017-01-10T00:00:00.000+0100"
            },
            {
               "concept":"d0c48e8d-c5f1-48c8-bf98-39440858a863",
               "value":"2nd",
               "obsDatetime":"2017-01-10T00:00:00.000+0100"
            }
         ]
      }
   ]
}

I’m getting an error:

{
  "error": {
    "message": "[provider on class org.openmrs.Encounter]",
    "code": "org.openmrs.module.webservices.rest.web.resource.impl.BaseDelegatingResource:697",
    "detail": "org.openmrs.module.webservices.rest.web.response.ConversionException: provider on class org.openmrs.Encounter\n\tat org.openmrs.module.webservices.rest.web.resource.impl.BaseDelegatingResource.getProperty(BaseDelegatingResource.java:697)\n\tat org.openmrs.module.webservices.rest.web.resource.impl.BaseDelegatingResource.setConvertedProperties(BaseDelegatingResource.java:600)\n\tat org.openmrs.module.webservices.rest.web.resource.impl.DelegatingCrudResource.convert(DelegatingCrudResource.java:103)\n\tat org.openmrs.module.webservices.rest.web.resource.impl.DelegatingCrudResource.create(DelegatingCrudResource.java:76)\n\tat org.openmrs.module.webservices.rest.web.v1_0.controller.MainResourceController.create(MainResourceController.java:92)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)\n\tat sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)\n\tat sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat java.lang.reflect.Method.invoke(Method.java:498)\n\tat org.springframework.web.bind.annotation.support.HandlerMethodInvoker.invokeHandlerMethod(HandlerMethodInvoker.java:177)\n\tat org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter.invokeHandlerMethod(AnnotationMethodHandlerAdapter.java:446)\n\tat org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter.handle(AnnotationMethodHandlerAdapter.java:434)\n\tat org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:943)\n\tat org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:877)\n\tat org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:966)\n\tat org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:868)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:650)\n\tat org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:842)\n\tat javax.servlet.http.HttpServlet.service(HttpServlet.java:731)\n\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)\n\tat org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)\n\tat org.openmrs.module.web.filter.ForcePasswordChangeFilter.doFilter(ForcePasswordChangeFilter.java:60)\n\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)\n\tat org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)\n\tat org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)\n\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)\n\tat org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)\n\tat org.openmrs.module.web.filter.ModuleFilterChain.doFilter(ModuleFilterChain.java:72)\n\tat org.openmrs.web.filter.GZIPFilter.doFilterInternal(GZIPFilter.java:64)\n\tat org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)\n\tat org.openmrs.module.web.filter.ModuleFilterChain.doFilter(ModuleFilterChain.java:70)\n\tat org.openmrs.module.webservices.rest.web.filter.AuthorizationFilter.doFilter(AuthorizationFilter.java:104)\n\tat org.openmrs.module.web.filter.ModuleFilterChain.doFilter(ModuleFilterChain.java:70)\n\tat org.springframework.web.filter.ShallowEtagHeaderFilter.doFilterInternal(ShallowEtagHeaderFilter.java:82)\n\tat org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)\n\tat org.openmrs.module.web.filter.ModuleFilterChain.doFilter(ModuleFilterChain.java:70)\n\tat org.openmrs.module.owa.filter.OwaFilter.doFilter(OwaFilter.java:57)\n\tat org.openmrs.module.web.filter.ModuleFilterChain.doFilter(ModuleFilterChain.java:70)\n\tat org.openmrs.module.xforms.web.XformsFilter.doFilter(XformsFilter.java:69)\n\tat org.openmrs.module.web.filter.ModuleFilterChain.doFilter(ModuleFilterChain.java:70)\n\tat org.openmrs.module.web.filter.ModuleFilter.doFilter(ModuleFilter.java:54)\n\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)\n\tat org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)\n\tat org.openmrs.web.filter.OpenmrsFilter.doFilterInternal(OpenmrsFilter.java:108)\n\tat org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)\n\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)\n\tat org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)\n\tat org.springframework.orm.hibernate4.support.OpenSessionInViewFilter.doFilterInternal(OpenSessionInViewFilter.java:150)\n\tat org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)\n\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)\n\tat org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)\n\tat org.openmrs.web.filter.StartupFilter.doFilter(StartupFilter.java:105)\n\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)\n\tat org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)\n\tat org.openmrs.web.filter.StartupFilter.doFilter(StartupFilter.java:105)\n\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)\n\tat org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)\n\tat org.openmrs.web.filter.StartupFilter.doFilter(StartupFilter.java:105)\n\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)\n\tat org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)\n\tat org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:88)\n\tat org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)\n\tat org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)\n\tat org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)\n\tat org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:218)\n\tat org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:122)\n\tat org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:505)\n\tat org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:169)\n\tat org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:103)\n\tat org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:956)\n\tat org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)\n\tat org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:442)\n\tat org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1082)\n\tat org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:623)\n\tat org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:316)\n\tat java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)\n\tat java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)\n\tat org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)\n\tat java.lang.Thread.run(Thread.java:745)\nCaused by: java.lang.NoSuchMethodException: Property 'provider' has no getter method\n\tat org.apache.commons.beanutils.PropertyUtilsBean.getSimpleProperty(PropertyUtilsBean.java:1127)\n\tat org.apache.commons.beanutils.PropertyUtilsBean.getNestedProperty(PropertyUtilsBean.java:686)\n\tat org.apache.commons.beanutils.PropertyUtilsBean.getProperty(PropertyUtilsBean.java:715)\n\tat org.apache.commons.beanutils.PropertyUtils.getProperty(PropertyUtils.java:290)\n\tat org.openmrs.module.webservices.rest.web.resource.impl.BaseDelegatingResource.getProperty(BaseDelegatingResource.java:691)\n\t... 78 more\n"
  }
}

Can you point out what is wrong with my request?

Remove the provider property because it was removed in platform 2.x which that server is running.

The error message suggests that one of the group members in the tree is invalid, it’s not like it’s because of the many levels of group members.

@mkruszynski can you verify if it works with two levels but fails with 3 levels? (Or rather, works with n levels but fails with n+1?)

@darius it is working for 2 levels, but for 3 it fails. When I want to create Observation with the most nested groupMembers from the latest above example (Concept with uuid 1d79a15e-0d4a-47f2-bc4a-9f98abec0977) I’m getting similar error like in first post - this Concept has no value. When I add some value to this Concept - groupMembers aren’t created. In API documentation value field is optional, why it is required here?

Also I want to mention that all Concepts used in my tries has type ‘Text’.

@mkruszynski do you think you can create a failing unit test for this?

@dkayiwa We have already created a test, the code is added below.

private Map prepareObservations() throws ConceptNameAlreadyInUseException {

    Map observations = new HashMap<>();

        String parent = createConcept("Concept1");
        String child1 = createConcept("Concept2");
        String child1Child1 = createConcept("Concept3");
        String child1Child2 = createConcept("Concept4");
        String child2 = createConcept("Concept5");
        String child3 = createConcept("Concept6");

        observations.put(parent + "/" + child1 + "/" + child1Child1, "Yes");
        observations.put(parent + "/" + child1 + "/" + child1Child2, "No");
        observations.put(parent + "/" + child2, "2nd");
        observations.put(parent + "/" + child3, "Maybe");

        return observations;
    }

    private String createConcept(String name) throws ConceptNameAlreadyInUseException {
        Concept concept = new Concept();
        ConceptName conceptName = new ConceptName(name);

        concept.setNames(Collections.singletonList(conceptName));
        concept.setDatatype(new Concept.DataType("TEXT"));
        concept.setConceptClass(new Concept.ConceptClass("Test"));

        return conceptService.createConcept(DEFAULT_CONFIG_NAME, concept).getUuid();
    }

    private void checkEncounterObs(List observations) {
        assertEquals(1L, observations.size());

        Observation observation = observations.get(0);

        assertNull(observation.getValue());
        isConceptEquals(observation, "Concept1");

        List groupMembers = observation.getGroupMembers();
        assertEquals(3L, groupMembers.size());

        for (Observation obs : groupMembers) {
            if (isConceptEquals(obs, "Concept2")) {
                assertNull(obs.getValue());

                List childGroupMembers = obs.getGroupMembers();
                assertEquals(2L, childGroupMembers.size());

                for (Observation childObs : childGroupMembers) {
                    if (isConceptEquals(obs, "Concept3")) {
                        assertEquals("Yes", childObs.getValue().getDisplay());
                    } else if (isConceptEquals(obs, "Concept4")) {
                        assertEquals("No", childObs.getValue().getDisplay());
                    } else {
                        fail();
                    }
                }
            } else if (isConceptEquals(obs, "Concept5")) {
                assertEquals("2nd", obs.getValue().getDisplay());
                assertEquals(0L, obs.getGroupMembers().size());
            } else if (isConceptEquals(obs, "Concept6")) {
                assertEquals("Maybe", obs.getValue().getDisplay());
                assertEquals(0L, obs.getGroupMembers().size());
            } else {
                fail();
            }
        }
}

private boolean isConceptEquals(Observation obs, String conceptName) {
        return StringUtils.equals(obs.getConcept().getDisplay(), conceptName);
}

In method prepareObservations we are creating an Observations structure for our system. Value in map is assigning to the most nested child. If something is unclear, I can explain workflow of this test.

@mkruszynski i mean tests that i can just directly (without any changes) run in the rest webservices module. Something like https://github.com/openmrs/openmrs-module-webservices.rest/blob/master/omod-1.9/src/test/java/org/openmrs/module/webservices/rest/web/v1_0/controller/openmrs1_9/EncounterController1_9Test.java or https://github.com/openmrs/openmrs-module-webservices.rest/blob/master/omod-1.9/src/test/java/org/openmrs/module/webservices/rest/web/v1_0/controller/openmrs1_9/ObsController1_9Test.java

@dkayiwa I made some sample test. I placed it on my fork of given repository. I hope it will be enough to find out this issue.

@dkayiwa, are you looking into that? If not, @adamg or @tmarzeion, could you please create an issue and fix Marcin’s test at https://github.com/openmrs/openmrs-module-webservices.rest/compare/master...mkruszynski:ObsTest

@mkruszynski in your unit test, you have a NullPointerException because your obs does not have a concept hence the failure at obs.getConcept().getDatatype()

@mkruszynski I’ve created ticket for this issue: RESTWS-636 Ill try to take a look on this issue soon

@mkruszynski @raff @dkayiwa I’ve fixed @mkruszynski’s test, so its possible to reproduce this issue via test. Good news is that this error is somewhere in REST-WS, not int CORE.

Great work @tmarzeion! :slight_smile: So are you gonna fix it?

Yeah I’ll try @dkayiwa

Thanks for help. Is it possible to put this fix in version 1.12 too? We would be thankful as we are using this version in our system.

Fixed and merged:

@mkruszynski you should be able to use the module as it is on your platform.