We are facing a strange issue in our environment while making POST requests with query param (?v=full) for encounters. The issue occurs specifically when there is an obs having a group member as valueConceptUUID of TRUE/FALSE.
Caused by: org.hibernate.LazyInitializationException: failed to lazily initialize a collection of role: org.openmrs.Concept.names, could not initialize proxy - no Session
Number of observations which we had so for while debugging:
The saving of encounter is done, we can see new entry in db. The error occurs when the saved object is getting converted to the FullRepresentation.
There is no error when we don’t append ?v=full, i.e. the DefaultRepresentation is working fine.
There is no error when make a GET request with param ?v=full.
There is no error when the value is not for a group member. (Top level obs works)
There is no error when there are no group member with valueConceptUUID of TRUE/FALSE.
We don’t have a consistent steps to reproduce this problem. Sometime it works fine, sometimes it doesn’t.
Note: We are using the CIEL concepts for TRUE/FALSE instead of openmrs default TRUE/FALSE concepts. We have updated the global properties for concept.true and concept.false to the CIEL conceptIds. Not sure this is something which is causing the problem.
Thanks for the response. Following are the versions:
OpenMRS Platform Version: 2.3.0 Build b3ade0
Rest Web Services OMOD Version: 2.28.0.df1459
Here is the full stacktrace:
converting class org.openmrs.Encounter to org.openmrs.module.webservices.rest.web.representation.FullRepresentation@70c04e83]","code":"org.openmrs.module.webservices.rest.web.ConversionUtil:409","detail":"org.openmrs.module.webservices.rest.web.response.ConversionException: converting class org.openmrs.Encounter to org.openmrs.module.webservices.rest.web.representation.FullRepresentation@70c04e83
at org.openmrs.module.webservices.rest.web.ConversionUtil.convertToRepresentation(ConversionUtil.java:409)
at org.openmrs.module.webservices.rest.web.ConversionUtil.convertToRepresentation(ConversionUtil.java:367)
at org.openmrs.module.webservices.rest.web.resource.impl.DelegatingCrudResource.create(DelegatingCrudResource.java:79)
at org.openmrs.module.webservices.rest.web.v1_0.controller.MainResourceController.create(MainResourceController.java:92)
at sun.reflect.GeneratedMethodAccessor922.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.springframework.web.bind.annotation.support.HandlerMethodInvoker.invokeHandlerMethod(HandlerMethodInvoker.java:177)
at org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter.invokeHandlerMethod(AnnotationMethodHandlerAdapter.java:446)
at org.springframework.web.servlet.mvc.annotation.AnnotationMethodHandlerAdapter.handle(AnnotationMethodHandlerAdapter.java:434)
at org.springframework.web.servlet.DispatcherServlet.doDispatch(DispatcherServlet.java:943)
at org.springframework.web.servlet.DispatcherServlet.doService(DispatcherServlet.java:877)
at org.springframework.web.servlet.FrameworkServlet.processRequest(FrameworkServlet.java:966)
at org.springframework.web.servlet.FrameworkServlet.doPost(FrameworkServlet.java:868)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:647)
at org.springframework.web.servlet.FrameworkServlet.service(FrameworkServlet.java:842)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:728)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:303)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.openmrs.module.web.filter.ForcePasswordChangeFilter.doFilter(ForcePasswordChangeFilter.java:60)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:52)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.icrc.pearlii.tomcat.filter.SameSiteFilter.doFilter(SameSiteFilter.java:28)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.apache.catalina.filters.HttpHeaderSecurityFilter.doFilter(HttpHeaderSecurityFilter.java:126)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.openmrs.module.web.filter.ModuleFilterChain.doFilter(ModuleFilterChain.java:73)
at org.openmrs.web.filter.GZIPFilter.doFilterInternal(GZIPFilter.java:65)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.openmrs.module.web.filter.ModuleFilterChain.doFilter(ModuleFilterChain.java:71)
at org.openmrs.module.webservices.rest.web.filter.AuthorizationFilter.doFilter(AuthorizationFilter.java:105)
at org.openmrs.module.web.filter.ModuleFilterChain.doFilter(ModuleFilterChain.java:71)
at org.openmrs.module.webservices.rest.web.filter.ContentTypeFilter.doFilter(ContentTypeFilter.java:64)
at org.openmrs.module.web.filter.ModuleFilterChain.doFilter(ModuleFilterChain.java:71)
at org.springframework.web.filter.ShallowEtagHeaderFilter.doFilterInternal(ShallowEtagHeaderFilter.java:82)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.openmrs.module.web.filter.ModuleFilterChain.doFilter(ModuleFilterChain.java:71)
at org.openmrs.module.owa.filter.OwaFilter.doFilter(OwaFilter.java:57)
at org.openmrs.module.web.filter.ModuleFilterChain.doFilter(ModuleFilterChain.java:71)
at org.openmrs.module.datafilter.web.DataFilterWebFilter.doFilter(DataFilterWebFilter.java:56)
at org.openmrs.module.web.filter.ModuleFilterChain.doFilter(ModuleFilterChain.java:71)
at org.openmrs.module.spa.filter.SpaFilter.doFilter(SpaFilter.java:54)
at org.openmrs.module.web.filter.ModuleFilterChain.doFilter(ModuleFilterChain.java:71)
at org.openmrs.module.referenceapplication.filter.RequireLoginLocationFilter.doFilter(RequireLoginLocationFilter.java:93)
at org.openmrs.module.web.filter.ModuleFilterChain.doFilter(ModuleFilterChain.java:71)
at org.openmrs.module.web.filter.ModuleFilter.doFilter(ModuleFilter.java:57)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.openmrs.web.filter.OpenmrsFilter.doFilterInternal(OpenmrsFilter.java:109)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.springframework.orm.hibernate4.support.OpenSessionInViewFilter.doFilterInternal(OpenSessionInViewFilter.java:150)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.openmrs.web.filter.StartupFilter.doFilter(StartupFilter.java:107)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.openmrs.web.filter.StartupFilter.doFilter(StartupFilter.java:107)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.openmrs.web.filter.StartupFilter.doFilter(StartupFilter.java:107)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.springframework.web.filter.CharacterEncodingFilter.doFilterInternal(CharacterEncodingFilter.java:88)
at org.springframework.web.filter.OncePerRequestFilter.doFilter(OncePerRequestFilter.java:107)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:241)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:208)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:219)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:110)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:492)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:165)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:104)
at org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:755)
at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:1025)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:116)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:452)
at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1195)
at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:654)
at org.apache.tomcat.util.net.AprEndpoint$SocketProcessor.doRun(AprEndpoint.java:2532)
at org.apache.tomcat.util.net.AprEndpoint$SocketProcessor.run(AprEndpoint.java:2521)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.lang.Thread.run(Thread.java:748)\nCaused by: org.openmrs.module.webservices.rest.web.response.ConversionException: converting class org.openmrs.Obs to org.openmrs.module.webservices.rest.web.representation.DefaultRepresentation@42d87c87
at org.openmrs.module.webservices.rest.web.ConversionUtil.convertToRepresentation(ConversionUtil.java:409)
at org.openmrs.module.webservices.rest.web.ConversionUtil.convertToRepresentation(ConversionUtil.java:373)
at org.openmrs.module.webservices.rest.web.resource.impl.DelegatingResourceDescription$Property.evaluate(DelegatingResourceDescription.java:249)
at org.openmrs.module.webservices.rest.web.resource.impl.BaseDelegatingConverter.convertDelegateToRepresentation(BaseDelegatingConverter.java:144)
at org.openmrs.module.webservices.rest.web.resource.impl.BaseDelegatingResource.asRepresentation(BaseDelegatingResource.java:381)
at org.openmrs.module.webservices.rest.web.ConversionUtil.convertToRepresentation(ConversionUtil.java:406)
... 90 more\nCaused by: org.openmrs.module.webservices.rest.web.response.ConversionException: converting class org.openmrs.Obs to org.openmrs.module.webservices.rest.web.representation.DefaultRepresentation@42d87c87
at org.openmrs.module.webservices.rest.web.ConversionUtil.convertToRepresentation(ConversionUtil.java:409)
at org.openmrs.module.webservices.rest.web.ConversionUtil.convertToRepresentation(ConversionUtil.java:373)
at org.openmrs.module.webservices.rest.web.resource.impl.DelegatingResourceDescription$Property.evaluate(DelegatingResourceDescription.java:249)
at org.openmrs.module.webservices.rest.web.resource.impl.BaseDelegatingConverter.convertDelegateToRepresentation(BaseDelegatingConverter.java:144)
at org.openmrs.module.webservices.rest.web.resource.impl.BaseDelegatingResource.asRepresentation(BaseDelegatingResource.java:381)
at org.openmrs.module.webservices.rest.web.ConversionUtil.convertToRepresentation(ConversionUtil.java:406)
... 95 more\nCaused by: org.openmrs.module.webservices.rest.web.response.ConversionException: converting class org.openmrs.Concept to org.openmrs.module.webservices.rest.web.representation.DefaultRepresentation@42d87c87
at org.openmrs.module.webservices.rest.web.ConversionUtil.convertToRepresentation(ConversionUtil.java:409)
at org.openmrs.module.webservices.rest.web.ConversionUtil.convertToRepresentation(ConversionUtil.java:373)
at org.openmrs.module.webservices.rest.web.resource.impl.DelegatingResourceDescription$Property.evaluate(DelegatingResourceDescription.java:252)
at org.openmrs.module.webservices.rest.web.resource.impl.BaseDelegatingConverter.convertDelegateToRepresentation(BaseDelegatingConverter.java:144)
at org.openmrs.module.webservices.rest.web.resource.impl.BaseDelegatingResource.asRepresentation(BaseDelegatingResource.java:381)
at org.openmrs.module.webservices.rest.web.ConversionUtil.convertToRepresentation(ConversionUtil.java:406)
... 100 more\nCaused by: org.openmrs.module.webservices.rest.web.response.ConversionException: display on class org.openmrs.Concept
at org.openmrs.module.webservices.rest.web.resource.impl.BaseDelegatingResource.getProperty(BaseDelegatingResource.java:762)
at org.openmrs.module.webservices.rest.web.resource.impl.DelegatingResourceDescription$Property.evaluate(DelegatingResourceDescription.java:245)
at org.openmrs.module.webservices.rest.web.resource.impl.BaseDelegatingConverter.convertDelegateToRepresentation(BaseDelegatingConverter.java:144)
at org.openmrs.module.webservices.rest.web.resource.impl.BaseDelegatingResource.asRepresentation(BaseDelegatingResource.java:381)
at org.openmrs.module.webservices.rest.web.ConversionUtil.convertToRepresentation(ConversionUtil.java:406)
... 105 more\nCaused by: java.lang.reflect.InvocationTargetException
at sun.reflect.GeneratedMethodAccessor1171.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.openmrs.module.webservices.rest.web.resource.impl.BaseDelegatingResource.getProperty(BaseDelegatingResource.java:753)
... 109 more\nCaused by: org.hibernate.LazyInitializationException: failed to lazily initialize a collection of role: org.openmrs.Concept.names, could not initialize proxy - no Session
at org.hibernate.collection.internal.AbstractPersistentCollection.throwLazyInitializationException(AbstractPersistentCollection.java:575)
at org.hibernate.collection.internal.AbstractPersistentCollection.withTemporarySessionIfNeeded(AbstractPersistentCollection.java:214)
at org.hibernate.collection.internal.AbstractPersistentCollection.initialize(AbstractPersistentCollection.java:554)
at org.hibernate.collection.internal.AbstractPersistentCollection.read(AbstractPersistentCollection.java:142)
at org.hibernate.collection.internal.PersistentSet.iterator(PersistentSet.java:180)
at java.util.Spliterators$IteratorSpliterator.estimateSize(Spliterators.java:1821)
at java.util.Spliterator.getExactSizeIfKnown(Spliterator.java:408)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:472)
at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:566)
at org.openmrs.Concept.getNames(Concept.java:992)
at org.openmrs.Concept.getNames(Concept.java:978)
at org.openmrs.Concept.getName(Concept.java:450)
at org.openmrs.module.webservices.rest.web.v1_0.resource.openmrs1_8.ConceptResource1_8.getDisplayName(ConceptResource1_8.java:466)
... 113 more"
Please, let us know if there are any more questions or any hints on what could be causing the problem.
This is a bug in the openmrs core platform. trueConcept, falseConcept, and unknownConcept are fetched and stored in static variables without initialising their lazily loaded properties.
For instance, the getTrueConcept can be seen at:
The randomness of the failure is caused by whoever makes the first call that loads the concept. If they access the lazy property, then you will not experience this problem accessing the same property in another call. But if the first caller does not, then any subsequent caller will see this error whenever they try to access any lazily loaded property that the first caller did not access. This is all due to the fact that each REST call opens a new session which is closed at the end of the call.
The problem is exposed by your full representation request which results into concept get display name and hence concept.getNames, a lazily loaded property. If this call was made first, after restarting OpenMRS, then you are good.
My recommendation is that these cached concepts should be fully loaded before they are stored.
An example of how to reproduce the same problem is by temporarily modifying this method in your PatientResource1_8 as here: https://pastebin.com/yGzTDEWr
Then fetching any patient with the default representation first, would result into an error for any subsequent fetching of a patient using the full representation. But if you refreshed spring, or restarted openmrs and started with fetching a patient using the full representation, all subsequent calls will pass. All because of accessing the lazily loaded property in the first call before the session is closed.
Thanks for detailed reply. If I understand it correctly these are the steps leading to this problem:
Concepts TRUE/FALSE has been cached as a static variable on the very first usecase. Probably because we need to use global property which could be costly if read everytime.
The lazy prperties(name details, member details etc) for these concepts gets cached based on the first time use of this concept. For example, if there was no use of concept.getname this information is not cached at all.
If in first call there was no use of concept.getname, this information would not be cached. All the subsequent calls which would try to use concept.getname would error out because it has not been cached and it will not be fetched again for performance reason unless restarted.
If above is the case:
We are not sure why it is working when the TRUE/FALSE has been provided as a top level obs vaue instead of a group member value.
We have also seen it working sometimes for TRUE concept at every level. Not sure if it was used somewhere before though.
If we make a POST with ?v=full it fails. After this if we perform a GET for the same object with ?v=full there is no error. However if we do the POST with ?v=full it fails again. Not sure if the same concept is being used in GET and POST both. Does GET not uses these static variables and directly fetched using hibernate?
Proposed workaround:
However, if we try to provide this workaround:
It would start failing for other lazy proeprties, i.e. members, answers since a ?v=full requires all the fields to be present. Please refer to screenshot with error in the first post.
As mentioned by you, it relies on the assumption that the first call are coming from REST module.
Would it be a good idea to either remove the caching OR make sure to make dummy calls to lazy properties getName, getMembers etc to make sure they are initialized fully. I will give these a try sometime.
Apologies for a long post with multiple questions. Please let us know your thoughts.
Are you in position to use version 2.3.1-SNAPSHOT of the platform to test this? If yes, create a ticket and i will show you what the fix could look like.
Sure, I can try upgrading to 2.3.1-SNAPSHOT on my local and try putting a fix. However, we are almost next to our release in the project so upgrading it on project might not be possible.
The proper fix should go to the openmrs core platform. But since this is out of the options because of your soon coming release, then you can do a hack in any module’s activator by doing something like this:
You would have to initialise any other lazily loaded properties that you are interested in, and also do the same for Context.getConceptService().getFalseConcept()
That way, all properties are loaded in the same session, before it gets closed.
If we were to do a fix in an , would it be similar to do in openmrs-core where we are initializing the boolean concepts. I will try to raise a PR on core as well.
The only difference would be that core would do it right from within the implementation of ConceptService().getTrueConcept(), when the first call is made.
