Error: Report Evaluation Failing for XLS reports (with Core 2.4)

icrc.thonorio · May 11, 2021, 9:09am

Hi,

I took a look into the Reporting Module, and noticed that I was not able to generate reports of type XLS.

Running:

Core 2.4.0
Reporting Module 1.22.0-SNAPSHOT

The error details can be seen here:

And the view logs here:

At first glance, this seems to be related with OpenMrs 2.4, given that I was able to create a simple XLS report using OpenMrs 2.3.

Do you have any clue @dkayiwa, @mksd, @mseaton

icrc.thonorio · May 11, 2021, 9:23am

For reference,

I tried to upgrade POI/POI-OOXML version from 3.12 to 3.15, but even then was not able to get past this.

Also, don´t know if its related to the current issue, but may be a clue. If I try to run a report using the reportingUI interface, the error details will show the following error instead:

gcliff · May 11, 2021, 11:45am

@icrc.thonorio i see this in your error logs

java.lang.NoClassDefFoundError: Could not initialize class org.apache.poi.openxml4j.opc.internal.marshallers.ZipPackagePropertiesMarshaller

which means the JVM was unable to find the required class, so check classpath configurations to make sure required classes are not missed.

Also kindly share your server side logs here to

Thanks

mseaton · May 11, 2021, 12:33pm

@icrc.thonorio what are the full details of your environment? War version, all modules and versions, java version, operating system?

It looks to me like maybe something in your environment is causing a different XML parser or different set of POI libraries from getting loaded than expected.

The reporting module hasn’t been updated to have any build profiles against the 2.3.x, 2.4.x, or 2.5.x lines of OpenMRS core yet (unfortunately). It would be great if someone had the bandwidth to add these profiles into the reporting module and then corresponding build plans into Bamboo as is done with the 1.9-2.2 version we are currently building and testing against. This would help identify this kind of problem sooner.

Mike

icrc.thonorio · May 24, 2021, 11:04am

Hi @mseaton,

You can find the environment information in the link bellow:

Also, this environment may have to much clutter, if you feel the need, I can ty to use the reference application for further testing.

icrc.thonorio · May 24, 2021, 2:18pm

Hi @gcliff

By doing a quick search in the imported dependencies I’m able to found the class ZipPackagePropertiesMarshaller as a part of POI

And since the POI version at rune time seems to remain the same, I´m not sure why its not found.

ibacher · May 24, 2021, 3:03pm

The key part here is the Could not initialize class which is different from a plain NoClassDefFoundError. NoClassDefFoundError means that a valid class couldn’t be found and is raised either when the class isn’t present on the classpath or, as in this case, when there’s an exception thrown when trying to initialize the class (in which case the classloader can’t load the class definition).

As @mseaton said, this is likely because of a different XML parser being loaded. Specifically, the ZipPackagePropertiesMarshaller extends the PackagePropertiesMarshaller which has this static initializer block, which is likely throwing an exception.

Unfortunately, without the full log it’s hard to tell more than that. There’s likely to be a “Caused by” entry later in the stack trace you provided a snippet of that might have more detail on what happened.

If I had to make a guess this looks like a commit that might’ve broken things (assuming 1.21.0 works). (The reasoning here is that adding a module to the aware of list makes that’s module’s classpath part of the existing module’s classpath, so it’s entirely possible that after that change, it’s loading some other class from it’s changed classpath than it was using before).

icrc.thonorio · May 31, 2021, 3:44pm

Hi @ibacher

I tried to test if the problem was related to the version 1.22.0-SNAPSHOT by running 1.21.0.

But still seem to have the same issue.

On another note, Not sure if this may be relevant, but I notice that the error I get is different the first time I try to run a report.

The Error in question:

mksd · June 1, 2021, 10:44am

@mseaton for the record, we have decided to do just that. Hopefully errors will popup that way that can be solve through debugging Spring tests.

@icrc.thonorio will start looking into adding a 2.4 profile first and will ask for help on the Slack channel if necessary.

mseaton · June 1, 2021, 12:08pm

That’s great - thanks @mksd and @icrc.thonorio ! Definitely reach out on slack with any issues.

icrc.thonorio · July 7, 2021, 3:15pm

Hi @mseaton, @mksd, @dkayiwa

I did create 2 new profiles (for 2.3 and 2.4), however the api-test seems to throw a lot of ERRORS.

Because of that, I needed to Update the file TestingApplicationContext (can be seen bellow) and add a new ReportTestDataset-openmrs-“VERSION”.xml file.

This solved the great majority of the Errors but 7 of them still stand.

All seem to be related with “EvaluationException: Failed to evaluate Error evaluating because: null”

One of the failing tests:

public void evaluate_shouldReturnAllOrdersForAVisit() throws Exception {
		VisitEvaluationContext context = new VisitEvaluationContext();
		context.setBaseVisits(new VisitIdSet(1));
		OrderForVisitDataDefinition d = new OrderForVisitDataDefinition();
		EvaluatedVisitData vd = Context.getService(VisitDataService.class).evaluate(d, context);
		Assert.assertEquals(expectedOrders.intValue(), ((List) vd.getData().get(1)).size());
}

Pom File:

<profile>
    <id>2.3</id>
    <properties>
        <openMRSVersion>2.3.0</openMRSVersion>
        <openMRSMinorVersion>2.3</openMRSMinorVersion>
        <jacksonVersion>1.9.13</jacksonVersion>
        <reportingApiConditionalArtifactId>reporting-api-2.2</reportingApiConditionalArtifactId>
    </properties>
</profile>
<profile>
    <id>2.4</id>
    <properties>
        <openMRSVersion>2.4.0</openMRSVersion>
        <openMRSMinorVersion>2.4</openMRSMinorVersion>
        <jacksonVersion>1.9.13</jacksonVersion>
        <reportingApiConditionalArtifactId>reporting-api-2.4</reportingApiConditionalArtifactId>
    </properties>
</profile>

TestingApplicationContext: (Similar issue Problem running tests against OpenMRS Core 2.3.0- - #3 by mogoodrich)

<property name="packagesToScan">
    <list>
        <value>org.openmrs</value>
    </list>
</property>

dkayiwa · July 7, 2021, 4:28pm

Do you have a draft pull request that reproduces it?

icrc.thonorio · July 8, 2021, 9:34am

Hi @dkayiwa

I just created the following PR: RA-1947 - Added profiles for Openmrs platform 2.3 and 2.4 by icrc-toliveira · Pull Request #217 · openmrs/openmrs-module-reporting · GitHub

Also I should also specify that the majority the tests I did where using the profile for 2.4.

dkayiwa · July 8, 2021, 11:01am

I have just pulled the changes in your pull request and locally run mvn clean install successfully.

icrc.thonorio · July 8, 2021, 2:08pm

Hi @dkayiwa

I´m not to familiar with maven profiles, but I don´t think this one is executed from the get-go

How it looks from my side if I don´t enable it (Intellij):

I have been using this mvn package -P 2.4 to run the mentioned profile

mseaton · July 12, 2021, 8:29pm

@icrc.thonorio - I also get errors while trying to build with your 2.4 profile, but that’s not surprising to me. I do get different errors than you were describing - mine appear to be related to Mockito version changes.

jnsereko · July 16, 2021, 1:14pm

Hello @dkayiwa @mseaton @icrc.thonorio @ibacher @sharif @isears

I am trying to implement anti-csrf strategies to make OMRS more secure. Below is a PR of what i have done in core.

github.com/openmrs/openmrs-core

MM-860: Implemented anti-csrf tokens for OpenMRS protection against CSRF attacks

openmrs:master ← jnsereko:MM-860

opened 01:43PM - 02 Jul 21 UTC

jnsereko

+216 -1

## I have created a repository class that extends CsrfTokenRepository, to creat…e, save and generate a web token associated to the HttpServletRequest. ## I have added a filter to create a cookie having a the token value ## Issue I worked on see https://issues.openmrs.org/browse/TRUNK- ## Checklist: I completed these to help reviewers :) - [x] My IDE is configured to follow the [**code style**](https://wiki.openmrs.org/display/docs/Java+Conventions) of this project. No? Unsure? -> [configure your IDE](https://wiki.openmrs.org/display/docs/How-To+Setup+And+Use+Your+IDE), format the code and add the changes with `git add . && git commit --amend` - [x] I have **added tests** to cover my changes. (If you refactored existing code that was well tested you do not have to add tests) No? -> write tests and add them to this commit `git add . && git commit --amend` - [x] I ran `mvn clean package` right before creating this pull request and added all formatting changes to my commit. No? -> execute above command - [x] All new and existing **tests passed**. No? -> figure out why and add the fix to your commit. It is your responsibility to make sure your code works. - [x] My pull request is **based on the latest changes** of the master branch. No? Unsure? -> execute command `git pull --rebase upstream master`

My ambition is to create a csrf token from core, which can also be accessed in the referenceapplication-module on the login page, to verify that the request is coming from an authorised source.

I think what i added in core works, but the reference application was using an earlier version on openmrs so i decided to update it to the version having my security configurations, 2.5-SNAPSHOT This also raised the need to change the version of some modules to the latest release.

Currently seeking, i am facing an error allergy_reaction refers to an unmapped class: org.openmrs.Allergy - Pastebin.com Even adding the “package to scan” property didn’t work.

<property name="packagesToScan">
    <list>
        <value>org.openmrs</value>
    </list>
</property>

If anyone is interested i reproducing it locally, here are my changes in this module

Any suggestions are warmly appreciated. Thanks

mseaton · July 16, 2021, 2:37pm

@jnsereko - these errors can be annoying to track down. What I’m seeing in your error log is this:

[jar:file:/home/joshua/.m2/repository/org/openmrs/module/appframework-api/2.16.0/appframework-api-2.16.0-tests.jar!/TestingApplicationContext.xml]: Invocation of init method failed; nested exception is org.hibernate.MappingException: An association from the table allergy_reaction refers to an unmapped class: org.openmrs.Allergy
Caused by: org.hibernate.MappingException: An association from the table allergy_reaction refers to an unmapped class: org.openmrs.Allergy

This tells me that it isn’t the referenceapplication module’s TestingApplicationContext.xml file that is at issue still, but rather the one in a dependency - specifically in the appframework module. It is likely that this module has not yet been upgraded to support building against OpenMRS core 2.5. See here:

github.com

openmrs/openmrs-module-appframework/blob/master/api/src/test/resources/TestingApplicationContext.xml

<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
	xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:context="http://www.springframework.org/schema/context"
	xmlns:util="http://www.springframework.org/schema/util"
	xsi:schemaLocation="http://www.springframework.org/schema/beans
                http://www.springframework.org/schema/beans/spring-beans-3.0.xsd
                http://www.springframework.org/schema/context
                http://www.springframework.org/schema/context/spring-context-3.0.xsd
                http://www.springframework.org/schema/util
                http://www.springframework.org/schema/util/spring-util-3.0.xsd">

    <bean id="sessionFactory" class="org.openmrs.api.db.hibernate.HibernateSessionFactoryBean">
        <property name="configLocations">
            <list>
                <value>classpath:hibernate.cfg.xml</value>
                <value>classpath:test-hibernate.cfg.xml</value>
            </list>
        </property>
        <property name="mappingJarLocations">
            <ref bean="mappingJarResources"/>

This file has been truncated. show original

Hope this helps, Mike

jnsereko · July 16, 2021, 3:41pm

Hey @mseaton This makes alot of sense. Maybe understanding it will help.

This was really difficult to track on my side

jnsereko · July 19, 2021, 11:22am

Hello @mseaton I have added the packagesToScan property to the appframework and changed its version in the reference-application-module to 2.17.0-SNAPSHOT, so that this change is accessed.

However, i am having an error

Caused by: org.hibernate.cache.CacheException: 
net.sf.ehcache.CacheException: Another CacheManager with same name 'hibernateCache' already exists in the same VM. Please provide unique names for each CacheManager in the config or do one of the following:
1. Use one of the CacheManager.create() static factory methods to reuse same CacheManager with same name or create one if necessary
2. Shutdown the earlier cacheManager before creating new one with same name.
The source of the existing CacheManager is: DefaultConfigurationSource [ ehcache.xml or ehcache-failsafe.xml ]
Caused by: net.sf.ehcache.CacheException:

the full log is CacheManager with same name 'hibernateCache' already exists in the same VM. - Pastebin.com

I have spent three days now finding possible ways of resolving this but all resources I have accessed haven’t solved my issue.

below are my attempts;

Attempt one; Making the cache shareable from core openmrs-core/CacheConfig.java at master · openmrs/openmrs-core · GitHub through cacheManagerFactoryBean.setShared(true); to make this shared
Attempt two; tried adding the ehcache.xml file and the ehcache-api.xml to the TestingApplicationContext.xml like;

.......
        <!--  default properties must be set in the hibernate.default.properties -->
    </bean>
    <bean id="hibernateCache"  class="org.springframework.cache.ehcache.EhCacheManagerFactoryBean" scope="singleton">
         <property name="shared" value="true"/>
         <property name="configLocation">
              <value>classpath:ehcache.xml</value>
         </property>
     </bean>
............

How can i solve this? cc @ibacher @dkayiwa @mksd and anyone else.

Thank you so much