Xstream serialization of a ReportData instance

mksd · February 19, 2016, 12:26pm

Hi all,

Could anyone shed some light for me on serialization/Xstream? I marshalled an instance of ReportData, and I obtained such lines:

<org.openmrs.module.reporting.report.ReportData>
  <definition>
    <uuid>00c75fb1-e68c-4f32-9077-fabec935e8ca</uuid>
    <name>Patient History</name>
    <creator class="org.openmrs.User_$$_jvst9b6_41" resolves-to="org.openmrs.User">
      <uuid>709bd76e-3f84-11e5-b4c6-06f9acffb513</uuid>
      ...

Then when I tried to unmarshall it I got the following error:

com.thoughtworks.xstream.converters.ConversionException: org.openmrs.User_$$_jvst9b6_41 : org.openmrs.User_$$_jvst9b6_41

Unsurprisingly this class org.openmrs.User_$$_jvst9b6_41 could not be resolved and I could get away with the snippet below when unmarshalling:

XStream xstream = new XStream();
xstream.alias("org.openmrs.User_$$_jvst9b6_41", User.class);
xstream.omitField(User.class, "log");
xstream.omitField(Person.class, "log");
xstream.omitField(Person.class, "deathdateEstimated");

However it seems that this should precisely be addressed by Xstream through the resolves-to="org.openmrs.User" attribute, and it does not. Am I wrong?

Also as you can see I had to omit some fields here and there (thankfully irrelevant for what I was doing) but I have no idea why I had to that. If I were not to omit them, Xstream would complain that the fields could not be found.

lluismf · February 19, 2016, 1:11pm

I wouldn’t do that because this class is a dynamic proxy and its name is variable (created on the fly by Hibernate, extending User).

Can you add the whole stack trace? It’s not clear why the ConversionException is thrown.

I guess the problem is the serialization (must unproxy the persistent or at least get the superclass if it’s a proxy - you never know).

mseaton · February 19, 2016, 1:53pm

@mksrom, are you using xstream directly or is this happening through normal use of the serialization.xstream / reporting module API?

The first place I would start investigating is with versions. What version of OpenMRS, reporting, serialization.xtream, and Java are you running?

Mike

mksd · February 19, 2016, 2:53pm

Thanks @lluismf and @mseaton, see below.

http://paste.ubuntu.com/15132076/

I was using Xstream directly for both marshalling and unmarshalling.

lluismf · February 19, 2016, 3:39pm

I’m not sure how XStream does the marshalling, but if there’s a way of setting creator.class I’d do the following:

  Class clazz = (user instanceof HibernateProxy ? user.getClass().getSuperclass() : user.getClass());

mseaton · February 19, 2016, 3:45pm

@mksd, try using the ReportingSerializer rather than using XStream directly and let us know if you experience the same problems.

mksd · February 20, 2016, 10:10am

I have the feeling, looking at the output XML, that ReportingSerializer probably does the right thing. The problem is: I need to unmarshall it within JUnit… and I am then getting this:

com.thoughtworks.xstream.converters.ConversionException: Service not found: interface org.openmrs.module.reporting.report.definition.service.ReportDefinitionService

mseaton · February 20, 2016, 3:42pm

@mksd, not sure exactly what you are trying to do, but if you can share your junit test we can have a look. Presumably you have the reporting module as a dependency already if you are working with ReportData instances. You’ll need to make sure you are using a context sensitive test, and likely also need to ensure you have the reporting module hibernate mapping files referenced from your test resources like this:

github.com

openmrs/openmrs-module-reportingui/blob/master/api/src/test/resources/test-hibernate.cfg.xml

<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE hibernate-configuration PUBLIC
	"-//Hibernate/Hibernate Configuration DTD 3.0//EN"
	"http://hibernate.sourceforge.net/hibernate-configuration-3.0.dtd">

<hibernate-configuration>
	<session-factory>
        <mapping resource="ReportDesign.hbm.xml"/>
        <mapping resource="ReportRequest.hbm.xml"/>
    </session-factory>
</hibernate-configuration>

Mike

mksd · February 21, 2016, 8:02am

Thanks @mseaton. I followed your steps:

My test class extends BaseModuleContextSensitiveTest.
I added the test-hibernate.cfg.xml test resource.

My unit test actually doesn’t do much for now, it just attempts to unmarshall the XML representation of a ReportData instance:

public class MyTest extends BaseModuleContextSensitiveTest {
  
  @Before
  public void setUp() throws IOException, SerializationException {
    InputStream inStream1 = getClass().getClassLoader().getResourceAsStream("sampleReportData.xml");
    String strXml = IOUtils.toString(inStream, "UTF-8");
    
    ReportingSerializer serializer = new ReportingSerializer();
    ReportData reportData = serializer.deserialize(strXml, ReportData.class);
  }
}

sampleReportData.xml was serialized and saved to disk through running a report within OpenMRS and using ReportingSerializer of course.

I am now getting the error below:

com.thoughtworks.xstream.converters.ConversionException: null : null
---- Debugging information ----
cause-exception     : java.lang.NullPointerException
cause-message       : null
class               : java.util.LinkedHashMap
required-type       : java.util.LinkedHashMap
converter-type      : com.thoughtworks.xstream.converters.collections.MapConverter
class[1]            : org.openmrs.module.reporting.dataset.DataSetRow
converter-type[1]   : org.openmrs.module.serialization.xstream.converter.CustomReflectionConverter
class[2]            : org.openmrs.module.reporting.dataset.SimpleDataSet
class[3]            : org.openmrs.module.reporting.report.ReportData
version             : null
-------------------------------

P.S. Note that adding or not the test-hibernate.cfg.xml test resource has no effect on the above error.

mksd · February 21, 2016, 11:09am

@lluismf, your suggestion led me to look into customising the Xstream Converter to handle HibernateProxy instances specifically. Even better, I found the official Converter for it: com.thoughtworks.xstream.hibernate.converter.HibernateProxyConverter

However, although I can see that it produces a slightly different output XML, it still leaves Hibernate proxies class names in those <creator/> and <changedBy/> tags in the XML:

<org.openmrs.module.reporting.report.ReportData>
  <definition>
    <uuid>811b4cca-b6a4-45c1-acfe-c3049bb17d48</uuid>
    <name>Patient History</name>
    <creator class="org.openmrs.User_$$_jvst75e_41">
      <uuid>709bd76e-3f84-11e5-b4c6-06f9acffb513</uuid>
      <creator class="org.openmrs.User_$$_jvst75e_41" reference=".."/>
      <dateCreated class="sql-timestamp">2005-01-01 00:00:00.0</dateCreated>
      <changedBy class="org.openmrs.User_$$_jvst75e_41">
    ...

And so in the end the issue remains

dkayiwa · February 21, 2016, 11:25am

Is this in any way related to? Error: Unable to deserialize object

mksd · February 21, 2016, 12:09pm

Probably. So those funny named classes are not Hibernate proxies but Javaassist related objects… Anyway then I tried this both for marshalling and unmarshalling:

xstream.registerConverter(new CustomJavassistEnhancedConverter(new JavassistMapper(xstream.getMapper()), xstream.getConverterLookup()));

To no avail.

I could see that the <creator/> and <changedBy/> tags were still holding those funny org.openmrs.User_$$_jvst.. package names after marshalling (so I had not much hope regarding the way back anyway). Just to be sure, is this how I am supposed to use the custom Converter?

I am also on OpenMRS 1.10.+ so I guess it all makes sense… that it doesn’t work

On the other thread, when @mogoodrich says

Well, it is basically what I do when I register my alias before unmarshalling. See my initial message.

dkayiwa · February 21, 2016, 4:24pm

What are the exact versions of the xstream.serialization module and openmrs platform that you are using?

mksd · February 22, 2016, 2:51am

openmrs-core 1.11.4
serialization.xstream 0.2.10
reporting 0.9.8.1

But note that, set aside my last answer to Mike, I have been using Xstream directly.

dkayiwa · February 22, 2016, 7:19pm

@mksd can i look at sampleReportData.xml and anything else i would need to reproduce this locally?

mksd · February 23, 2016, 4:36am

You could use this: http://paste.ubuntu.com/15177033/

You don’t need anything else, just get this into a String str in a unit test or a console application and then

XStream xstream = new XStream();
xstream.alias("org.openmrs.User_$$_jvst705_41", User.class);
xstream.omitField(User.class, "log");
xstream.omitField(Person.class, "log");
xstream.omitField(Person.class, "deathdateEstimated");
reportData = (ReportData) xstream.fromXML(str);

The idea of course, is that this should just work with the two liner:

XStream xstream = new XStream();
reportData = (ReportData) xstream.fromXML(str);

But it doesn’t.

lluismf · February 23, 2016, 9:22am

This is wrong

As I said before, this class will not exist when deserializing.

dkayiwa · February 23, 2016, 10:17am

@mksd can we look at your unit test where you marshal with ReportingSerializer and get Service not found: interface org.openmrs.module.reporting.report.definition.service.ReportDefinitionService?

mksd · February 23, 2016, 11:02am

On the contrary, that’s the workaround that makes it work by telling Xstream to reconstruct org.openmrs.User_$$jvst70541 as a org.openmrs.User. It would fail without setting this alias. As I said earlier, it is the equivalent of Mark’s trick explained in the other thread.

The question is rather: how do we get Xstream to not generate those org.openmrs.User_$$jvst* markups when marshalling in the first place?

mksd · February 23, 2016, 11:08am

I provided you with an XML that was marshalled with Xstream directly, I believe you may need one that would be marshalled using ReportingSerializer then? Please confirm.

Please also have a look above at my last answer to @mseaton regarding my trials in that direction.