For years, we’ve relied on XStream for object serialization and deserialization, primarily to and from XML. It’s served us well with its flexibility and ease of use. However, as our system evolves toward modern standards — particularly Java upgrades, improved performance, and tighter security and safety — we’re adopting Jackson, a faster and more robust and maintained serialization framework.
@burke’s post above outlines this well, and I support the direction. Based on that, here’s what we’ll be doing:
Build a Jackson-based API (using JSON) to closely mirror the XStream API, integrated into serialization.xstream within OpenMRS Core.
Create a migration module to help older implementations transition gracefully to the new API.
Let me re-echo some of the reasons for opting in Jackson
I’m happy to share that we’ve made progress on the migration from XStream to Jackson!
A new PR has been opened that accomplishes the first step toward adopting Jackson-based serialization and deserialization in OpenMRS. Here’s a quick summary of what’s been accomplished:
JacksonSerializer: A new OpenmrsSerializer implementation using Jackson for robust (de)serialization.
UuidReferenceModule: A Jackson module for resolving references to OpenMRS domain objects by UUID during deserialization.
DomainService and DomainFetcher: Utilities to dynamically resolve domain object fetch/get methods from services, making UUID-based entity hydration flexible and reusable across types.
These components establish a cleaner, more maintainable serialization foundation while allowing for a smooth transition from XStream. The design also supports future extensibility.
@dkayiwa this has been relatively straightforward for the Address Hierarchy module since it doesn’t require many changes. In parallel, I’ve been working on the Reporting module — first updating it to run on the latest OpenMRS Core. Once that’s done, I’ll integrate Jackson-based serialization into it and can accompany that with a pull request for testing purposes.
I could submit an early pull request without tests, but I’d prefer to first ensure that the OpenMRS Core (de)serialization API is properly integrated. This would help avoid unnecessary rework later on.
Here’s the draft PR — it still needs some cleanup and a few additional changes in OpenMRS Core, particularly around the Address* objects, which are currently tightly coupled to XML.
Thanks @ruhanga . I had understood the goal / approach of this initiative was not to change the serialization format from xml to json and thus result in an incompatibility with existing configuration files, but rather to simply change the serialization library from one that uses xstream to one that uses jackson (eg. using jackson-dataformat-xml).
Thanks for your input @mseaton. While it’s true that the initial goal was to migrate the serialization library from XStream to Jackson, JSON was preferred as the target format because it’s widely appreciated for its readability and user-friendliness, and it’s where Jackson truly excels.
That said, regardless of whether we used Jackson with XML or JSON, a transformation or migration step would still be necessary. Jackson’s handling of XML differs significantly from XStream’s, so existing XML would need to be adapted either way to conform to Jackson’s structure and expectations. Choosing JSON helps us move forward with a more modern and developer-friendly format, while still requiring similar migration effort. Also as part of this effort, there will be a one-time migration module to help with the transition.
I understand there may be unforeseen risks with this approach, and I’d appreciate any insights you might have. This could help us refine the direction to better align with the original goal while addressing any potential issues early on.
I’d generally agree that json is preferred over xml these days. I’m not really sure why or if Jackson excels more at json than at xml. They are both structured-text representations of an object. The complexity comes in the deserialization, and determining what types of objects to construct and how to construct them, from text representations, regardless of whether that is a json or xml format.
I think this may be true for some of the implicated areas, most notably with the reporting module or with metadatasharing. And we will need to make a plan for those specific areas. But for other areas, like core global properties, and addresshierarchy configuration, I think things are pretty well controlled and known, and supporting the exact same xml format with Jackson should not be difficult at all.
I would recommend a more step-wise approach, where we do not remove the xstream capabilities or API from core (and thus do not introduce backwards incompatibility with known and unknown dependencies on this), but rather work to identify each of the places where xstream is currently utilized and migrate each to use Jackson as it’s serialization engine in turn. If - for a given use case - we feel that the right approach is to include a migration and convert from xml to json, then that can be taken as it comes.
Mostly makes sense @mseaton. Thanks for sharing above.
Sorry, I should clarify — what I meant is that JSON has become more widely adopted and is better supported within the Jackson ecosystem. Jackson was originally built with JSON in mind, so its core features are more robust and flexible in that context. In contrast, XML support was added later as an extension, and while functional, it’s likely to come with limitations and caveats — especially when trying to replicate XStream’s more dynamic XML handling or convention.
This is all fair. My main concern is simply to ensure that we do not introduce any impediments to existing implementations to continue to upgrade to the latest versions of OpenMRS when needed, and so I think we need to be very cautious and deliberate where we might introduce backwards incompatibilities if we are doing so.
I am personally in support of a world in which we migrate away from xml and towards json, but this is not a requirement for solving the crux of this ticket, which is about security rather than syntax. If we want to migrate over to a json format because we think this is an improvement, that’s fine, especially if that also solves the security problem, but in many cases this may be something we can accomplish on a case-by-case basis within individual modules (eg addresshierarchy) or minor core features (like the address and name template global properties), by simply creating tickets to migrate these features to json, and which does not require any major new serialization framework additions or changes.
Absolutely @mseaton! And that’s exactly why one of the acceptance criteria on the ticket emphasizes supporting a smooth transition for implementations, primarily through a dedicated migration module…
An additional approach discussed during today’s Platform call was to include the migration logic as a utility method within OpenMRS Core. This method would eventually be deprecated, and affected modules could invoke it during their startup phase.
True — though even for the security concern, some attention to syntax is inevitable. Jackson cannot seamlessly handle XStream’s XML formats, making backward compatibility a challenge. That said, JSON does offer significant advantages, which is why it’s being considered as a better default moving forward.
IMO, while working on this, it has proven quite feasible to have this implemented in one go, through/along-side the main PR, and it’s already helped us better understand the impact of the migration effort.
That said, as you pointed out, and as discussed during today’s call — an alternative and perhaps more practical route may be to support both XStream and Jackson side by side for now. We could deprecate XStream in favor of Jackson for all new serialization, giving implementations time to adapt gradually. Given that most are not aggressively upgrading to newer ‘Java versions’ yet (which introduced the security risk and therefore the root motivation for the effort), this approach aligns better with their current needs while still laying the groundwork for future compatibility. I believe this would allow for a smoother, more flexible transition that better supports the pace at which different implementations are evolving.
@ruhanga - thanks for all of the thoughtful responses and sorry for the late reply. Clearly you are giving this the attention and thought it deserves. I’m happy with your approach, I just want to make sure we are giving backwards compatibility the proper consideration. Thanks for that, and I am happy to collaborate and/or review any PRs that come out of this.
