HFE: Better handling of timezones

Software Add-on Modules
,
1

Hey @mseaton and @mogoodrich,

So we have been running into issues with timezones and HFE. Actually we have been running into issues with timezones and the Ref App in general, but for the sake of concision I’ll focus on HFE only here.

In a nutshell the core issue is that the code is written in a way that assumes that client and server are on the same timezone, which is indeed a valid assumption for local deployments (most likely accounting for 99% of the installations count out there.)

One starts running into issues when 1) client and server are on different timezones and 2) date-times come into play. What I mean is that this is only relevant for actual timestamps type of data: visit start and end date-times, encounter date-times, obs date-times, … etc. This is largely irrelevant for simple dates where the time component doesn’t matter, or for times that are disassociated from a date (cfr TimeWidget).

We would like to start to introduce the possibility to switch on (through a new GP) the following rule: client and server to exchange date-times in UTC.
The consequence of this is that both client code and server code need to convert to/from UTC in a number of places, and it’s quite a hunt to identify those views and controllers. For now we have been focusing on encounter dates through HTML-755.

This new rule will be optional for backward compatibility reasons, but I wonder if you had any thoughts on this before we start actively submitting PRs in that direction?

Cc @jfigueiredo

1 Like
2

I hate timezomes! :slight_smile:

Yeah, I know we talked about this a few months back and I’ve been seeing the commits going into the branches and have been dreading when the PR comes along, but I know it’s got to be done!

My main concern (well, besides something inadvertently breaking because of the changes) is what I mentioned in the ticket:

I haven’t looked into it, but the ugliness might come with the fix… specifically, I think we only want to apply this in the case when the showTime is true, otherwise you want do want to ignore the time zone.

For instance, if you are not using showTime and your server is 3 hours behind the client and the client submits “2020-10-12 00:00” you don’t want the server to shift the time to “2020-10-11 21:00” an actually change the date of the encounter. Whenever you had clients that weren’t in the same time zone as the server it would cause havoc.

Fixing for the showTime case without breaking the above case might be problematic… we had multiple problems working around issues like this when working on the appointments module… (though that was a bit different use case)

Take care! Mark

3

Thanks @mogoodrich.

So yeah first to clarify that point in the case of <encounterDate/> nothing will change when showTime="false", there will be backward incompatible changes only when showTime="true".

Btw @ssmusoke, is that ok with you?

Then, I stated the rule incorrectly, sigh. We’ve been doing so many roundtrips on that one with @jfigueiredo that I ended up messing stuff up in my grand post. Here it is.

When the handling of timezones is turned on:

  1. The server sends UTC dates to the client.
  2. The server should assume that the client submits timezone’d dates.

The latter means that the server should expect to be submitted dates like this (just a sample format):

15/11/2020 18:25:58 +0500

In essence this means that the server expects three pieces of info when being submitted a date: date + time + timezone.
Parsing such submitted date string data into a java.util.Date object will set it to be on the server timezone automatically, at that point it is therefore good to be used on the server side.

I would like to know if people think that this pattern made of 1) and 2) sounds like the right approach? @burke @dkayiwa @ibacher?

P.S. I haven’t done that yet, but I ought to study REST WS to see 1) what it does with submitted dates, and 2) how it represents dates in its resources. But let’s say that for now it is out of scope for this thread that is about HFE.

4

I think I share some of Mark’s trepidation about this, but it’s also (to me) something we obviously need to support.

I’d recommend using ISO 8601 / RFC 3339 as the standard format, e.g., YYYY-MM-DDThh:mm:ss.sssZ+05:00. It looks a bit much, but it’s unambiguous and widely supported in most programming languages.

Is there any particular reason not to expect the client to return UTC dates? There’s a standard JS function to return a UTC string.

My only other concern would be the amount of code we’re potentially going to need two branches for. Obviously, one HFE tag or even HFE in general probably isn’t too large of a surface area, but the more parts of the application we cover, the more complexity we add to the code.

This also reminds me that I should probably verify whatever it is that the FHIR module is actually doing with dates and times is correct for FHIR (which would be sending them in the ISO 8601 format I noted above).

1 Like
5

Thanks @ibacher, and in particular for sharing about ISO 8601 / RFC 3339, that’s definitely useful (for later at this stage of our WIP PR). I just kept the format simple just for the sake of the example in my post, I just wanted something readable.

Having a backend that can deal with client-submitted timezone’d dates (rather than always UTC dates) will “maybe” lead to less refactoring.
But otherwise no, there is no good reason here. I’m just trying to find a consensus around a pattern to adopt. So if people say that the client should send over UTC dates to the backend, we can also stick to that as well.

Others? @burke @dkayiwa @mogoodrich @mseaton

And yes it would good to validate that the FHIR2 module operates on sound standards as well :+1: I am actually worried about what may (not) be happening in REST WS…

6

My bias would be to move toward an assumption that all timestamps should follow the RFC standard for formatting, we should promote use of explicit timezone, and any timestamp without an explicit timezone is assumed to be UTC. For backwards compatibility, we could create a global property to override the default timezone to something other than UTC.

7

I generally agree with the sentiments here and have been deferring somewhat to @mogoodrich , who has often worked through these issues at PIH.

It makes sense to me to ensure that date/time widgets send enough data to be unambiguous, and UTC / ISO 8601 makes sense to me.

There are some things in HFE that would need more thinking through - in particular there are lots of widgets that are rendered server-side (including date widgets, particularly in view mode) - which might not have an easy way to render with the client’s timezone unless the implementation was changed to render client-side. Maybe this is a non-issue but struck me as something to look out for.

Mike

8

Thanks everyone! So, a couple of takeaways. People seem to agree on the following:

  • The server should send UTC dates to the client.
  • The client should send ISO 8601 / RFC 3339 formatted dates to the server.
    • This implies that the client does not always send UTC dates to the server but more generally that the client sends dates in a wide format that contains all necessary info about date + time + timezone.
    • This implies that the server is responsible for parsing a timezone info that may (or may not) be present in those ISO 8601 / RFC 3339 formatted dates.
  • This handling of timezones will be controlled by (at least) one GP.

Agreed?

Now specifically about HFE. @mseaton when you say this:

The way I look at this is that what’s rendered on the server side is a server response to the client. Therefore the dates that the response may contain should be UTC’d.

9

I wouldn’t really characterize it this way, as HFE doesn’t return a response that contains structured data - just the html to render. But yes, we need to return the html in such a way that any dates in the html are formatted based on the client timezone.

What I was trying to note is that currently the view mode for a date widget in HFE does this on the server side:

String toPrint = "";
if (initialValue != null) {
    toPrint = dateFormat().format(initialValue);
    return WidgetFactory.displayValue(toPrint);
} else {
    toPrint = "________";
    return WidgetFactory.displayEmptyValue(toPrint);
}

So we’ll either need to find a way for the client to pass the server it’s timezone and use this in our server-side date formatting, or as you suggest we need to change this widget’s “toPrint” to something that can be recognized and operated on in javascript when the client renders. For example, we could return something like:

return "<time datetime=\"" + isoFormattedDate + "\"></time>";

(or don’t use time, and use standard div and classes, depending on browser support).

And then add a function to the htmlForm.js that is invoked during htmlForm.initialize and looks for all these tags, grabs the datetime value, formats it, and populates the body of the tag with the formatted date.

There are other solutions, just wanted to explain what I meant.

Mike

10

The latter is exactly what we’ve done so far in our WIP:

  • Server-rendered content contains UTC dates.
  • Client code and libraries (such as htmlForm.js) take care of making sure dates show in the client timezone.
2 Likes
11

My feeling is that the server shouldn’t have to know about the client timezone ahead of sending data to it. This doesn’t feel right to me, but I may be wrong.

12

Time zones are such a pain!

I started writing a long post, but I ended up with "what you said above sounds like the best approach to me:

13

Actually, @mksd what was the reason that server-rendered content contains UTC dates, rather than just making sure that server-rendered content includes dates with properly-formatted timezone components (likely the ISO standard) so that client code and libraries can make sure that dates show in the proper timezone for the client?

I’m wondering because in the case where something is missed/not converted, using the server time zone seems like it would be “more correct” than UTC for most implementations.

14

@mogoodrich :slight_smile: I know, I thought of you when I came across this:

15

Thanks for sharing, looking forward to watching! Need to carve out some time to do it later though so I don’t get sucked even deeper in this morning… :slight_smile:

16

I think that most frontend libs just expect that: a server that “speaks UTC” to them. @nmalyschkin @florianrappl, coud you confirm?

I’m not sure, really depends where the server is :slight_smile:

17

@mogoodrich I had asked the question to @florianrappl last year here, quoting:

The frontend should deal with UTC exclusively, but always display times local to the user (i.e., potentially configured, otherwise as inferred by the browser similar to the right locale).

1 Like
18

I was thinking for the typical use case, where the server is in the same country… but if the standard is that the frontend should deal with UTC exclusively, I’m fine with that.

19

If the override GP was set to a non-UTC timezone OR a request to the server supported specifying a timezone (e.g., the client said give me timestamps in EAT and the server honored it), then server responses could be in a non-UTC timezone in some cases. I think it’s more important that timestamps from the server contain a timezone.

The primary problem is having timestamps without a timezone (regardless of client or server side). We should aim to eliminate any code that generates timestamps without an explicit timezone (whenever we are refactoring existing code or writing new code). A timestamp without a timezone is inherently ambiguous. Even with assumptions perfectly defined and client & server running on the same machine in a single building, timestamps without an explicit timezone can cause (potentially dangerous) problems twice a year in a location that observed daylight savings time.

tl;dr – it’s not timezones that cause the pain, but the lack (and therefore assumptions) of timezones.

p.s. ( I’m not saying the UI and users must always deal with timezones; rather, our code – client-side or server-side – shouldn’t be communicating timestamps under the hood without a timezone. I’d love to have all timestamps in the database as actual timestamps with explicit timezone… maybe for Platform 2.5, @mksd. :wink: )

20

That’s debatable though. One could also say that you can as well write nothing rather than writing +0000. In other words, in a system that handles timezones, no timezone means UTC.