Enabling GZIP request responses

Development
1

Hello,

How can I set up GZIP request responses on OpenMRS? I’ve switched ‘Enabled’ to true in Settings/Gzip on my Administration Panel (and clicked ‘Save’ pop-up), but I still gets uncompressed JSONS in response (there should be Content-Encoded : gzip header, right?). I check response compression on sites like http://www.whatsmyip.org/http-compression-test/, and they all show up the same (negative) result.

Also, if I will compress my POST request body, will OpenMRS decode it? Or I’m forced to send uncompressed files? Thanks in advance!

3

Hi Krzysztof,

Our GZIPFilter is coded in a way that requires restarting the server for the change in the gzip.enabled property to take effect.

4

Thanks Rafał, it works! However, I found one issue, I guess. I will explain it on two cases:

  • After server restart, when I don’t put “Accept-Encoding: gzip” header into my requests, I can see that all responses are not compressed, which is good, but also, one request: http://devtest04.openmrs.org:8080/openmrs/ws/rest/v1/encounter?patient=74bde8fe-9be9-4c8d-bf1d-95108355cce8&encounterType=67a71486-1a54-468f-ac3e-7091a9a79584&v=custom:(obs:full)&limit=1&order=desc gives me an error (there was no error before restart): com.android.volley.NoConnectionError: java.io.EOFException.

  • But when I will put above header into request, all requests are GZIPE’d and I can succesfully decode them, despite the same request I pointed out, with is NOT Gziped (it gots Content encoded header though), It returns 304 status code (all the others return 200 or 201). Also if I try somehow to decode it with GZIP, I will get this error: IOException reading from GZIPOutputStream :unknown format (magic number 227b) If I will encode it (because my guess is, that is not compressed) I will recive around 1000 Bytes, when on the beggining it was 7776.

Currently to make it work, i simply decompress every request, where response status code is different than 304, and it works, but it seems that there is error somewhere on the server side.

Thanks in advance!

5

I guess that this problem may not be caused by server side, but by android client cachinig system (I’ve read about it here). Because my request did not changed since last time, device is not downloading new response (with would be GZIPed), but using one from local storage (that’s why 304). And this one is already unzipped. But still I don’t understand why it crashes when GZIP is turned off (NoConnectionError).

However, my second question is still valid - is it possible to send GZIPP’ed body in POST requests? I’ve enabled Content-encoded header as gzip, and encoded post body, but server returns 500:

Could not read JSON: Illegal character ((CTRL-CHAR, code 31)): only regular white space (\r, \n, \t) is allowed between tokens at [Source: org.apache.catalina.connector.CoyoteInputStream@f920003; line: 1, column: 2]; nested exception is org.codehaus.jackson.JsonParseException: Illegal character ((CTRL-CHAR, code 31)): only regular white space (\r, \n, \t) is allowed between tokens at [Source: org.apache.catalina.connector.CoyoteInputStream@f920003; line: 1, column: 2]

6

Please follow https://issues.openmrs.org/browse/TRUNK-4646 for adding support for gzip requests.

7

I don’t see a reason why OpenMRS should be dealing with this. Most web servers will provide this feature out-of-box. For example tomcat uses the following

compression="on"
compressionMinSize="2048" 
compressableMimeType="text/xml, application/json"

Add this to the tomcat Connector and you’ll get compressed json. May be you want to do something else that I’m not understanding?

8

@sunbiz, containers indeed have proprietary compression configuration. Some like jetty simply provide its own gzip filter class, which you have to add to web.xml (see http://stackoverflow.com/questions/26001872/how-to-enable-gzip-for-jetty-9, it’s particularly cumbersome as you must customize the .war file). Some like tomcat have specific connector settings that you mentioned. In OpenMRS we have GZIPFilter, which you can use regardless of the container. It’s simpler that way.

Krzysztof asks to also have a possibility to send gzipped requests (big REST POSTs or XForms), which is not supported by containers, but can be easily added to our custom GZIPFilter.

9

@raff, There is a risk of performance hit if we start running decompression on the server side. That is the reason why two-way compression is a bad idea. It increases the risk of DOS attacks by simply sending large posts that cannot be gzipped. The post size limit exists to mitigate this, but I still think its a bad idea to allow compressed requests. May be only on a specific endpoint/resource is acceptable?

10

Some discussion about compressing requests can be found at http://stackoverflow.com/questions/424917/why-cant-browser-send-gzip-request. Note that browsers do not send gzip requests and only clients like mobile apps that are aware of the fact that this feature is supported will use it.

@owais_hussain mentioned limiting the size of packets (especially since we frequently use long UUIDs) send over the mobile network as the important factor when implementing a mobile client for their use case.

The mobile client will only use gzip if a request is big enough and makes sense. But still we can add an option to enable this feature on the server side for specific resources only. A global property with comma separated path matchers will do the trick.