I'm making simple http service with Spring Boot RestController, and what I was found, when I try to request via GET Json object I didn't get content-length in header and transfer-encoding becomes chunked.
With simple ResponseEntit<String> all headers set as expected.
What kind of problem may lead to this behavior?
Ths is not a problem, Transfer-encoding chuncked and no content length means that response was compressed. If compression is enabled in Spring boot it will compress responses larger than certain amount (2048 bytes by default). I think your ResponseEntit<String> is simply smaller than required for compression.
You can read more about compression settings in documentation.
If you want consistency you can either disable compressing, or set server.compression.min-response-size to a very small value. But I would suggest to keep it as is.
Related
How do I disable the transfer-encoding chunked in a spring boot web app (2.7.1) or latest version?
I already have these properties.
server:
compression:
enabled: true
min-response-size: 1024
mime-types: application/json
I also send the Accept-encoding header as gzip.
Even though I see the gzip in the response header, I do not think that it is really compressed (with or w/o gzip, the response size is same with 150 KB). I also do NOT want transfer encoding chunked.
UPDATE: I have another webflux application which works great. I do not see chunked. gzip works just great. This is not anything to do with postman.
when I do not send the accept-encoding header
for the same request, when I send the accept-encoding header
Your screenshot seems to be from Postman, which shows decompressed size and not the compressed size.
Here's a screenshot from Postman for my Gzip enabled REST service
Content-Encoding is telling client that it content transferred was gzip compressed. It may be incorrect, if you're just setting this header without compressing the response content. However, since you're relying on Spring to do this, it's most likely correct and response is compressed.
Transfer-Encoding just means that response may be sent into multiple HTTP packets. This is done when server does not know the size of response content in advance, before it starts writing to ServletOutputStream. Server can know the size of response content in two ways. First is to buffer serialized response into memory and then get it's byte array length. Second one is to discard serialized content while calculating the size. In the second scenario, server will have to serialize the response twice, once to find size and once to write to ServletOutputStream. As you can see that both of these methods to find response size in advance are not desirable. Server will set this Transfer-Encoding to chunk, even when your response is not Gzipped. That's because JSON serialization in Spring is handled in such a way to allow response to be written to ServletOutputStream as soon as any part of JSON becomes available in the buffer. This way server can serve first byte faster. It will happen even for the smallest response. Having Transfer-Encoding: chunked doesn't mean that your response is not compressed. What it means is that Gzip bytes are written to ServletOutputStream as soon as they become available
Postman shows decompressed size only. You can see actual transferred size and decompressed size in Google Chrome separately.
Alternatively you can see both sizes using CURL commands
Here are the commands in text
#Find uncompressed size
curl -so target/test.json localhost:8080/order -w "%{size_download}"
#Find compressed size
curl --compressed -so target/test.json localhost:8080/order -w "%{size_download}"
I was reading Google performance document about HTTP caching at https://developers.google.com/web/fundamentals/performance/optimizing-content-efficiency/http-caching, This document says that we should use ETags when possible. I am using ASP.NET Web Api 2.2. I am now trying to implement ETag in all of my public apis. I am thinking to implement ETag using MD5. I mean, I will hash the json response on each request using MD5. Is there any performance hit of using MD5.calculateHash on each request? My json response size are not too big(within the range of 1 to 20KB).
Yes there will be a performance hit. Calculating a hash takes time. However, you may find that the cost of calculating that hash is not significant in comparison to the performance saved by transferring unchanged bytes over the wire to the client.
However, there is no guarantee that you will get a perf improvement with Etags. It depends on many things. Are you going to regenerate the Etag on the server on every request to compare it with the incoming request? Or are you going to create a cache of etag values and invalidate them when the resource changes?
If you are going to regenerate the etag on every request then it is possible that the time spent pulling data from the database and formatting the representation will be significantly higher than the time it takes to send a few bytes over the wire. Especially if the entire representation can fit in a single network packet.
The key here is to ask if you really need the performance gain of Etags and is it worth the cost of doing the implementation. Setting cache control headers to enable client side private caching may give you all the benefits that you need without having to implement etags.
I have a number of posts that go into this subject in more detail:
http://www.bizcoder.com/using-etags-and-last-modified-headers-to-improve-performance-with-http-conditional-requests
http://bizcoder.com/implementing-conditional-request-handling-for-your-api
Today, I wanted to utilize the Accept-Encoding header to request an image as base64. Come to find out, the XMLHttpRequest spec prevents setting that header!
http://www.w3.org/TR/XMLHttpRequest/#the-setrequestheader()-method
Note: The above headers are controlled by the user agent to let it control those aspects of transport. This guarantees data integrity to some extent. Header names starting with Sec- are not allowed to be set to allow new headers to be minted that are guaranteed not to come from XMLHttpRequest.
Why in the world would they write a spec like this? It'd make more sense if the browser just provided a default value (eg. gzip,deflate,sdch) if none was specified.
The browser is responsible for accepting and processing the response. It wouldn't make much sense to manipulate your XHR to say it accepts gzip, for example, when you couldn't do anything with it. Can you just set a custom header value?
Why in the world would they write a spec like this?
In one word: laziness.
Why add in extra semantics describing secure predictable browser behavior for every possible header that would only be used by a few people like us, when they can just declare all the headers prohibited in one paragraph.
Has anyone used OkHttpResponseCache on an authenticated HTTPS connection successfully? I'm facing this strange behavior where the second time content is served from cache it return garbage instead of the correct content.
The service I'm using as an example is protected with HTTP basic auth and accessed over HTTPS. I added the must-revalidate response header to allow the cache to store the response. It uses ETags as a mechanism for cache validation.
It works perfectly for the first cached response:
1 - First time I make a service call, the server return 200 OK. Debugging the response cache source code I could see the response if being passed to the put() method (which stores it in the file store).
2 - The second request is made. Server is hit and returns a 304 response. Checking the cache headers and step debugging confirmed me that the server indeed returned 304. One strange behavior tough: on the second response (now served by the cache), the ETag header now contains the value duplicated (instead of having a single value, it has this value in a list twice).
3 - I make a third request. Same behavior as above, same "duplication" of the ETag value but when I get the input stream, instead of the correct json text, I get garbage (as in a bunch of black diamonds with an interrogation inside).
I don't know if I'm doing something wrong (when configuring the cache). I don't know if this is an encoding problem, or if the cache tried to update the file store and messed the input up. I suspect after the first cache response (in the second attempt) the presence of a second ETag value invalidates the headers and the cache attempts to store the response once again and ends messing it up. I wasn't able to step debug through it to confirm yet.
I tried to "translate" the garbage text with UTF-8, 16, ASCII, ISO to no avail. I tried using HTTP instead of HTTPS and got the same behavior.
Did someone encounter something similar and was able to solve it? Is it a bug in the cache or could I be doing something wrong?
There was a bug in the way that OkHttp updates the Content-Encoding: gzip header when doing conditional GETs. It's fixed in the OkHttp 1.3.
I have a TIBCO BW process which starts with the 'HTTP Receiver' Activity.
I'd like to obtain (via a custom Java Code activity) the size of the original HTTP request.
(The purpose is to collect statistics, measure response times, request/response sizes etc.)
I'd like to measure the data in bytes and not in characters, is it possible to get the request
as a byte array, ServletInputStream or something similar?
I don't think you can get the original request from Java code - simply because you cannot access the HTTPServletRequest object of the HTTP Receiver from a Java activity. If you'd like to access the raw request data, you may write a proxy servlet and access BusinessWorks via this proxy servlet.
If your task is only to get stats on the request size, there is a simpler solution. You can use the "Content-length" header parameter for this reason (http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html).
The Content-Length entity-header field indicates the size of the entity-body,
in decimal number of OCTETs, sent to the recipient
Content-length is an output parameter of the HTTP receiver. You may need to add the length of "RequestURI", "PostData" and "Header" parameters as they are not part of the entity body.
Update: PostData is part of entity-body, therefore its size is included in Content-length