UTL_HTTP has a limit for the value of a cookie of 1024 characters - however I have cookies that exceed that length with a value of around 1700 characters. Is there any way I can submit an HTTP request via UTL_HTTP using cookies with a value greater than 1024 characters?
Was able to figure this out, at least enough to get it working in my case. Instead of using the UTL_HTTP.ADD_COOKIES / UTL_HTTP.GET_COOKIES procedures I'm reading/setting the HTTP headers manually. Looping through the response headers and parsing out the contents of the "Set-Cookie" header and then passing that in to the next request using the "Cookie" header via UTL_HTTP.SET_HEADER. Not very elegant, but it's working.
Related
Sometimes, when an object is not in the cache, varnish will send an object that has a real size smaller than the size declared in the content-length header. For example - only part of the picture.
Is it possible to construct such a rule...?
if (beresp.http.content-lenght <> real_object_body_size) { return(retry); }
I wrote a script that tests the same request against the varnish and the backend. It compares the downloaded size with the content-lenght header. The backend, unlike varnish, sometimes ends up with a timeout but the size is always fine. The problem is rare but annoying because the objects are set to long user cache time.
After a few days I can say that the problem was in occasional backend problems with varnish's ability to send a chunked transfer if the object is not in the cache.
Thank you #Thijs Feryn for pointing this out. I knew about that property but until I read it here, I didn't connect it to my problem at all.
It seems that "set beresp.do_stream = false;" solved the problem.
I need to limit the header size of request to 16KB in spring boot. I added the maxHttpHeaderSize flag to 16000 to do this.
I noticed that when I make a call from postman to test this out, the request starts failing(Request header is too large) from the time the header size is around 15KB. I was wondering what happens to the other 1KB?
This might be an issue with confusion of what a KB means. In your case, it's most likely the Base 2 value, which is 1024, not 1000. This means that 16000 bytes are actually around 15.7KB.
You can set the maxHttpHeaderSize either to 16 * 1024 = 16384, or just to 16KB, which should work if you have Spring Boot 2.1+
Also, this value limits not just one particular header field size, but the full HTTP header, including all field names and values. Postman may also include some hidden headers, like Host, Content-Size, User-Agent, etc.
Is there any max size for send post payload ?
Actually, I have a service which send a json payload to another micro-service, In side of JSON payload, there is a filed which hold encoded String (basically i have converted file into encoded String and set it into that payload field value).
File size is not fixed, So i am curious about it, is there any max size support for Post json payload ?
I know spring-boot have following properties for set max size for file upload.
spring.http.multipart.max-file-size
spring.http.multipart.max-request-size
But as i mentioned i am not upload file, i am convert file into encoded string and send this string with json payload.
So how can i set the max size support for post request.
Also In another scenario I am hitting a Get request to micro-service and then this micro service return a large payload,is there any limitation of receive payload size?
It depends, not only in spring configuration as you've pointed, but also in server configuration.
Most servers have a post request loadout limit. For example in tomcat you can configure it with the "maxPostSize" property. Quoting from the documentation:
The maximum size in bytes of the POST which will be handled by the container FORM URL parameter parsing.
The limit can be disabled by setting this attribute to a value less than or equal to 0. If not specified, this attribute is set to 2097152 (2 megabytes).
I am sending 2 large query string in AJAX requests, which are basically, a Base64 encoding of a jpeg(s). When the Camera is not a high-resolution one, AJAX request doesn't abort.
At first, I thought its a Nginx issue, Because I was getting an error as request entity too large I resolved it, Then I made changes to my Plug as
plug Plug.Parsers,
parsers: [
:urlencoded,
{:multipart, length: 20_000_000},
:json
],
pass: ["*/*"],
query_string_length: 1_000_000,
json_decoder: Poison
After defining query_string_length, Now I am not getting any errors like above but ajax request still abort.
Base64 encoding string size is 546,591 bytes or max.
I have tried to increase the AJAX request timeout to a very large timespan as well but it still fails. And I don't have any clue where the problem is right now.
How can we receive long strings in Plug?
Some of few answers on StackOverflow about this issue where people used AJAX and PHP, suggesting to change post_max_size, How can we do that in Elixir Plug?
As you are sending AJAX request with JSON data, you should put the length config of json in the plug.
plug Plug.Parsers,
parsers: [
:urlencoded,
{:multipart, length: 20_000_000},
{:json, length: 80_000_000},
],
pass: ["*/*"],
json_decoder: Poison
I suppose you will not put the data in the query string of the post, so the query_string_length - the maximum allowed size for query strings is not needed.
---Original answer---
For plug version around 1.4.3 and have no query_string_length option.
When you post the data as string, you are using Plug.Parsers.
If you are willing to process larger requests, please give a :length
to Plug.Parsers.
You should change the code query_string_length: 1_000_000 to length: 20_000_000.
I have built an audio stream for mp3 files, and each time client hits the audio it receives something like this:
But what it does is just plays 1 minute sample instead of 120 minute
What am I doing wrong here?
Not 100% sure because you didn't provide code or an example stream to test, but your handling of HTTP range requests is broken.
In your example request, the client sends Range: bytes=0-, and your server responds with a 1MiB response:
Content-Length: 1048576 (aka. 1 MiB)
Content-Range: 0-1048575/...
This is wrong, the client did not request this! It did request bytes=0-, meaning all data from position 0 to the end of the entire stream (See the http 1.1 RFC), i.e. a response equal to one without any Range. (IIRC, Firefox still sends the Range: bytes=0- to detect if the Server handles ranges in the first place).
This, combined with the Content-Length, leads the client (Firefox) to think the whole resource is just 1MiB in size, instead of the real size. I'd imagine the first 1 MiB of your test stream comes out as 1:06 of audio.
PS: The Content-Duration header (aka. RFC 3803) is something browsers don't usually implement at all and just ignore.
Just an idea. Did you tried some of the http 3xx header like:
'308 Resume Incomplete' or '503 Service Temporarily Unavailable' plus 'retry-after:2' or '413 Request Entity Too Large' plus 'retry-after:2'