I am trying to send a very long POST request as an AJAX call in my application. Up until the query length exceeds ~7585 characters (incl. page name) the request goes through fine. However, there is a definite limit going on as adding a single character will make the request fail.
I have ensured that the tomcat server.xml config does not specify maxPostSize, and the documentation says it defaults to 2MB, which is way more room than I am using.
Am I missing something here? I am suspicious that this is a limitation being imposed by the web browser (Firefox 3.6.18).
If this is just an unfortunate truth, how do people get around this? By batching the data into smaller chunks manually? It seems like this would be a common problem for people. Thanks!
Related
My url is as follows
/fcgi-bin/clireports.fcgi?sfPageId=param1&sfBoxId=param2&sfPagecId=param3&sfUsername=param4&sfSession=param5&sfSubmit=param6&showSampleReport=param7&saveAndAdd=param8
My questions is this:
Is there any performance issue when sending so many parameters in the query string ? In my case I am sending 8 parameters in the query string .
Will my website become slow because of this ?
Please enlighten me on this .
The performance is not about number of parameters (or just a little) but about number of bytes send.
The more data you send, the more it's costly to transfer/parse.
Remember that every browser has a hard limit on the maximum number of characters a GET request can handle. I recomment you not to use more than 255 characters to be on the safe side, but you can go higher (IE's limit is around 2000).
If you need more data, use POST.
Finally, your website will not become slow because of this. Because many other factors take much time than parsing GET request (take something like DB connection, or just php warmup)
Shouldn't be any issues, might be worth doing a POST request to clean up your request a bit, but other than that you shouldn't notice any issues as far as performance...just don't exceed the "limits" that vary depending on browser
Sometimes, I request a page and it takes too long to receive a response for the request and then load the page (sometimes the request times out and I never get a response).
However, if I open a new tab, copy the exact URL, and then append it with an arbitrary GET variable (with an arbitrary value), the request gets a response very fast (as the normal state is) and the page then loads, although the request wasn't getting a response without that arbitrary GET variable.
For a fake example, if I request:
http://example.com/
It might take a long time just loading, not receiving any response yet, but if I just open a new tab (at the same time), and request:
http://example.com/?foo=bar
It loads like magic!
Why is that happening to me? what could be the reason along the road between my browser and the page's server? does that have any relevance to ISP servers caching?
Any explanation is much, much appreciated, as I really am eager to know the reason!
P.S: I'm in Syria (where anything crazy is possible in Internet network), and this doesn't happen only to me, but to all people I know.
EDIT:
Note that it happens even if a URL has a GET variable already, for a real example I have a blog, and sometimes requesting this page (I changed the domain):
http://myblogdomain.com/wp-admin/admin.php?page=jetpack
Takes too long time (and sometimes it times out), but if I open a new tab and request:
http://myblogdomain.com/wp-admin/admin.php?page=jetpack&foo=bar
It loads fast (as normal).
It's likely that there is a caching proxy and/or firewall between you and the rest of the internet. There is probably a rule in the proxy that says URLs with GET parameters can pass through since they are likely to return unique content, but URL's without parameters must be fetched through a cache. The cache is likely overloaded or broken.
You probably have a proxy that needs to do some lengthy process (content check, DNS lookup, etc) once per domain.
When you open the second tab, that length process would have already started (for the first tab), so it wouldn't take as long.
If this is the case, opening the first tab with a querystring and the second tab without would still result in the second tab loading faster.
1-form a web developer perspective :
to get more details about what is taking this time , i could the network tab of my best friend (firebug )
as you may see above , i can see how much time spent on each step on the page .
2-even though i think this question should be moved to https://serverfault.com/ to get answers form networks geeks
How exactly does adding a random number to the end of an AJAX server call prevent the database server or browser (not entirely sure which one is intended) from caching? why does this work?
It is intended to prevent client-side (or reverse proxy) caching.
Since the cache will be keyed on the exact request, by adding a random element to the request, the exact request URL should never be seen twice; so it won't be used more than once, and an intelligent cache won't bother keeping around something that's never been seen more than once, at least, not for long.
It's to prevent your browser (and to a reasonable amount, a web proxy) from caching requests. Typically, a query parameter - like ?rand2024= tells the browser/proxy to send the onward request with a parameter telling your application to behave differently. That's why such requests are useful to bust caches.
Your browser caches the web page keyed by the exact text of the URL, so adding a random-number parameter ensures that the URL is different every time - thus no real caching. Your browser doesn't know that the server is (hopefully) ignoring this parameter.
I'm building a Twitter app that requires to check user data somewhat frequently, but I'm facing trouble with a cache that's oddly on Twitter's side, not mine.
Try the following user:
users/show in XML: http://twitter.com/users/show.xml?screen_name=technolocus
users/show in JSON: http://twitter.com/users/show.json?screen_name=technolocus
normal page: http://twitter.com/technolocus
All these methods of accessing data should return the same values, right? Check the statuses_count for each of them.
XML: 12548
JSON: 12513
normal: 12498
The normal method (i.e. just visiting the profile non-programatically) serves up the most correct value of 12498. If I post or delete tweets to this account, it gets updated on the profile page instantly, but the XML and JSON methods still return cached data.
At this point, the values of the XML and JSON methods are 12 to 18 hours old respectively.
I first tried to access these methods from my website (hosted on Dreamhost). I thought it was Dreamhost caching the responses. Then I tried to access the API directly from my browser. I did a cURL from the command line from my machine after that. It wasn't dreamhost. I thought it was probably my ISP (I think they use NetApp or something like that). Then I asked a friend in another corner of India to try it. He's getting the exact same cached responses as I am.
So it isn't Dreamhost's cache; it isn't my ISP or my country's cache. There's only one conclusion - Twitter is caching responses.
How in the heavens do I get around this?!?
Forgot to mention this: The script on the server is in PHP and is using cURL to retrieve the XML and JSON data from Twitter, while the local tests have been just using the browser. Both have the exact same result!
First, I think you should report this a a bug to Twitter. I see the same discrepancy as you, and no matter what that seems like a bug. Even if they're caching, I'd expect that a cache on their side would store an abstract form that would then be rendered into HTML, JSON, and XML. I wonder if what's actually going on is that these requests are performing similar but different queries.
Are you sure that the values are "old"? For example, did you actually delete about 50 updates recently (since you say the HTML one is newest but shows a lower count than the other two)? If you create another update do you see the HTML number increment while the other numbers stay the same, or do they all increment simultaneously?
If what you are saying is accurate, and it probably is, generally, you can't get around it. Twitter would want to be caching its responses since they are costly to reproduce every single time.
When you use Twitter's APIs, you end up being bound by its conventions, even if that includes caching.
Your best bet is to tweet to #twitterapi and get them to give you a response as to why the two representations are divergent.
Add ?blah=xxxx to all urls.
I don't develop anything against twitter and ocassionaly manually "follow" three tweets by going to them in my browser. They always lag behind by half a day. I add ?asdsadsadsad to the url (everytime something different) and it always updates. I don't know what Twitter is doing here and came here while searching for the problem. But I guess this trick of appending a random value to the url via GET will probably work for your api requests, too.
Several of my ajax applications in the past have used GET request but now I'm starting to use POST request instead. POST requests seem to be slightly more secure and definitely more url friendly/pretty. Thus, i'm wondering if there is any reason why I should use GET request at all.
I generally set up the question as thus: Does anything important change after the request? (Logging and the like notwithstanding). If it does, it should be a POST request, if it doesn't, it should be a GET request.
I'm glad that you call POST requests "slightly" more secure, because that's pretty much what they are; it's trivial to fake a POST request by a user to a page. Making it a POST request, however, prevents web accelerators or reloads from re-triggering the action accidentally.
As AJAX, there is one more consideration: if you are returning JSON with callback support, be very careful not to put any sensitive data that you don't want other websites to be able to see in there. Wikipedia had a vulnerability along these lines where the user anti-CSRF token was revealed via their JSON API.
All good points, however, in answer to the question, GET requests are more useful in certain scenarios over POST requests:
They can be bookmarked
They can be cached
They're faster
They have known consequences (assuming they don't change data), so visiting them multiple
times is not a problem.
For the sake of posterity, updating this comment with the blog notes re: point #3 here, all credit to Omar AL Zabir (the author of the referenced blog post):
"Atlas by default makes HTTP POST for all AJAX calls. Http POST is
more expensive than Http GET. It transmits more bytes over the wire,
thus taking precious network time and it also makes ASP.NET do extra
processing on the server end. So, you should use Http Get as much as
possible. However, Http Get does not allow you to pass objects as
parameters. You can pass numeric, string and date only. When you make
a Http Get call, Atlas builds an encoded url and makes a hit to that
url. So, you must not pass too much content which makes the url become
larger than 2048 chars. As far as I know, that’s what is the max
length of any url.
Another evil thing about http post is, it’s actually 2 calls. First
browser sends the http post headers and server replies with “HTTP 100
Continue”. When browser receives this, it sends the actual body."
You should use GET where you're doing a request which has no side effects, e.g. just fetching some info. This request can:
Be repeated without any problem - if the browser detects an error it can silently retry
Have its result cached by the browser
Be cached by a proxy
These things are all good. Anything which is only retrieving data (particularly public data) should really be a GET. The server should send sensible Last-Modified: and Expires: headers to allow caching if required.
There is one other difference not mentioned by anyone.
GET requests are passed in the URL string and are therefore subject to a length limit usually dependent on the browser. It seems that most are around 2000 chars.
POST requests can be much much larger - in fact not limited really. So if you're needing to request data from a web server and you're passing in lots of parameter information then a POST request might be the only option.
So, as mentioned before really a GET request is for requesting data (no side effects) while a POST request is generally used for transmitting data back to the server to be stored (with side effects). e.g. Use POST to upload a file. GET to retrieve a file.
There was a time when IE I believe had a very short GET URL string. Some applications like Lotus notes use large numbers of random characters to represent document id's. I had the displeasure of using another product that generated random strings so the page URL was unique each time. The random string was HUGE... and it didn't always work with IE6 from memory.
This might help you to decide where to use GET and where to use POST:
URIs, Addressability, and the use of HTTP GET and POST.
POST requests are just as insecure as GETs. The main difference is that POST is used to modify the state of the server application, while GET only requests data from it.
The difference matters when you use clean, "restful" URLs, where the URL itself specifies the resource, and the different methods trigger different actions on the server side.
Perhaps most importantly, GET is book-markable / viewable in url history, and searchable with Google.
POST is important where you don't want the event to be bookmarkable or able to be typed in as a URL - otherwise you (or Google crawling your URLS) could end up accidentally doing things like deleting users from your system, for example.
GET
POST
In GET method, values are visible in the URL
In POST method, values are not visible in the URL.
GET has a limitation on the length of the values, generally 255 characters.
POST has no limitation on the length of the values since they are submitted via the body of HTTP.
GET performs are better compared to POST because of the simple nature of appending the values in the URL.
It has lower performance as compared to GET method because of time spent in including POST values in the HTTP body
This method supports only string data types.
This method supports different data types, such as string, numeric, binary, etc.
GET results can be bookmarked.
POST results cannot be bookmarked.
GET request is often cacheable.
The POST request is hardly cacheable.
GET Parameters remain in web browser history.
Parameters are not saved in web browser history.
Source and more in depth analysis: https://www.guru99.com/difference-get-post-http.html