Given that the Bing API docs consist primarily of an error-ridden, two-page Word document, I have not been able to find the answer to this online. Trial and error has produced inconsistent results, and given that queries cost money, I'd rather not have to guess and check any more.
What are the length limits are on the new 'Azure marketplace' Bing WebSearch API? I know there is a limit of some sort, because long queries tend to return, "The request was not accepted by the data provider's service. The team is investigating the issue. We are sorry for the inconvenience." However, the query length at which this occurs, both in terms of characters or words, tends to vary based on the query entered. I am using fairly complex queries with advanced operators like site: and instreamset:, so it is very difficult to narrow down what the limitation is. ie: I have successfully made queries of over 1800 characters, and I have had them fail with less than 1200 characters.
Got the answer from Microsoft Support:
It looks like requests that arrive into Data Market have a query
string limit of 10240. However; the content provider (in this case
Bing Search ) has a limitation of 2048 characters.
Note that the 'query string' in this case is the entire URL, not just the 'Query' argument.
The reason why some URLs shorter than 2048 characters appeared to fail is due to the encoding of spaces. Before passing the query on to Bing, the Azure Data Market re-encodes it, and it uses %20 instead of + to encode spaces. So to check whether your request URL is within the 2048 characters, be sure to hex-encode all special characters, including spaces.
Updating this answer for the new Cognitive Services (v5.0) version of the Bing Search API:
from https://msdn.microsoft.com/en-us/library/dn760794.aspx#URL%20to%20Send%20Requests%20To:
Note: The maximum URL length is 2,048 characters. To ensure that your URL length does not exceed the limit, the maximum length of your query parameters should be less than 1,500 characters. If the URL exceeds 2,048 characters, the server will return 404 Not found.
Related
What is the maximum number of characters allowed in an Event Description (including spaces, punctuation, and URL
There is currrntly no documented limit for the length of the description in an event in google calendar.
I suspect that this would be due to the fact that google could change it.
I suggest you just try and save as much as you can in there and then handle any errors as they may come. TBH if memory services they just chop off what doesn't fit and dont respond with an error.
My applications shows nearby places by using the Google Places Web API.
The user has control over the place types searched in the requests.
Multiple types are concatenated with the pipe symbol |. I use &rankby=distance, because prominence does not matter for the app.
I have noticed, that requesting nearby places with "a lot" of types returns less results than filtering by a single type.
Example
returns 10 results near where I live:
&types=airport|bank|bar|bicycle_store|book_store|bus_station|casino|cafe|city_hall|clothing_store|food|furniture_store|grocery_or_supermarket|gym|hardware_store|library|liquor_store|movie_theater|museum|night_club|park|place_of_worship|police|post_office|restaurant|school|shoe_store|shopping_mall|spa|stadium|store|subway_station|train_station|university|zoo
returns 20 results and a next_page_token (so at least 20 results):
&types=store
I happen to live across a shopping mall, so I know for sure that there are more than 20 stores nearby. The first query contains store as a filter, too.
Questions
I would like to always show as many results as possible. Has anybody experienced the same issue? Is there any document that I did not see, anything on this topic?
I'm a bit lost, since I don't know where to start looking or how to approach this problem.
Be aware that searching for more than one type at a time is deprecated. See http://googlegeodevelopers.blogspot.com.au/2016/02/changes-and-quality-improvements-in_16.html:
Beginning Feb 16, 2016, we are replacing the types restriction
parameter with a new type search parameter. If you have been using the
types parameter for Nearby Search, Text Search or Radar Search you
will be affected.
Type search works similarly to types restriction, but it only supports
one type per request.
Requests using the types parameter and those specifying multiple types
(for example, types=hospital|pharmacy|doctor) will continue to return
results until Feb 16, 2017, but we do not recommend using multiple
types in a search request. After that date, requests with multiple
types will no longer be supported. To ensure the best possible search
results for your users, we recommend using a single type in search
requests.
Thanks to #AndrewR 's comment, I stumbled upon a comment on an issue, that states the following:
[Google Places API does not] support specifying more than 20 types at a time
– Comment on bug report from Sep 1, 2014
which solves my problem in a few words. I wish the docs had stated this.
I am using jquery, JSON, and AJAX for a comment system. I am curious, is there a size limit on what you can send through/store with JSON? Like if a user types a large amount and I send it through JSON is there some sort of maximum limit?
Also can any kind of text be sent through JSON. for example sometime I allow users to use html, will this be ok?
JSON is similar to other data formats like XML - if you need to transmit more data, you just send more data. There's no inherent size limitation to the JSON request. Any limitation would be set by the server parsing the request. (For instance, ASP.NET has the "MaxJsonLength" property of the serializer.)
There is no fixed limit on how large a JSON data block is or any of the fields.
There are limits to how much JSON the JavaScript implementation of various browsers can handle (e.g. around 40MB in my experience). See this question for example.
It depends on the implementation of your JSON writer/parser. Microsoft's DataContractJsonSerializer seems to have a hard limit around 8kb (8192 I think), and it will error out for larger strings.
Edit:
We were able to resolve the 8K limit for JSON strings by setting the MaxJsonLength property in the web config as described in this answer: https://stackoverflow.com/a/1151993/61569
Surely everyone's missed a trick here. The current file size limit of a json file is 18,446,744,073,709,551,616 characters or if you prefer bytes, or even 2^64 bytes if you're looking at 64 bit infrastructures at least.
For all intents, and purposes we can assume it's unlimited as you'll probably have a hard time hitting this issue...
Implementations are free to set limits on JSON documents, including the size, so choose your parser wisely. See RFC 7159, Section 9. Parsers:
"An implementation may set limits on the size of texts that it accepts. An implementation may set limits on the maximum depth of nesting. An implementation may set limits on the range and precision of numbers. An implementation may set limits on the length and character contents of strings."
There is really no limit on the size of JSON data to be send or receive. We can send Json data in file too. According to the capabilities of browser that you are working with, Json data can be handled.
If you are working with ASP.NET MVC, you can solve the problem by adding the MaxJsonLength to your result:
var jsonResult = Json(new
{
draw = param.Draw,
recordsTotal = count,
recordsFiltered = count,
data = result
}, JsonRequestBehavior.AllowGet);
jsonResult.MaxJsonLength = int.MaxValue;
What is the requirement? Are you trying to send a large SQL Table as JSON object? I think it is not practical.
You will consume a large chunk of server resource if you push thru with this.
You will also not be able to measure the progress with a progress bar because your App will just wait for the server to reply back which would take ages.
What I recommend to do is to chop the request into batches say first 1000 then request again the next 1000 till you get what you need.
This way you could also put a nice progress bar to know the progress as extracting that amount of data could really take too long.
The maximum length of JSON strings. The default is 2097152 characters, which is equivalent to 4 MB of Unicode string data.
Refer below URL
https://learn.microsoft.com/en-us/dotnet/api/system.web.script.serialization.javascriptserializer.maxjsonlength?view=netframework-4.7.2
I am using ColdFusion 8 and jQuery 1.7.2.
I am using CFAJAXPROXY to pass data to a CFC. Doing so creates a JSON array (argument collection) and passes it through the URL. The string can be very long, since quite a bit of data is being passed.
The site that I am working has existing code that limits the length of any URL query string to 250 characters. This is done in the application.cfm file by testing the length of the query string. If any query string is great than 250 characters, the request is aborted. The purpose of this was to ensure that hackers or other malicious code wouldn't be passed through the URL string.
Now that we are using the query string to pass JSON arrays in the URL, we discovered that the Ajax request was being aborted quite frequently.
We have many other security practices in place, such as stripping any "<>" tags from code and using CFQUERYPARAM.
My question is whether limiting the length of a URL string for the sake of security a good idea or is simply ineffective?
There is absolutely no correlation between URI length and security rather more a question of:
Limiting the amount of information that you provide to a user agent to a 'Need to know basis'. This covers things such as the type of application server you run and associated conventions, the web server you run and associated conventions and the operating system on the host machine. These are essentially things that can be considered vulnerabilities.
Reducing the impact of exploiting those vulnerabilities i.e introducing patches, ensuring correct configuration etc.
As alluded to above, at the web tier, this doesn't only cover GET's (your concern), but also POST's, PUT's, DELETE's on just about any other operation on a HTTP resource.
Moved this into an answer for Evik -
That seems (at best) completely unnecessary if the inputs are being properly sanitized. I'm sure someone clever can quickly defeat a "security by small doorway" defense, assuming that's the only defense.
OWASP has some good, sane guidelines for web security. As far as I've read, limiting the size of the url is not on the list. For more information, see: https://www.owasp.org/index.php/Top_10_2010-Main
I would also like to echo Hereblur's comment that this makes internationalization tricky, or maybe impossible.
I'm not a ColdFusion developer. But I think it's the same with other language.
I think It's help just a little bit. The problem of malicious code or sql injection should be handle by your application.
I agree that limited length of query string value is safer and add more difficult to hackers. But you cant do this with POST data. and It's limit some functionality. For example,
For one utf-8 character, It may take 9 characters after encoded. that's mean you can put only 27 non-english characters.
The only reason to limit has to do with performance and DOS attack - not security per se (though DOS is a security threat by bringing down your server). Web servers and App servers (including CF) allow you to limit the size of POST data so that your server won't be degraded by very large file uploads. URL data if substantial can result in long running requests as the server struggles to parse or handle or write or whatever.
So there is some modest risk here related to such things. Back in the NT days IIS 3 had a number of flaws that were "locked down" by limiting the length of the URL - but those days are long gone. There are far more exploits representing low hanging fruit that I would look at first before examining this issue too closely - unless of course you feel like you are hanging a specific problem with folks probing you (with long URLs I mean :).
I am building a mobile web application that targets webkit. I have a requirement to perform a live search (on keypress) against a database of ~5000 users.
I've tried a number of different techniques:
On page load, making an AJAX call which loads an in-memory representation of all 5000 users, and querying them on the client. I tried sending JSON, which proved to be too large, and also a custom delimited string, which was then parsed using split(). This was better, but ultimately searches against this array of users was slow.
I tried using a conventional AJAX call, which would return users based on a query, also using the custom delimited string technique. This was better, but I was forced to tune it so that searches were only performed with a minimum of 3 characters. This is not optimal, as I would like to be able to start filtering after 1 character. I could also throttle the calls so that not every keystroke within a certain threshold triggered a request. This could help with performance, but I'd rather not have to fiddle with that sort of thing.
Facebook mobile does this very well if you try their friend search. Searches happen instantaneously, and are triggered after 1 character.
My question is, does anyone have any suggestions for faster live searches for a mobile app? Should I be looking at localStorage? Is this reliable, feasible?
Is there any reason you can't use a binary search? The names you're looking for should be in a block. If you want first and last name search, you could create a second copy of the data sorted by last name and look in both sets.
Some helpful but more complicated data structures that address this type of problem include:
http://en.wikipedia.org/wiki/Directed_acyclic_word_graph
http://en.wikipedia.org/wiki/Trie