I needed to find out the highest point of a region.
However when I iterate through all the points isn't efficient enough for a my CPU.
So does Google Earth API provide any relevant approach ?
Thanks !
Did not see Google Earth API provides anything about this, but you may add elevation as a parameter to reduce results, and give it an increment until the result's count is acceptable.
Since this point doesn't change often, I would just use a constant.
'Mount Everest (WGS84)
highest_lat = 27.988056
highest_lon = 86.925278
Without seeing any code it is impossible to say where the overhead is. However, in general, you should pass your multiple calls to the api using the google.earth.executeBatch method.
https://developers.google.com/earth/documentation/reference/google_earth_namespace#ab26414915202d39cad12bcd5bd99e739
This method efficiently executes an arbitrary, user-defined function minimizing the amount of overhead incurred during cross-process communication between the web browser and Google Earth Plugin.
Here is a working example
http://earth-api-samples.googlecode.com/svn/trunk/examples/batch.html
Related
I'm pondering over how to efficiently represent locations in a database, such that given an arbitrary new location, I can efficiently query the database for candidate locations that are within an acceptable proximity threshold to the subject.
Similar things have been asked before, but I haven't found a discussion based on my criteria for the problem domain.
Things to bear in mind:
Starting from scratch, I can represent data in any way (eg. long&lat, etc)
Any result set is time-sensitive, in that it loses validity within a short window of time (~5-15mins) so I can't cache indefinitely
I can tolerate some reasonable margin of error in results, for example if a location is slightly outside of the threshold, or if a row in the result set has very recently expired
A language agnostic discussion is perfect, but in case it helps I'm using C# MVC 3 and SQL Server 2012
A couple of first thoughts:
Use an external API like Google, however this will generate thousands of requests and the latency will be poor
Use the Haversine function, however this looks expensive and so should be performed on a minimal number of candidates (possibly as a Stored Procedure even!)
Build a graph of postcodes/zipcodes, such that from any node I can find postcodes/zipcodes that border it, however this could involve a lot of data to store
Some optimization ideas to reduce possible candidates quickly:
Cache result sets for searches, and when we do subsequent searches, see if the subject is within an acceptable range to a candidate we already have a cached result set for. If so, use the cached result set (but remember, the results expire quickly)
I'm hoping the answer isn't just raw CPU power, and that there are some approaches I haven't thought of that could help me out?
Thank you
ps. Apologies if I've missed previously asked questions with helpful answers, please let me know below.
What about using GeoHash? (refer to http://en.wikipedia.org/wiki/Geohash)
I have the following setup:
boolean data: (userid, itemid)
hadoop based mahout itemSimilarityJob with following arguements:
--similarityClassname Similarity_Loglikelihood
--maxSimilaritiesPerItem 50 & others (input,output..)
item based boolean recommender:
-model MySqlBooleanPrefJDBCDataModel
-similarity MySQLJDBCInMemoryItemSimilarity
-candidatestrategy AllSimilarItemsCandidateItemsStrategy
-mostSimilarItemsCandidateStrategy AllSimilarItemsCandidateItemsStrategy
Is there a way to use similarity cooccurence in my setup to get final recommendations? If I plug SIMILARITY_COOCCURENCE in the job, the MySqlJDBCInMemorySimilarity precondition checks fail since the counts become greater than 1. I know I can get final recommendations by running the recommender job on the precomputed similarities. Is there way to do this real time using the api like in the case of similarity loglikelihood (and other similarity metrics with similarity values between -1 & 1) using MysqlInMemorySimilarity?
How can we cap the max no. of similar items per item in the item similarity job. What I mean here is that the allsimilaritemscandidatestrategy calls .allsimilaritems(item) to get all possible candidates. Is there a way I can get say top 10/20/50 similar items using the API. I know we can pass a --maxSimilaritiesPerItem to the item similarity job but i am not completely sure as to what is stands for and how it works. If I set this to 10/20/50, will I be able to achieve what stated above. Also is there way to accomplish this via the api?
I am using a rescorer for filtering out and rescoring final recommendations. With rescorer, the calls to /recommend/userid?howMany=10&rescore={..} & to /similar/itemid?howMany=10&rescore{..} are taking way to longer (300ms-400ms) compared to (30-70ms) without the rescorer. I m using redis as an in memory store to fetch rescore data. The rescorer also receives some run-time data as shown above. There are only a few checks that happen in rescorer. The problem is that as the no. of item preferences for a particular user increase (> 100), the no. of calls to isFiltered() & rescore() increase massively. This is mainly due to the fact that for every user preference, the call to candidateStrategy.getCandidatItems(item) returns around (100+) similar items for each and the rescorer is called for each of these items. Hence the need to cap the max number of similar items per item in the job. Is this correct or am I missing something here? Whats the best way to optimise the rescorer in this case?
The MysqlJdbcInMemorySimilarity uses GenericItemSimilarity to load item similarities in memeory and its .allsimilaritems(item) returns all possible similar items for a given item from the precomputed item similarities in mysql. Do i need to implement my own item similarity class to return top 10/20/50 similar items. What about the if user's no. of preferences continue to grow?
It would be really great if anyone can tell me how to achieve the above? Thanks heaps !
What Preconditions check are you referring to? I don't see them; I'm not sure if similarity is actually prohibited from being > 1. But you seem to be asking whether you can make a similarity function that just returns co-occurrence, as an ItemSimilarity that is not used with Hadoop. Yes you can; it does not exist in the project. I would not advise this; LogLikelihoodSimilarity is going to be much smarter.
You need a different CandidateItemStrategy, particularly, look at SamplingCandidateItemsStrategy and its javadoc. But this is not related to Hadoop, rather than run-time element, and you mention a flag to the Hadoop job. That is not the same thing.
If rescoring is slow, it means, well, the IDRescorer is slow. It is called so many times that you certainly need to cache any lookup data in memory. But, reducing the number of candidates per above will also reduce the number of times this is called.
No, don't implement your own similarity. Your issue is not the similarity measure but how many items are considered as candidates.
I am the author of much of the code you are talking about. I think you are wrestling with exactly the kinds of issues most people run into when trying to make item-based work at significant scale. You can, with enough sampling and tuning.
However I am putting new development into a different project and company called Myrrix, which is developing a sort of 'next-gen' recommender based on the same APIs, but which ought to scale without these complications as it's based on matrix factorization. If you have time and interest, I strongly encourage you to have a look at Myrrix. Same APIs, the real-time Serving Layer is free/open, and the Hadoop-based Computation Layer backed in also available for testing.
I am using the Google Places API to retrieve all the POI (Places of Interest) around a the current location, it works ok but I have noticed that whatever the value of the radius is, I always get the same number of results (~ 20). As a result, if I give a radius that is too big, I don't necessarily get the nearest POIs. If I reduce the amount of the radius to be small enough, I will retrieve those nearest places again (from experimentation, I have noticed that 100 meters is a proper value) but that means that I will not get any POIs beyond 100 meters which is not quite what I want.
My question is: is there any way by which I can get all the POIs (with no limitations) within a certain radius.
Thank you!
The Google Places API always returns 20 results by design, selecting the 20 results that best fit the criteria you define in your request. The Developer's Guide / Docs don't explicitly cite that number anywhere that I have seen. I learned about the limit watching the Autocomplete Demo & Places API Demo & Discussion Video, given by Paul Saxman, a Developer Advocate at Google and Marcelo Camelo, Google's Technical Lead for the Places API.
The entire video is worth watching, but more specific to your question, if you set the playback timer at about 11:50, Marcelo Camelo is contrasting the Autocomplete tool versus the general Places API, and that's the portion of the video where he mentions the 20 result limit. He mentions 20 as the standard result count several times.
There are many other good Places API and Google Maps videos linked to that area on YouTube as well.
As mentioned on the Google Places Issue Tracker here: http://code.google.com/p/gmaps-api-issues/issues/detail?id=3425
We are restricted by our data provider licenses to enable apps to display no more than 20 places results at a time. Consequently we are not able to increase this limit at this time.
It does sound like you are however trying to return results that are closest to a specified location, this is now possible by using the 'rankby=distance' parameter instead of 'radius' in your request.
e.g.
https://maps.googleapis.com/maps/api/place/search/json?location=-33.8670522,151.1957362&rankby=distance&types=food&name=harbour&sensor=false&key=YOUR_API_KEY
Try google.maps.places.RankBy.DISTANCE; as default is google.maps.places.RankBy.PROMINENCE;
An easy example of this is shown Here
(Chrome only)
I am using Bing Maps with Ajax and I have about 80,000 locations to drop pushpins into. The purpose of the feature is to allow a user to search for restaurants in Louisiana and click the pushpin to see the health inspection information.
Obviously it doesn't do much good to have 80,000 pins on the map at one time, but I am struggling to find the best solution to this problem. Another problem is that the distance between these locations is very small (All 80,000 are in Louisiana). I know I could use clustering to keep from cluttering the map, but it seems like that would still cause performance problems.
What I am currently trying to do is to simply not show any pins until a certain zoom level and then only show the pins within the current view. The way I am currently attempting to do that is by using the viewchangeend event to find the zoom level and the boundaries of the map and then querying the database (through a web service) for any points in that range.
It feels like I am going about this the wrong way. Is there a better way to manage this large amount of data? Would it be better to try to load all points initially and then have the data on hand without having to hit my web service every time the map moves. If so, how would I go about it?
I haven't been able to find answers to my questions, which usually means that I am asking the wrong questions. If anyone could help me figure out the right question it would be greatly appreciated.
Well, I've implemented a slightly different approach to this. It was just a fun exercise, but I'm displaying all my data (about 140.000 points) in Bing Maps using the HTML5 canvas.
I previously load all the data to the client. Then, I've optimized the drawing process so much that I've attached it to the "Viewchange" event (which fires all the time during the view change process).
I've blogged about this. You can check it here.
My example does not have interaction on it but could be easily done (should be a nice topic for a blog post). You would have thus to handle the events manually and search for the corresponding points yourself or, if the amount of points to draw and/or the zoom level was below some threshold, show regular pushpins.
Anyway, another option, if you're not restricted to Bing Maps, is to use the likes of Leaflet. It allows you to create a Canvas Layer which is a tile-based layer but rendered in client-side using HTML5 canvas. It opens a new range of possibilities. Check for example this map in GisCloud.
Yet another option, although more suitable to static data, is using a technique called UTFGrid. The lads that developed it can certainly explain it better than me, but it scales for as many points as you want with a fenomenal performance. It consists on having a tile layer with your info, and an accompanying json file with something like an "ascii-art" file describing the features on the tiles. Then, using a library called wax it provides complete mouse-over, mouse-click events on it, without any performance impact whatsoever.
I've also blogged about it.
I think clustering would be your best bet if you can get away with using it. You say that you tried using clustering but it still caused performance problems? I went to test it out with 80000 data points at the V7 Interactive SDK and it seems to perform fine. Test it out yourself by going to the link and change the line in the Load module - clustering tab:
TestDataGenerator.GenerateData(100,dataCallback);
to
TestDataGenerator.GenerateData(80000,dataCallback);
then hit the Run button. The performance seems acceptable to me with that many data points.
When timing my (web) application for performance/latency, should I use the minimum time measured in n runs, or the average? Or something else?
If it's the latter case, when to use what?
If your answer is going to be something along the lines of, "research it, dude", could you point me to a good resource?
You could retrieve the values, store them by time, with couple (time_of_the_call, response_time). Then you may treat these data with tools, draw graphs, make statistics.
I think an average, minimum, etc, is not sufficient, you need a set of measures.
For example you may put you data in a csv file and import in excel, or even use google graph api to draw real time graphs.
Use median rather than mean (or average).
For reasoning, see the page Mean Delay Considered Harmful by Stanislav Shalunov (the author of thrulay).
I would say you have to figure that one out your self. Are you benchmarking for
Request throughput
User experience
Checking timeout thresholds
Depending on what you want your analysis to answer different metrics should be used.
http://www.webperformancetoday.com/2012/02/13/non-geeky-guide-to-performance-measurement/
This is a good article on "response time" timing. (Websites.)