I'm trying to match a latitude and longitude pair to a road segment that has a start and end latitude and longitude. All of the formulas I've been able to find query by the closest match to a single latitude and longitude, but not to a pair. I suppose one option is to get the average, or center of the segment, but this is not ideal. I'm querying this in SQLLite since my data is in GeoPackage format, but if anyone even has a formula to use I can translate that to SQLLite.
Thanks!
Hmm interesting. Conceptually, and without considering performance, would the following work? (consider this untested pseudo):
SELECT
MIN(ABS(lon_field - :lon1)) AS start_lon,
MIN(ABS(lat_field - :lat1)) AS start_lat,
MIN(ABS(lon_field - :lon2)) as end_lon,
MIN(ABS(lat_field - :lat2)) AS end_lat;
Related
I have a data set of about 20 million coordinates. I want to be able to pass in a latitude, longitude, and distance in miles and return all coordinates that are within the mile range of my given coordinates. I need the response time to ideally be sub 50ms.
I have tried loading all coordinates in memory in a golang service which, on every request, will loop through the data and using haversine filter all coordinates which are within the given miles distance of my given coordinate.
This method sees the results return in around 2 seconds. What approach would be good to increase the speed of the results? I am open to any suggestions.
I am toying around with the idea of grouping all coordinates by degree and only filtering by the nearest to the given coordinates. Haven't had any luck improving the response times yet though. My data set is only a test one too as the real data could potentially be in the hundreds of millions.
I think that this is more of a data structure problem. One good way to store large sets of geospatial coordinates is with an R-tree. It provides logn M search. I have limited knowledge of Go, but I have used an R-Tree to great effect for similarly sized datasets in a similar use case in a JS application. From a quick search it appears as though there are at least a couple Go R-Tree implementations out there.
Idea would be to have a "grid" that partitions coordinates, so that when you do need to do a lookup you can safely return all coordinates in particular cell, do not return any from the cells too far away from target, and only do per coordinate comparison for coordinates that are in the cells that contains some coordinates within distance and some outside the distance.
Simplified to 1D:
Coordinates are from 1 to 100
you partition into 5 blocks of 20
When somebody looks for all coordinates within distance 25 from 47
you return all coordinates in blocks [30,39], [40,49],[50,59],[60,69] and then after doing per coordinate analysis for blocks [20,29] and [70,79] you additionally return 22,23,24,25,26,27,28,29, 70,71,72.
Unfortunately I have no realistic way to estimate speedup of this approach so you would need to implement it and benchmark it by yourself.
MongoDB has various geographic searches $geoNear will allow you to search for points within a specific distance from a point or within a shape.
https://docs.mongodb.com/manual/reference/operator/aggregation/geoNear/
PostGIS for Postgres has something similar, but I am not too familiar with it.
I'm working on a project with 500,000 participants. We have in our database the precise coordinates of their home, and we want to release this data to someone who needs it to evaluate how close our participants live to one another.
We are very reluctant to release the precise coordinates, because this is an anonymized project and the risk for re-identification would be very high. Rounded coordinates (to something like 100m or 1km) are apparently not precise enough for what they're trying to achieve.
A nice workaround would have been to send them a 500,000 by 500,000 matrix with the absolute distance between each pair of participants, but this means 250 billion entries, or rather 125 billion if we remove half the matrix since |A-B| = |B-A|.
I've never worked with this type of data before, so I was wondering if anyone had a clever idea on how to deal with this? (Something that would not involve sending them 2 TB of data!)
Thanks.
Provided that the recipient of the data is happy to perform the great circle calculation to calculate the distance themselves, then you only need to send the 500,000 lines, but with transposed latitudes and longitudes.
First of all identify an approximate geospatial centre of your dataset, and then work out the offsets needed to transpose this centre to 0°N and 0°E. Then apply these same offsets to the users' latitudes and longitudes. This will centre the results around the equator and the prime meridian.
Provided your real data isn't too close to the poles, the distance calculated between real points A and B will be very close to the corresponding offset points.
Obviously the offsets applied need to be kept secret.
This approach may not work if it is known that your data is based around a particular place - the recipient may be able to deduce where the real points are - but that is something you'll need to decide yourself.
How to search geo-point with polygon across dateline(International Date Line,or 180 and -180 longitude) use java api?
Hello everyone:
I use ElasticSearch 2.1 and its java api, I want to search documents with polygon geo filter(the polygon is rectangle usually),but when the polygon across the dateline (International Date Line,or 180 and -180 longitude),it go wrong. For example:
My code:
BoolQueryBuilder mustQuery = QueryBuilders.boolQuery().must(QueryBuilders.matchAllQuery());
......
GeoPolygonQueryBuilder qb = QueryBuilders.geoPolygonQuery("description.device_location.es_geo");
qb.addPoint(0,100);//the left down vertex of the polygon(rectangle),patams is (lat , lon)
qb.addPoint(0,-170);//right down
qb.addPoint(80,-170);//right up
qb.addPoint(80,100);//left up
qb.addPoint(0,100);//left down,same to the first vertex
mustQuery = mustQuery.must(qb);
SearchResponse searchResponse = EsClient.getClient().prepareSearch(Config.indexName)
.setTypes(Config.typeName)
.setSearchType(SearchType.DFS_QUERY_THEN_FETCH)
.setQuery(mustQuery)
.setFrom(0).setSize(size).setExplain(true)
.execute()
.actionGet();
diagrammatic sketch:
I want search in A area，but ES search in B area in fact
As the picture above, I provide ES points [0,100],[0,-170],[80,-170],[80,100],[0,100] to describe A area,and I want docs in A area, A area across dteline.
But according to the result, there is longitude 82,98,-121 etc., but no docs between [100,180]and between [-180,-170],so I suppose ES search in B area in fact(It analyze the polygon in error).
I search for the solution , on the ES's website, I found some words about this problem:
(form www.elastic.co/guide/en/elasticsearch/reference/2.1/geo-shape.html)
IMPORTANT NOTE: GeoJSON does not mandate a specific order for vertices thus ambiguous polygons around the dateline and poles are possible. To alleviate ambiguity the Open Geospatial Consortium (OGC) Simple Feature Access specification defines the following vertex ordering:
Outer Ring - Counterclockwise
Inner Ring(s) / Holes - Clockwise
For polygons that do not cross the dateline, vertex order will not matter in Elasticsearch. For polygons that do cross the dateline, Elasticsearch requires vertex ordering to comply with the OGC specification. Otherwise, an unintended polygon may be created and unexpected query/filter results will be returned.
The following provides an example of an ambiguous polygon. Elasticsearch will apply OGC standards to eliminate ambiguity resulting in a polygon that crosses the dateline.
But this is for geo-Shape datatype, and my docs' location is geo-point, I can't found similar words on geo-points's webpage(www.elastic.co/guide/en/elasticsearch/reference/2.1/geo-point.html).
I hava try some way:
1. Counterclockwise and clockwise vertex order;
2. start from every vertex of rectangle;
3. replace -170 lon with 190 lon;
but these ways all don't work.
Does anyone know how to search with a polygon across dateline ?
Thanks!
(sorry, I'm a Chinese developer and I can't speak English well, if there is solecism,please give me advice or comments, thank you.)
Here is the translated text of Chinese.Chinese characters couldn't be insert to question direct.
I'm the asker, and I have an answer already.
I have ask this question at https://discuss.elastic.co/t/search-geo-point-with-polygon-across-dateline/39103/3 ,and get answer there.
Before,I have ever think if there isn't perfect solution,I can split the polygon into 2 by dateline.I have know ES don't suport to search geo-point with polygon across dateline now:
At the moment geo_polygon queries for geo_point types do not work
across the dateline. You will either have to:
manually split the polygon into 2 and combine them using a boolean AND;
reindex the geo_points as geo_shapes (recommend setting points_only = true) and using a geo_shape (www.elastic.co/guide/en/elasticsearch/reference/2.1/geo-shape.html) query.
But if you know the polygon is a
rectangle, you can use a geo_bounding_box (www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-geo-bounding-box-query.html) query which will properly
handle the dateline (as in your example).
Thanks for #nknize and #medcl1 (at discuss.elastic.co)!
Given a square area, what is the best way to find the approximate coordinates of every street intersection within the given area ?
Since there is no description of your application, I can't tell if you need to use Google Maps or if another data source would answer your needs.
If http://openstreetmap.org fulfills the requirements of your application, then it's easy:
the OSM API has a request to pull data from a rectangular region. You get XML data.
filter this data to keep only the street you are interested in, probably the "key=highway" tags
filter this to keep only the points belonging to two or more lines.
Please disregard this if Google Maps is a requirement.
But still: since the roads exist independently of the database, the above method will yield roads intersections (in lat/long coordinates) with a pretty high correlation with what you would get from Google maps ;-) You can then use those points to display them over a Google map, knowing that both datasets aren't identical so it won't be perfect.
Might not be the easiest method but I used a seperate database of our countries roads with their linestrings.
I took the first and last points of each line string, then counted the number of roads within 50 m of each start/end point. I then took the nodes from navigation route and used these to compare the number of roads intersecting with each node. I then looked at the direction each start point and the next point along that road, which gives you direction. from that with a bit of maths you can work out the number and angle of the roads at the next intersection. I then made a road rules application that tells you which vehicles to give way to
In WP7, is it possible to show compass direction based on the given latitude and longitude values. (For example, if I am in India and if the latitude and longitude values of a place in some other country is given). If yes, please give some idea on how to achieve this.
This is what is known as the Second (inverse) geodetic problem.
"Given two points, determine the azimuth and length of the line
(straight line, arc or geodesic) that connects them." Wikipedia
You can get the distance using System.Device.Location.GeoCoordinate.GetDistanceTo(). Otherwise have a look at the excellent C# Geodesy Library for GPS – Vincenty’s Formulae. In particular the GeodeticCurve.