All 3 cases validate in jsonlint, but only the first one is displayed in mapshaper.
I do not understand why case 2 and 3 are not displayed .
So how can I set up this correct using '-' in the arcs ref for a property ?
Displays OK in https://mapshaper.org/
{"type":"Topology", "crs":{"type":"name","properties":{"name":"EPSG:25832"}},"objects":{"collection": { "type": "GeometryCollection", "geometries":[
{ "type": "MultiPolygon", "arcs": [[[0,1]]],"properties":{"id":9005309}}]}},"arcs":
[
[[565546,7786890.97],[565545.69,7786859.81]],
[[565545.69,7786859.81],[565545.06,7786876.95],[565546,7786890.97]]]
}
Does not displays in https://mapshaper.org/
{"type":"Topology", "crs":{"type":"name","properties":{"name":"EPSG:25832"}},"objects":{"collection": { "type": "GeometryCollection", "geometries":[
{ "type": "MultiPolygon", "arcs": [[[-0,-1]]],"properties":{"id":9005309}}]}},"arcs":
[
[[565545.69,7786859.81],[565546,7786890.97]],
[[565546,7786890.97],[565545.06,7786876.95],[565545.69,7786859.81]]
]
}
Does not displays in https://mapshaper.org/
{"type":"Topology", "crs":{"type":"name","properties":{"name":"EPSG:25832"}},"objects":{"collection": { "type": "GeometryCollection", "geometries":[
{ "type": "MultiPolygon", "arcs": [[[-0,1]]],"properties":{"id":9005309}}]}},"arcs":
[
[[565545.69,7786859.81],[565546,7786890.97]],
[[565545.69,7786859.81],[565545.06,7786876.95],[565546,7786890.97]]]
}
You have the wrong index for negative (/reversed) arcs:
Each arc must be referenced by numeric zero-based index into the
containing topology’s arcs array. For example, 0 refers to the first
arc, 1 refers to the second arc, and so on.
A negative arc index indicates that the arc at the ones’ complement of
the index must be reversed to reconstruct the geometry: -1 refers to
the reversed first arc, -2 refers to the reversed second arc, and so
on. In JavaScript, you can negate a negative arc index i using the
bitwise NOT operator, ~i. (docs)
Instead of -0 you should be using -1. So your 2nd topojson example should draw as follows:
{"type":"Topology",
"crs":{"type":"name","properties": {"name":"EPSG:25832"}},
"objects":{
"collection": { "type": "GeometryCollection",
"geometries":[
{ "type": "MultiPolygon",
"arcs": [[[-1,-2]]],
"properties":{"id":9005309}}]}},
"arcs":[
[[565545.69,7786859.81],[565546,7786890.97]],
[[565546,7786890.97],[565545.06,7786876.95],[565545.69,7786859.81]]
]
}
Related
How do I index a document with below data in elasticsearch(geo datatype)?
<west>5.8663152683722</west>
<north>55.0583836008072</north>
<east>15.0418156516163</east>
<south>47.2701236047002</south>
I tried geo_point and its working for lon and lat's, not sure how to save this data. any help is highly appreciated.
You'll have to use the geo_shape datatype and convert your XML (I assume) semi-points into a line string or polygon before sync.
I'm gonna go with a polygon here.
Let's visualise the conventional cardinal directions:
North (+90)
|
(-180) West ——+—— East (+180)
|
South (-90)
geo_shape expects GeoJSON-like inputs so you'll need five coordinate points, the first and last of which are identical (according to the GeoJSON spec).
Therefore, borrowing from TurfJS and going from bottom left counter-clockwise,
const lowLeft = [west, south];
const topLeft = [west, north];
const topRight = [east, north];
const lowRight = [east, south];
return
[
[
lowLeft,
lowRight,
topRight,
topLeft,
lowLeft
]
]
Finally, let's create our index and plug your numbers in
PUT /example
{
"mappings": {
"properties": {
"location": {
"type": "geo_shape"
}
}
}
}
POST /example/_doc
{
"location":{
"type":"polygon",
"coordinates":[
[
[
5.8663152683722,
47.2701236047002
],
[
15.0418156516163,
47.2701236047002
],
[
15.0418156516163,
55.0583836008072
],
[
5.8663152683722,
55.0583836008072
],
[
5.8663152683722,
47.2701236047002
]
]
]
}
}
Then verify that the center of your square if indeed inside of your indexed polygon:
GET example/_search
{
"query": {
"geo_shape": {
"location": {
"shape": {
"type": "point",
"coordinates": [
10.45406545999425,
51.1642536027537
]
},
"relation": "intersects"
}
}
}
}
I have a cosmosdb collection (sql api) that I've populated with documents representing CIDR Network Ranges.
The relevant part of each document is
{
"Network": "31.216.102.0/23",
"IPRangeStart": 534275584,
"IPRangeEnd": 534276095,
Each CIDR block has it's start and end IP addresses converted to uint and stored in hte RangeStart and RangeEnd properties.
When I run a query to search for a specific entry by it's start range, it works as expected and is quite fast.
SELECT top 1 * FROM c WHERE c.IPRangeStart = 532361216
Request Charge: 3.02 RUs
However when I introduce a between query using <= / => operators, it gets VERY expensive.
SELECT top 1 * FROM c WHERE c.IPRangeStart <= 534275590 AND c.IPRangeEnd >= 534275590
Request Change: 1647.99 RUs
I've reviewed the index setup on the collection
I've also applied 2 additional integer range indices on the collection for the two specific properties in question. Though there doesn't appear to be a way to check for progress of these indices being applied/created in the background.
Is there something obvious that I might be missing.
{
"indexingMode": "consistent",
"automatic": true,
"includedPaths": [
{
"path": "/*",
"indexes": [
{
"kind": "Range",
"dataType": "Number",
"precision": -1
},
{
"kind": "Hash",
"dataType": "String",
"precision": 3
}
]
},
{
"path": "/IPRangeStart/?",
"indexes": [
{
"kind": "Range",
"dataType": "Number",
"precision": -1
},
{
"kind": "Hash",
"dataType": "String",
"precision": 3
}
]
},
{
"path": "/IPRangEnd/?",
"indexes": [
{
"kind": "Range",
"dataType": "Number",
"precision": -1
},
{
"kind": "Hash",
"dataType": "String",
"precision": 3
}
]
}
],
"excludedPaths": []
}
Think I solved it. The problem stemmed from the fact that I had a greater than query on one property and a less than query on a different property.
It appears that cosmos was merging the full set of documents that satisfied each independent filter clause.
Since the largest CIDR range in the set was a /18 (16k address block) was able to get it working by saying.
Where start <= value
And start >= value-32786
And end >= value
And end <= value+32768
I have following mapping in elastic search. I am able to PUT documents using Sense plugin but unable to do so using XContentBuilder to set the geo_shape field value. I am getting following error:
error:
[106]: index [streets], type [street], id [{dc872755-f307-4c5e-93f6-bba9c95791c7}], message [MapperParsingException[failed to parse [shape]]; nested: ElasticsearchParseException[shape must be an object consisting of type and coordinates];]
mapping:
PUT /streets
{
"mappings": {
"street": {
"properties": {
"id": {
"type": "string"
},
"shape": {
"type": "geo_shape",
"tree": "quadtree"
}
}
}
}
}
code:
val bulkRequest:BulkRequestBuilder = esClient.prepareBulk()
//inloop
xb = jsonBuilder().startObject()
xb.field("id", guid)
xb.field("shape", jsonString) // removing this line creates the index OK but without the geo_shape
xb.endObject()
bulkRequest.add(esClient.prepareIndex("streets", "street", guid).setSource(xb))
//end loop
val bulkResponse:BulkResponse = bulkRequest.execute().actionGet()
if(bulkResponse.hasFailures){
println(bulkResponse.buildFailureMessage())
}
jsonString:
{
"id": "{98b8fd8d-074c-4349-a83b-6e892bf2d0ef}",
"shape": {
"type": "LineString",
"coordinates": [
[-70.81866815832467, 43.12187109162505],
[-70.83054813653018, 43.15917412985851],
[-70.81320737213957, 43.23522269547419],
[-70.90108590067649, 43.28102004268419]
],
"crs": {
"type": "name",
"properties": {
"name": "EPSG:4326"
}
}
}
}
Appreciate any feedback?
Thanks
It might be a bit late for you, but this could help someone facing a similar issue even nowadays.
Following your index mapping for the document streets, we have these properties: id and shape.
In your error message, it's described that:
shape must be an object consisting of type and coordinates
So for your concrete case, the crs array is just not accepted (don't know exactly why you can't add extra parameters).
This is an example for how to add a document into the streets index using CURL:
curl -X POST "localhost:9200/streets/_doc?pretty" -H 'Content-Type: application/json' -d '
{
"id": 123,
"shape": {
"type": "Polygon",
"coordinates": [
[
[
32.85444259643555,
39.928694653732364
],
[
32.847232818603516,
39.9257985682691
],
[
32.837791442871094,
39.91947941109337
],
[
32.837276458740234,
39.91579296675271
],
[
32.85392761230469,
39.913423004886894
],
[
32.86937713623047,
39.91329133793421
],
[
32.88036346435547,
39.91539797880347
],
[
32.85444259643555,
39.928694653732364
]
]
]
}
}'
If you need to add a LineString, instead of a Polygon, just change the 'type' attribute from the 'shape'.
I hope this helps people having to add documents with shapes into an ElasticSearch database.
I have a TopoJSON file containing the boundaries of various districts in Uttar Pradesh, India. When you load the data on a map, you see only the outlines of the districts; the districts themselves are not filled.
I believe the problem is that each district is of type GeometryCollection that has its own geometries made up of a series of LineStrings.
Instead, I want each district to be of type Polygon that just has arcs.
For example, the first object is:
{
"type": "GeometryCollection",
"geometries": [{
"type": "GeometryCollection",
"properties": {
"district_number": 1,
"district_name": "Ghaziabad"
},
"geometries": [{
"type": "LineString",
"arcs": [0]
}, {
"type": "LineString",
"arcs": [1]
}, {
"type": "LineString",
"arcs": [2]
}, {
"type": "LineString",
"arcs": [3]
}, {
"type": "LineString",
"arcs": [4]
}, {
"type": "LineString",
"arcs": [5]
}]
}
I think I want to convert it, and every other object, to:
{
"type": "Polygon",
"properties": {
"district_number": 1,
"district_name": "Ghaziabad"
},
"arcs": [[0,1,2,3,4,5]]
}
I could fix it manually, but that seems insane. Is there a better way?
Update
So I figured out how to convert the object into the result I thought I wanted, but I got some very wacky polygons. Here is my (very clunky) code. Thanks to Saeed Adel Mehraban for some guidance with this.
d3.json('map.topojson',function(error,data){ // get my json that needs to be converted
var arr = data.objects.collection.geometries; // this is the relevant array
var newArr = []; // in order to map each object, i need to put each one into a new array as a single-item array
arr.forEach(function(d,i){
var curr = [d];
newArr.push(curr);
})
newArr.forEach(function(e,i){ // now that i have my new array, i want to convert each object that contains a LineString into a Polygon
var output = e.map(function(d){
var arcsArr = []; // an empty array to push each single value of the LineString arcs into
return {
"type": "Polygon", // change the type to polygon
"properties": d.properties, // keep the properties
"arcs": d.geometries.map(function(g) { // a single key-value pair for arcs, made up of the individual arcs from the LineString
arcsArr.push(g.arcs[0]);
return [arcsArr]; // the array of arcs must be in another array
})
};
});
var output = output[0]; // get only the first item in the output array, which is the object i have modified
output.arcs = output.arcs[0]; // and change the arcs so we're only taking the first array (as we've duplicated the arrays)
$('body').append(JSON.stringify(output)+','); // append the whole thing to the body so I can copy it and paste it into the appropriate part of the JSON
});
});
This "worked" in the sense that my LineStrings were indeed converted to Polygons, retaining the original border. But the polygons themselves are a nightmare, with straight lines crisscrossing the map at all kinds of angles.
Is there something like a command line tool that can convert boundaries made of LineStrings into Polygons?
I believe I ran into the same problem being described.
This is Zambia drawn as a svg polyline foreach arc (red being the first arc listed, and magenta being the last):
However when attempting to create a polygon by concatenating the arcs:
What happened was the arcs for every object were listed clockwise, but the points in every individual arc were listed counterclockwise. Without seeing the topojson that OP is using I cannot 100% confirm this, but I suspect that this was the case.
I solved this by reversing the points in an arc before pushing them to the array of points to draw the polygon and now all is well:
Maybe a map function like below? (I write that with simplistic assumption about data schema. I can't guarantee that it works for complex linestrings since I'm not familiar with topojson format. But it works with your provided data)
var foo = [
{
"type": "GeometryCollection",
"geometries": [{
"type": "GeometryCollection",
"properties": {
"district_number": 1,
"district_name": "Ghaziabad"
},
"geometries": [{
"type": "LineString",
"arcs": [0]
}, {
"type": "LineString",
"arcs": [1]
}, {
"type": "LineString",
"arcs": [2]
}, {
"type": "LineString",
"arcs": [3]
}, {
"type": "LineString",
"arcs": [4]
}, {
"type": "LineString",
"arcs": [5]
}]
}]
}
];
var bar = foo.map(function(d) {
return {
"type": "Polygon",
"properties": d.geometries[0].properties,
"arc": d.geometries.map(function(g1) {
return g1.geometries.map(function(g) {
return g.arcs[0];
});
})
};
});
console.log(bar);
Using Elasticsearch 2.3.1 and the python client I'm getting unexpected results with geo search. In the code below I create a simple index. I then index two shapes - a big square and a small square completely contained in that square (look at them here). I then query the index using with the small square using all available strategies {'intersects', 'disjoint', 'within', 'contains'}.
from elasticsearch import Elasticsearch
es_client = Elasticsearch()
# CREATE INDEX
INDEX = "geo_shapes_2"
DOC_TYPE = "metros"
es_client.indices.delete(INDEX, ignore=404)
body = {
"mappings": {
"metro": {
"properties": {
"id": {
"type": "string",
"index": "not_analyzed"
},
"geometry": {
"type": "geo_shape",
"tree": "quadtree",
"precision": "50m",
"distance_error_pct": 0.025
}
}
}
}
}
es_client.indices.create(INDEX, body)
# INDEX TWO DOCS
# small_square is completely contained within big_square
small_square = 'small square'
body = {
'id': small_square,
'geometry': {
"type": "Polygon",
"coordinates": [
[
[
-106.248779296875,
35.12889434101051
],
[
-106.34765625,
35.88014896488361
],
[
-105.062255859375,
35.8356283888737
],
[
-105.13916015625,
35.05698043137265
],
[
-106.248779296875,
35.12889434101051
]
]
]
}
}
es_client.create(INDEX, DOC_TYPE, body=body, id=small_square)
big_square = 'big square'
body = {
'id': big_square,
'geometry': {
"type": "Polygon",
"coordinates": [
[
[
-108.97338867187499,
32.93492866908233
],
[
-108.69873046875,
38.19502155795573
],
[
-102.095947265625,
38.12591462924157
],
[
-102.39257812499999,
32.87036022808355
],
[
-108.97338867187499,
32.93492866908233
]
]
]
}
}
es_client.create(INDEX, DOC_TYPE, body=body, id=big_square)
# ISSUE ALL 4 STRATEGY QUERIES FOR SMALL SQUARE
body = {
'filter': {
'geo_shape': {
'geometry': {
'indexed_shape': {
'id': small_square,
'type': DOC_TYPE,
'index': INDEX,
'path': 'geometry',
},
}
}
}
}
for strategy in ['intersects', 'disjoint', 'within', 'contains']:
body['filter']['geo_shape']['geometry']['relation'] = strategy
response = es_client.search(INDEX, body=body)
print strategy + ': ' + (', '.join([r['_source']['id'] for r in response['hits']['hits']]) or 'none')
Here I would expect the response to be:
intersects: big square, small square
disjoint: none
within: none (or maybe include small square depending upon the semantics of "within")
contains: big square, small square (or maybe omit small square depending upon the semantics of "contains")
Instead I see:
intersects: none
disjoint: big square, small square
within: none
and this is followed by an error
...
/Library/Python/2.7/site-packages/elasticsearch/connection/base.pyc in _raise_error(self, status_code, raw_data)
106 pass
107
--> 108 raise HTTP_EXCEPTIONS.get(status_code, TransportError)(status_code, error_message, additional_info)
109
110
RequestError: TransportError(400, u'search_phase_execution_exception', u'')
Am I doing something wrong? Or is this a bug?
Figured it out mostly. The above code has a typo in the mapping where the doc type is 'metro' but everywhere else I use DOC_TYPE of 'metros'.
But contains still returns an error.