I'm trying to get Kibana to see my location info as a geo_point. I've been running about mapping and how to do it, but I can't seem to figure it out. The data looks like this in mongodb:
{ "_id" : "3", "loc" : "[-122.0574, 37.41919999999999]", "fs" : "Clean", "name" : "www.googleapis.com", "timestamp" : "2016-07-02T21:02:53.623Z", "destination" : "192.168.79.136", "source" : "216.58.212.138", "vt" : "0" }
How would I map the values that are stored in the key "loc" as geo_point?
In mongoDB the first value is longitude and the next is latitude.You can add a geo point data type to your elasticsearch mapping.
PUT my_index
{
"mappings": {
"my_type": {
"properties": {
"loc": {
"type": "geo_point"
}
}
}
}
}
And when you add data,
PUT my_index/my_type/3
{
"loc": {
"lat": 37.41919999999999,
"lon": -122.0574
}
}
Refer to this
Related
Followings are the steps on how using nested field in elastersearch.
First step:
curl -XPUT 'localhost:9200/my_index/my_type/1?pretty' -d'
{
"group" : "fans",
"user" : [ // 1
{
"first" : "John",
"last" : "Smith"
},
{
"first" : "Alice",
"last" : "White"
}
]
}'
Second step:
curl -XPUT 'localhost:9200/my_index?pretty' -d'
{
"mappings": {
"my_type": {
"properties": {
"user": {
"type": "nested" // 1
}
}
}
}
}'
Before i copy the code, i have delete all the index on my machine.
However, after running step 2, something went woring like the following .
{
"error" : {
"root_cause" : [
{
"type" : "resource_already_exists_exception",
"reason" : "index [my_index/yHhgr8iEQqGnHo5Ugex2dA] already exists",
"index_uuid" : "yHhgr8iEQqGnHo5Ugex2dA",
"index" : "my_index"
}
],
"type" : "resource_already_exists_exception",
"reason" : "index [my_index/yHhgr8iEQqGnHo5Ugex2dA] already exists",
"index_uuid" : "yHhgr8iEQqGnHo5Ugex2dA",
"index" : "my_index"
},
"status" : 400
}
I really don't konw what to do about this.(I have also tried create nested field first. It also went wrong)
I'm new to elastersearch, really need help. Thankyou very mutch!!!
Since you are using Elasticsearch version 7.10, you cannot add the mapping type in the index mapping definition. Refer to this to know more about the removal of mapping types.
You can not change the mapping of an index that already exists, you need to delete it and index the data with the new mapping, or reindex into a new index with the new mapping.
You need to first create the index with the following mapping:
PUT /my_index
{
"mappings": {
"properties": {
"user": {
"type": "nested" // 1
}
}
}
}
And then index the documents into the index. Refer to this official documentation, to know more about nested type.
PUT /my_index/_doc/1
{
"group" : "fans",
"user" : [ // 1
{
"first" : "John",
"last" : "Smith"
},
{
"first" : "Alice",
"last" : "White"
}
]
}
Is it possible to create an initial script for Elasticsearch?
For example, I prepare one JSON file with index 20 users and 20 books.
I want to load it by the single request.
Example file:
PUT eyes
{
"settings" : {
"number_of_shards" : 1
},
"mappings" : {
"_doc" : {
"properties" : {
"name" : { "type" : "text" },
"color" : { "type" : "text" }
}
}
}
}
PUT eyes/_doc/1
{
"name": "XXX"
"color" : "red"
}
PUT eyes/_doc/2
{
"name": "XXXX"
"color" : "blue"
}
You can use bulk API for populating your index in one single call.
https://www.elastic.co/guide/en/elasticsearch/reference/master/docs-bulk.html
PUT /eyes/_doc/_bulk
{"index":{"_id":1}}
{"name":"XXX","color":"red"}
{"index":{"_id":2}}
{"name":"XXX","color":"blue"}
{"index":{"_id":3}}
{"name":"XXX","color":"green"}
I have defined an Index in elastic cache 6
PUT my_index
{
"mappings": {
"_doc": {
"properties": {
"user": {
"type": "nested"
}
}
}
}
}
and loaded some same data as follows
PUT my_index/_doc/1
{
"group" : "coach",
"user" : [
{
"first" : "John",
"last" : "Frank"
},
{
"first" : "Hero",
"last" : "tim"
}
]
}
PUT my_index/_doc/2
{
"group" : "team",
"user" : [
{
"first" : "John",
"last" : "term"
},
{
"first" : "david",
"last" : "gayle"
}
]
}
Now I am trying to search in the discover page or the visualization page, but I receive a blank
after a bit of trial and error and googling around i found that does not support nested type for aggregation and search out of the box. To enable this you must install a plugin and the best plugin i found is listed below.
https://ppadovani.github.io/knql_plugin/overview/
The plugin provides all the features from the discover tab to the visualization tab.
elasticsearch update with partial document overwrite the original document instead of merging it.
I thought merge will just update corresponding properties and or insert new ones. did I miss what the merge supposed to do?
This is how I did:
mappings:
PUT my_index
{
"mappings": {
"my_type": {
"properties": {
"user": {
"type": "nested"
}
}
}
}
}
index doc:
PUT my_index/my_type/1
{
"group" : "fans",
"user" : [
{
"first" : "John",
"last" : "Doe",
"age": 31
},
{
"first" : "Foo",
"last" : "Bar",
"age" : 26
}
]
}
partial update:
POST my_index/my_type/1/_update
{
"doc": {
"group" : "fans",
"user" : [
{
"first" : "Joe",
"last" : "Smith",
},
{
"first" : "Alice",
"last" : "Baz"
}
]
}
}
the result is just new document without the age property.how can I preserve the properties not in the partial update?
Your POST as provided is not using the same index and type names as the original PUT. Try POST my_index/my_type/1/_update instead of POST test/type1/1/_update.
I'm putting data in ES and check the mapping which is created,
I'm executing this:
curl -XPOST 'http://localhost:9200/testnested2/type1/0' -d '{
"p1": ["1","2","3","4"],
"users" : {
"first" : "John",
"last" : "Sm11ith"
}
}'
and this is the schema it created:
{
"testnested2":{
"mappings":{
"type1":{
"properties":{
"p1":{"type":"string"},
"users":{
"properties":{
"first":{"type":"string"},
"last":{"type":"string"}
}
}
}
}
}
}
}
I was wondering if it's possible to tell it that "users" is nested or I have to create the mapping for myself.
I would like that ES could create an shema like this:
curl -XPOST http://180.5.5.93:9200/testnested3 -d '{
"settings" : {
"number_of_shards" : 1
},
"mappings" : {
"type1" : {
"properties" : {
"propiedad1" : { "type" : "string"},
"users" : {
"type" : "nested",
"include_in_parent": true,
"properties": {
"first" : {"type": "string" },
"last" : {"type": "string" }
}
}
}
}
}
}'
By default, the dynamic mapping feature of ElasticSearch will map users as an object instead of nested.
If you want to tune this behavior, you have to define explicitely a users attribute as nested either in :
the type1 mapping
the default mapping of your index. This way, for any type created, the users attribute will be set automatically to nested(see here for default mapping information)