I tried to this solution from here, but it didn't worked for me.
I needed to convert the nested text fields like one.second.third to ip type. So I added mapping like
PUT .the-index-2022.12.20-0000025/_mapping
{
"properties": {
"one": {
"type": "object",
"properties": {
"second": {
"type":"object",
"properties": {
"ip_address1": {
"type": "ip"
},
"ip_address2": {
"type": "ip"
}
}
}
}
}
}
}
Then I used
POST .the-index-2022.12.20-0000025/_update_by_query?wait_for_completion=false&slices=auto
It executed successfully but still didnt change the fields type to ip.
What can I do else or what is the problem here?
Thanks in advance
Also tried like:
PUT .the-index-2022.12.20-0000025/_mapping
{
"properties": {
"one": {
"properties": {
"second": {
"properties": {
"ip_address1": {
"type": "ip"
},
"ip_address2": {
"type": "ip"
}
}
}
}
}
}
}
Could you please help?
I've added 3 new fields in ip type:
{
".the-index-2022.12.20-0000025" : {
"mappings" : {
"one.sec.ip1" : {
"full_name" : "one.sec.ip1",
"mapping" : {
"ip1" : {
"type" : "ip"
}
}
},
"one.sec.ip2" : {
"full_name" : "one.sec.ip2",
"mapping" : {
"ip2" : {
"type" : "ip"
}
}
},
"one.sec.ip3" : {
"full_name" : "one.sec.ip3",
"mapping" : {
"ip3" : {
"type" : "ip"
}
}
}
}
}
}
Related
I have the following mapping
"mappings": {
"dynamic": "strict",
"properties": {
"_class": {
"type": "keyword"
},
"id": {
"type": "keyword"
},
...
"user": {
"properties": {
"username": {
"type": "keyword"
},
"userType": {
"type": "keyword"
}
}
},
...
My java object looks like this
class foo {
String id,
...
User user;
...
}
class User {
String username;
UserType userType; // this is enum
}
In my repository I am trying to find by
public List<Foo> findByUser(User user);
This will not return anything, I double and triple checked the data and mapping and the user object that I am passing. All looks good, but I won't get any results.
If I write a query like so, it does work:
{
"size": 100,
"query": {
"bool" : {
"must" : [
{
"term" : {
"user.username" : {
"value" : "user1"
}
}
},
{
"term" : {
"user.userType" : {
"value" : "ACTIVE"
}
}
}
]
}
}
}
What am I doing wrong?
I have been trying to update my mapping but not able to do that. Majorly this question is related to updating the nested part. Suppose there is a field "Anand" which contains a field "hello"
{
"properties": {
"anand": {
"hello": {
"type": "short"
}
}
}
}
But I am getting the error
"error" : {
"root_cause" : [
{
"type" : "mapper_parsing_exception",
"reason" : "No type specified for field [anand]"
}
],
"type" : "mapper_parsing_exception",
"reason" : "No type specified for field [anand]"
},
"status" : 400
}
Current Mapping is
{
"anandschool" : {
"mappings" : {
"properties" : {
"anand" : {
"type" : "nested"
},
"doc" : {
"properties" : {
"properties" : {
"properties" : {
"shop_tier" : {
"type" : "long"
}
}
}
}
},
"message" : {
"type" : "byte"
},
"properties" : {
"properties" : {
"shop_tier" : {
"type" : "long"
},
"shop_type" : {
"type" : "text",
"fields" : {
"keyword" : {
"type" : "keyword",
"ignore_above" : 256
}
}
}
}
},
"shop" : {
"type" : "long"
}
}
}
}
}
I even created a nested type anand so that it can work
{
"properties": {
"anand": {
"type": "nested"
}
}
}
Self Answer
When updating mapping for nested things need to update properties of the nested field.
For above example update by
"properties": {
"anand": {
"properties":{
"hello": {
"type": "short"
}
}
}
}
}
THough This will not work inside a field which is nested.Ex if anand type was "nested", it would not work. If anyone knows solution for that let me know.
Hi All I am using elasticsearch-template.json to set data type of all of my fields to string. Below is the snippet of the template:
{
"template": "logstash-*",
"settings": {
"index.refresh_interval": "5s",
"number_of_shards": 1,
"number_of_replicas": 0
},
"mappings": {
"logs": {
"_all": {
"enabled": true
},
"properties": {
"level1": {
"properties": {
"level2": {
"properties": {
"_all": {"type": "string"}
}
}
}
}
}
}
}
}
Here under level2 i have got lots of fields which get created i want to set all of them to string how can i set it. I have tried "*" character as well as "%" character to select all the fields. but unfortunately it only gets added as a new field to the mapping. How to specify in template to select all the fields under a certain level?
I believe what you are looking for is a dynamic_templates and using path_match instead of match. This demonstrates how that might work:
curl -DELETE localhost:9200/test-*
curl -XDELETE http://localhost:9200/_template/test
curl -XPOST http://localhost:9200/_template/test -d '
{
"template": "test-*",
"mappings": {
"_default_": {
"dynamic_templates": [
{
"level1_level2_all": {
"path_match": "level1.level2.*",
"match_mapping_type": "*",
"mapping": {
"index": "not_analyzed",
"type": "string"
}
}
}
]
}
}
}
'
curl -XPOST http://localhost:9200/test-1/a -d '
{
"level1": {
"level2": {
"x":1
}
}
}'
curl -XPOST http://localhost:9200/test-1/a -d '
{
"level1": {
"level2": {
"y":1
}
}
}'
curl http://localhost:9200/test-1/_mapping?pretty
The output of which is:
"test-1" : {
"mappings" : {
"_default_" : {
"dynamic_templates" : [ {
"level1_level2_all" : {
"mapping" : {
"index" : "not_analyzed",
"type" : "string"
},
"match_mapping_type" : "*",
"path_match" : "level1.level2.*"
}
} ],
"properties" : { }
},
"a" : {
"dynamic_templates" : [ {
"level1_level2_all" : {
"mapping" : {
"index" : "not_analyzed",
"type" : "string"
},
"match_mapping_type" : "*",
"path_match" : "level1.level2.*"
}
} ],
"properties" : {
"level1" : {
"properties" : {
"level2" : {
"properties" : {
"x" : {
"type" : "string",
"index" : "not_analyzed"
},
"y" : {
"type" : "string",
"index" : "not_analyzed"
}
}
}
}
}
}
}
}
}
}
Say I have a document like this:
{
"events" : [
{
"event_id" : 123,
"props" : {
"version": "33"
},
{
"event_id" : 124,
"props" : {
"version": "44a"
}
]
}
Is it possible to specify that the events.props.version be mapped to some type?
I've tried:
{
"template" : "logstash-*",
...
"mappings" : {
"_default_" : {
"properties" : {
"events.props.version" : { "type" : "string" }
}
}
}
}
But that doesn't seem to work.
Please have a look at mapping API in elasticsearch Mapping API.
To set any analyzer in the inner element we need to consider each and every inner field as a separate properties set. try the following
{
"mappings": {
"properties": {
"events": {
"properties": {
"event_id": {
"type": "string",
"analyzer": "keyword"
},
"props": {
"properties": {
"version": {
"type": "string"
}
}
}
}
}
}
}
}
if this not works please provide me you mapping.
Sure, but you need to use the "object" type:
From the doc ( https://www.elastic.co/guide/en/elasticsearch/reference/1.5/mapping-object-type.html ) if you want to map
{
"tweet" : {
"person" : {
"name" : {
"first_name" : "Shay",
"last_name" : "Banon"
},
"sid" : "12345"
},
"message" : "This is a tweet!"
}
}
you can write:
{
"tweet" : {
"properties" : {
"person" : {
"type" : "object",
"properties" : {
"name" : {
"type" : "object",
"properties" : {
"first_name" : {"type" : "string"},
"last_name" : {"type" : "string"}
}
},
"sid" : {"type" : "string", "index" : "not_analyzed"}
}
},
"message" : {"type" : "string"}
}
}
}
I have documents in elastic search in the following format
{
"stringindex" : {
"mappings" : {
"files" : {
"properties" : {
"BaseOfCode" : {
"type" : "long"
},
"BaseOfData" : {
"type" : "long"
},
"Characteristics" : {
"type" : "long"
},
"FileType" : {
"type" : "long"
},
"Id" : {
"type" : "string"
},
"Strings" : {
"properties" : {
"FileOffset" : {
"type" : "long"
},
"RO_BaseOfCode" : {
"type" : "long"
},
"SectionName" : {
"type" : "string"
},
"SectionOffset" : {
"type" : "long"
},
"String" : {
"type" : "string"
}
}
},
"SubSystem" : {
"type" : "long"
}
}
}
}
}
}
My requirement is when I search for a particular string (String.string) i want to get only the FileOffSet (String.FileOffSet) for that string.
How do i do this?
Thanks
I suppose that you want to perform a nested query and retrieve only one field as the result, but I see problems in your mapping, hence I will split my answer in 3 sections:
What is the problem I see:
How to query nested fields (this is more ES background):
How to find a solution:
1) What is the problem I see:
You want to query a nested field, but you don't have a nested field.
The nested field part:
The field "Strings" is not nested in the type "files" (nested data without a nested field may bring future problems), otherwise your mapping for the field "Strings" would be something like this:
{
"stringindex" : {
"mappings" : {
"files" : {
"properties" : {
"Strings" : {
"properties" : {
"type" : "nested",
"String" : {
"type" : "string"
}
}
}
}
}
}
}
}
Note: yes, I cut most of the fields, but I did this to easily show that you didn't create a nested field.
With a nested field "in hands", we need a nested query.
The specific field result part:
To retrieve only one field as result, you have to include the property "_source" in your query.
2) How to query nested fields:
This is more for ES background, if you have never worked with nested fields.
Small example:
You define a type with a nested field:
{
"nesttype" : {
"properties" : {
"name" : { "type" : "string" },
"parents" : {
"type" : "nested" ,
"properties" : {
"sex" : { "type" : "string" },
"name" : { "type" : "string" }
}
}
}
}
}
You create some inputs:
{ "name" : "Dan", "parents" : [{ "name" : "John" , "sex" : "m" },
{ "name" : "Anna" , "sex" : "f" }] }
{ "name" : "Lana", "parents" : [{ "name" : "Maria" , "sex" : "f" }] }
Then you query, but only fetch the nested field "parents.name":
{
"query": {
"nested": {
"path": "parents",
"query": {
"bool": {
"must": [
{
"term": {
"sex": "m"
}
}
]
}
}
}
},
"_source" : [ "parents.name" ]
}
The output of this query is "the name of the parents of all people who have a parent of the sex 'm' ". One entry (Dan) has a father, whereas the other (Lana) doesn't. So it only will retrieve Dan's parents names.
3) How to find a solution:
To fix your mapping:
You only need to include the type "nested" in the field "Strings":
{
"files" : {
"properties" : {
...
"Strings" : {
"type" : "nested" ,
"properties" : {
"FileOffset" : { "type" : "long" },
"RO_BaseOfCode" : { "type" : "long" },
...
}
}
...
}
}
}
To query your data:
{
"query": {
"nested": {
"path": "Strings",
"query": {
"bool": {
"must": [
{
"term": {
"String": "my string"
}
}
]
}
}
}
},
"_source" : [ "Strings.FileOffSet" ]
}
Great answer by dan, but I think he didn't mention it all.
His solution don't work for your question, but I guess you even don't know that.
Consider a scenario where data is like ,
doc_1
{
"Id": 1,
"Strings": [
{
"string": "x",
"fileoffset": "f1"
},
{
"string": "y",
"fileoffset": "f2"
}
]
}
doc_2
{
"Id": 2,
"Strings": {
"string": "z",
"fileoffset": "f3"
}
}
When you run the like dan said, like say let's apply filter with Strings.string=x then response is like,
{
"hits": [
{
"_index": "stringindex",
"_type": "files",
"_id": "11961",
"_score": 1,
"_source": {
"Strings": [
{
"fileoffset": "f1"
},
{
"fileoffset": "f2"
}
]
}
}
]
}
This is because, elasticsearch will get hits from documents where any of the object inside nested field (here Strings) pass the filter criteria. (In this case in doc_1, Strings.string=x passed filter, so doc_1 is returned. But we don't know which nested object pass the criteria.
So, you have to use nested_aggregation,
Here is a solution for you..
POST index/type/_search
{
"size": 0,
"aggs": {
"StringsNested": {
"nested": {
"path": "Strings"
},
"aggs": {
"StringFilter": {
"filter": {
"term": {
"Strings.string": "x"
}
},
"aggs": {
"FileOffsets": {
"terms": {
"field": "Strings.fileoffset"
}
}
}
}
}
}
}
}
So, response is like,
"aggregations": {
"StringsNested": {
"doc_count": 2,
"StringFilter": {
"doc_count": 1,
"FileOffsets": {
"buckets": [
{
"key": "f1",
"doc_count": 1
}
]
}
}
}
}
Remember to have mapping of Strings as nested, as dan said.