I have an Index with nested Objects something like
"_index": "originindex",
"_source": {
"message": "",
"environment": "",
"nestedObj": {
"field1": "field1",
"field2": 1 },
"anotherfield": 1}
And I want to reindexit to something like
"_index": "newindex",
"_source": {
"message": "",
"nestedObj-field1":"field1",
"nestedObj-field2": 1 ,
"anotherfield": 1}
I'am new to all of this I'm using Nest on .Net V4.5, it proposes a ReindexAPI But don'tknow how to use it for this purpose
Thank you!
POST _reindex
{
"source": {
"index": "originindex"
},
"dest": {
"index": "newindex"
},
"script":{
"source":"ctx._source.nestedObj-field1 = ctx._source.remove(\"field1\");ctx._source.nestedObj-field2 = ctx._source.remove(\"field2\");"
}
Just make sure your mappings are in place on the dest index before you execute this.
Related
I'm new to Elasticsearch, and I cannot find a Delete query.
Here is an example of an document in myIndex :
{
"_index": "myIndex",
"_type": "_doc",
"_id": "IPc5kn8Bq7SuVr5qM9dq",
"_score": 1,
"_source": {
"code": "1234567",
"matches": [
{
"hostname": "hostnameA.com",
"url": "https://www.hostnameA.com/....",
},
{
"hostname": "hostnameB.com",
"url": "https://www.hostnameB.com/....",
},
{
"hostname": "hostnameC.com",
"url": "https://www.hostnameC.com/....",
},
{
"hostname": "hostnameD.com",
"url": "https://www.hostnameD.com/....",
},
]
}
}
Let's say this index contains 10k documents.
I would like a query to remove all the item from my array matches where the hostname is equal to hostnameC.com, and keeping all the others.
Anyone would have an idea to help me?
Is it possible to set particular nested fields for reindexing?
According to docs https://www.elastic.co/guide/en/elasticsearch/reference/current/docs-reindex.html#docs-reindex-filter-source, selected fields are array.
POST _reindex
{
"source": {
"index": "twitter",
"_source": ["user", "_doc"]
},
"dest": {
"index": "new_twitter"
}
}
For example, we need reindex only nested fields of user like "name" and "birthdate":
How could it be done? We need something like this:
POST _reindex
{
"source": {
"index": "twitter",
"_source": { "user": ["name", "birthdate"], "_doc"]
},
"dest": {
"index": "new_twitter"
}
}
POST _reindex
{
"source": {
"index": "twitter",
"_source": [ "user.name", "user. birthdate", "_doc"]
},
"dest": {
"index": "twitter_new"
}
}
}
You need to use . to refer them.
I would like to remove member2 from members. I saw script
ctx._source.list_data.removeIf{list_item -> list_item.list_id == remove_id}
for a list but in my case it's not working. Is that possible?
"_index": "test",
"_type": "test",
"_id": "5",
"_score": 1.0,
"_source": {
"id": "1",
"description": "desc",
"name": "ss",
"members": {
"member1": {
"id": "2",
"role": "owner"
},
"member2": {
"role": "owner",
"id": "3"
}
}
}
}
You can use the update API:
POST test/_update/5
{
"script": "ctx._source.members.remove('member2')"
}
removeIf is for list. Your members2 is of type object so you need to use remove
{
"script": "if(ctx._source.members.member2.id=='3')
ctx._source.members.remove('member2')"
}
I have an index for which I have set the mapping to "dynamic":"strict".
As expected, for the most part, if a field that is not listed in the mapping is introduced, Elasticsearch will reject it.
However I am finding that any field with a null value is not caught, and will make it into my index. Here is what my mapping looks like:
{
"myindex": {
"mappings": {
"mystuff": {
"dynamic": "strict",
"_id": {
"store": true,
"index": "not_analyzed"
},
"_timestamp": {
"enabled": true,
"store": true
},
"_index": {
"enabled": true
},
"_type": {
"store": true
},
"properties": {
"entitlements": {
"type": "nested",
"properties": {
"accountNumber": {
"type": "string",
"index": "not_analyzed"
},
"active": {
"type": "string",
"index": "not_analyzed"
},
"assetEndDate": {
"type": "date",
"format": "date_time_no_millis"
}
}
}
}
}
}
}
}
EDIT (including example scenarios)
With the mapping above, here are the scenarios I am seeing:
1) When Posting a valid document (one that follows the mapping), 200 OK.
posted document:
{
"entitlements": [
{
"accountNumber": "123213",
"active": "true",
"assetEndDate": "2016-10-13T00:00:00Z"
}
]
}
elasticsearch response:
{
"_index": "myindex",
"_type": "mystuff",
"_id": "5",
"_version": 1,
"created": true
}
2) When posting an invalid document (one that does not follow the mapping), 400 StrictDynamicMappingException.
posted document:
{
"entitlements": [
{
"accountNumber":"123213",
"XXXXactive": "true",
"assetEndDate": "2016-10-13T00:00:00Z"
}
]
}
elasticsearch response:
{
"error": "StrictDynamicMappingException[mapping set to strict, dynamic introduction of [Xactive] within [entitlements] is not allowed]",
"status": 400
}
3) When posting an invalid document (one that does not follow the mapping) with a value that is null for the invalid field, 200 OK.
posted document:
{
"entitlements": [
{
"accountNumber":"123213",
"XXXXactive": null,
"assetEndDate": "2016-10-13T00:00:00Z"
}
]
}
elasticsearch response:
{
"_index": "myindex",
"_type": "mystuff",
"_id": "7",
"_version": 1,
"created": true
}
4) When posting an invalid document (one that does not follow the mapping) with a value that is null for the invalid field, 200 OK.
posted document:
{
"entitlements": [
{
"accountNumber":"123213",
"XXXXactive": null,
"assetEndDate": "2016-10-13T00:00:00Z",
"THIS_SHOULD_NOT_BE_HERE": null
}
]
}
elasticsearch response:
{
"_index": "myindex",
"_type": "mystuff",
"_id": "9",
"_version": 1,
"created": true
}
It is the 3rd and 4th scenarios, that I am concerned about.
It looks like this issue (or one very similar) was raised one the Elasticsearch git repository here and has since been closed. However, the problem appears to still be an issue in version 1.7 .
This is being seen locally, as well as on indexes I have deployed with AWS Elasticsearch Service.
Am I making a mistake somewhere, or Has anyone found a solution to this problem ?
I have a index like this,
"_index": "test",
"_type": "products",
"_id": "URpYIFBAQRiPPu1BFOZiQg",
"_score": null,
"_source": {
"currency": null,
"colors": [],
"api": 1,
"sku": 9999227900050002,
"category_path": [
{
"id": "cat00000",
"name": "B1"
},
{
"id": "abcat0400000",
"name": "Cameras & Camcorders"
},
{
"id": "abcat0401000",
"name": "Digital Cameras"
},
{
"id": "abcat0401005",
"name": "Digital SLR Cameras"
},
{
"id": "pcmcat180400050006",
"name": "DSLR Package Deals"
}
],
"price": 1034.99,
"status": 1,
"description": null,
}
And i want to search only exact text ["Camcorders"] in category_path field.
I did some match query, but it search all the products which has "Camcorders" as a part of the text. Can some one help me to solve this.
Thanks
To search in nested field use like following query
{
"query": {
"term": {
"category_path.name": {
"value": "b1"
}
}
}
}
HOpe it helps..!
you could add one more nested field raw_name with not_analyzed analyzer and match against it.