I have a problem with Elasticsearch. vid is a long field in my index. I try to search with:
POST my-index/video/_search
{
"query": {
"term": {
"vid": 6518061120253592000
}
}
}
but get nothing. Try with other fields (not long type), it's successful.
I'm sure that the document is existed in Elasticsearch.
My Elasticsearch version is 5.4.0
Please help me. Thanks!
I finally found the root cause here:
https://github.com/elastic/elasticsearch/issues/15292
Related
In elasticsearch term query documentation: https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-term-query.html, it is stated there's a case_insensitive field
However, I can't get to set it. I can set boost and value without issue, but not case_insensitive.
GET movies/_search
{
"query": {
"term": {
"overview": {
"value" : "batman",
"boost": 0.5,
"case-insensitive": true
}
}
}
}
When I run, I get the error state "[term] query does not support [case_insensitive]".
Where did I get it wrong, or the documentation is wrong?
Looks like you are on less than the ES 7.10.0 version where it was not present, Even if you check the documentation of ES 7.9 option of case_insensitive is not present.
Please find the related link of Github issue and PR which added support of case-insensitive to term query.
Please refer to this diff where caseInsensitive field was added to TermQuery.
I am working with Elasticsearch and I am getting a query error:
elasticsearch.exceptions.TransportError: TransportError(500, 'search_phase_execution_exception', 'script score query returned an invalid score: NaN for doc: 32894')
It seems like my metric is returning NaN for document 32894 (NaN for doc: 32894). Naturally, the next step is to look at that document to see if there is anything wrong with it.
The problem is that I upload my documents using my own ID, so "32894" is meaningless for me.
A query like
curl -X GET "localhost:9200/my_index/_doc/one_of_my_ids?pretty&pretty"
works fine, but this fails if I try with the doc number from the error message.
I expected this to be trivial, but some Google has failed to help.
How can I then find this document? Or is using my own IDs not recommended and the unfixable source of this problem?
Edit: as requested, this is the query that fails. Note that obviously fixing this is my ultimate goal, but not the specific point of this question. Help appreciated in either case.
I am using the elasticsearch library in Python.
self.es.search(index=my_index, body=query_body, size=number_results)
With
query_body = {
"query": {
"script_score": {
"query": {"match_all": {}},
"script": {
"source": "cosineSimilaritySparse(params.queryVector, doc['embedding']) + 10.0",
"params": {"queryVector": query_vector}
}
}
}
}
Is it possible to delete all the documents of a particular type in the elasticsearch index ?
- Does it affect the type mapping too ?
- I want to retain the mapping for that type.
Using elasticsearch 2.2
Found an answer related to this here. Following content is directly from that answer.
You can use the delete-by-query plugin43 for that. Here's an example:
We create an index with two types and add some documents:
POST /_bulk
{"index":{"_index":"mammals","_type":"people"}}
{"tag_line":"I am Mike"}
{"index":{"_index":"mammals","_type":"people"}}
{"tag_line":"I am Hanna"}
{"index":{"_index":"mammals","_type":"people"}}
{"tag_line":"I am Bert"}
{"index":{"_index":"mammals","_type":"animals"}}
{"tag_line":"I am a dog"}
{"index":{"_index":"mammals","_type":"animals"}}
{"tag_line":"I am a cat"}
When we query for all documents, we get 5 results:
GET /mammals/_search?size=0
{
"query": {
"match_all": {}
}
}
Now we can delete all documents of the type "animals":
DELETE /mammals/animals/_query
{
"query": {
"match_all": {}
}
}
This will only work when the delete-by-query plugin is installed.
When we search once again for all documents, we only get 3 results as the animals are gone.
P.S: This plugin is there in 2.x version and not there in 5.x. So in 5.x there can be other ways to do this. I believe that this deletion does not affect the mapping because this just deletes individual documents.
When a time-based index is added to kibana, you have to pick the field that will act as a time field. If you want to switch from one field to another, normally I would delete the index and re-add it back. But you end up loosing scripted fields and filed formatting this way.
Is there any way to modify the existing index time field without loosing scripted fields/formatting?
It can probably be done by messing around directly with /.kibana/index-pattern/index_pattern_name but all my attempts with changing timeFieldName directly ended up dropping scripted fields.
This is what worked for me on Kibana 7.7.0 (Elastic Cloud):
find the {id} of the document whose title field corresponds to the index you want to make change
GET .kibana/_search
{
"query": {
"match": {
"index-pattern.title": "{NAME_OF_THE_INDEX}"
}
}
}
change the timefield with following code
POST .kibana/_update/{id}
{
"doc": {
"index-pattern": {
"timeFieldName" : "{NEW_TIME_FIELD_NAME}"
}
}
}
The easiest way seems to update the corresponding document:
POST /.kibana/index-pattern/YOUR_INDEX_NAME/_update
{
"doc": {
"timeFieldName": "NEW_TIME_FIELD_NAME"
}
}
It should preserve scripted fields.
This doesn't seem to work on Kibana 5.
Instead, following is the way worked on Kibana 5.
1. find the {id} of the document whose title field corresponds to the index you want to make change
GET .kibana/index-pattern/_search
{
"_source" : "_id",
"query" : {
"match" : {
"title": "{NAME_OF_THE_INDEX}"
}
}
}
2. change the timefield with following code
POST /.kibana/index-pattern/{id}/_update
{
"doc": {
"timeFieldName" : "{NEW_TIME_FIELD_NAME}"
}
}
This worked fine with me on Kibana 5.
I have documents which contains only "url"(analyzed) and "respsize"(not_analyzed) fields at first. I want to update documents that match the url and add new field "category"
I mean;
at first doc1:
{
"url":"http://stackoverflow.com/users/4005632/mehmet-yener-yilmaz",
"respsize":"500"
}
I have an external data and I know "stackoverflow.com" belongs to category 10,
And I need to update the doc, and make it like:
{
"url":"http://stackoverflow.com/users/4005632/mehmet-yener-yilmaz",
"respsize":"500",
"category":"10"
}
Of course I will do this all documents which url fields has "stackoverflow.com"
and I need the update each doc oly once.. Because category data of url is not changeable, no need to update again.
I need to use _update api with _version number to check it but cant compose the dsl query.
EDIT
I run this and looks works fine:
But documents not changed..
Although query result looks true, new field not added to docs, need refresh or etc?
You could use the update by query plugin in order to do just that. The idea is to select all document without a category and whose url matches a certain string and add the category you wish.
curl -XPOST 'localhost:9200/webproxylog/_update_by_query' -H "Content-Type: application/json" -d '
{
"query": {
"filtered": {
"filter": {
"bool": {
"must": [
{
"term": {
"url": "stackoverflow.com"
}
},
{
"missing": {
"field": "category"
}
}
]
}
}
}
},
"script" : "ctx._source.category = \"10\";"
}'
After running this, all your documents with url: stackoverflow.com that don't have a category, will get category: 10. You can run the same query again later to fix new stackoverflow.com documents that have been indexed in the meantime.
Also make sure to enable scripting in elasticsearch.yml and restart ES:
script.inline: on
script.indexed: on
In the script, you're free to add as many fields as you want, e.g.
...
"script" : "ctx._source.category1 = \"10\"; ctx._source.category2 = \"20\";"
UPDATE
ES 2.3 now features the update by query functionality. You can still use the above query exactly as is and it will work (except that filtered and missing are deprecated, but still working ;).
That all sounds great but just to add to #Val answer, Update By Query is available form ElasticSearch 2.x but not for earlier versions. In our case we're using 1.4 for legacy reasons and there is no chance of upgrading in forseeable future so another solution is using the Update by query plugin provided here: https://github.com/yakaz/elasticsearch-action-updatebyquery