Currently I have an elasticsearch index that rolls over periodically. We have an index mapping applied to a certain index pattern. We want to update the field type of the index for subsequent indices that gets rolled over.
If we change the mapping of a field from a string type to number for new rolled over indices, what happens in the index pattern when refreshed?
Would the index pattern have the field as one type over the other?
There is only one version of an index pattern at any given time. When you update it (i.e. change some mapping type), all the existing indices matching that index pattern remain unchanged. All future indices created out of that index pattern will get the modification (i.e. new field mapping type).
What you need to be aware is that you'll end up with (old) indices containing documents with a field having the old mapping type and (new) indices containing documents with a field having the new mapping type. Depending on the change you make, some of your queries running on old and new indices might not run correctly afterwards. Make sure that your queries still work with that mapping change.
Related
I've an alias setup for rolling indices in elastic search. Let's call the alias : "alias" for now. It points to a number of indexes and rolls over after every 100gb. Now, let's say the number of fields in previous indices associated with alias is 100 and I've added one more field while writing to latest index. so, the number of fields become 101.
I've setup an index pattern by the name of "alias" and I can see all the indices listed via that index pattern but I am unable to visualize the 101th field I just added in the recent indices. Is there a way to do it ?
Please let me know if more details are needed regarding the same.
Hope you added the new field in the write index that your alias is pointing to, an alias can have only one write index but can have many read index and if you added the new field to a read index of your alias, you will not be able to visualise it using your alias.
I have an index created by ElasticSearch 6.8.7. I query against some fields which don't correspond to document's fields, because they are merged copies of document's ones. At index creation their store value was set to false. Now I need to get highlights, but the query fields content is not stored. Can I update mapping and set store to true? Index's _source is enabled.
The docs don't mention this ability, and I can't try to update store on my production cluster.
No, it's not.
In general, the mapping for existing fields cannot be updated. There
are some exceptions to this rule. For instance:
new properties can be added to Object datatype fields.
new multi-fields can be added to existing fields.
the ignore_above parameter can be updated.
Source.
Also, I tried to update mapping on a sample index, and ES didn't allow me to update store value of an existing field.
That is understandable, but still sad.
I have a db of documents. Every document has a property(keyword) called index (noting to do with the elastic index) and a property(keyword) named superIndex. There can be multiple documents with the same index and multiple documents with the same superIndex in the DB, these fields are not unique.
I run a compound query searching free text on the text content of these documents, with sorting, and get the results I want. However, I get many documents having the same index and/or superIndex. Currently I programmatically filter the result list and take only the first result from each index and superIndex. My requirement is that at the end I'm left with the top results from the sort, the first from each index and superIndex.
Can this be done using elastic query. If so how?
Field collapsing allows you to collapse all search results having the same value in a field (e.g. index). (See Elasticsearch Reference: Field Collapsing)
I have a document whose structure changes often, how can I index nested documents inside it without changing the mapping on ElasticSearch?
You can index documents in Elasticsearch without providing a mapping yes.
However, Elasticsearch makes a decision about the type of a field when the first document contains a value for that field. If you add document 1 and it has a field called item_code, and in document 1 item_code is a string, Elasticsearch will set the type of field "item_code" to be string. If document 2 has an integer value in item_code Elasticsearch will have already set the type as string.
Basically, field type is index dependant, not document dependant.
This is mainly because of Apache Lucene and the way it handles this information.
If you're having a case where some data structure changes, while other doesn't, you could use an object type, http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/mapping-object-type.html.
You can even go as far as to use "enabled": false on it, which makes elasticsearch just store the data. You couldn't search it anymore, but maybe you actually don't even want that?
I'm trying to use the Wikipedia River
Is there a way / How can I customize the mapping so that ElasticSearch only index the title fields (I'd still like to access the whole text)?
The mapping is useful more to decide how you index data rather than what you index, unless you set it to dynamic: false which means that elasticsearch effectively accepts only the fields that are explicitly declared in the mapping.
The problem is that the wikipedia river always sends a set of fields for every document and this behaviour is not currently configurable, thus there's no way to index only a subset of those fields (e.g. only title and _source). What you could do is modify your search request so that you get back only the fields that you are interested in, but the content of the index will stay the same.