I have 2 indexes in Elastic search which are having the same alias name but only for one of them, is_write_index is true. Now, when I try to write to the indexes by alias name, I am getting following error:
Alias [alias_name] has more than one indices associated with it [[indexname1, indexname2]], can't execute a single index op
This is something I am doing which is failing:
let doc = {
script: {
source: ...,
lang: "painless",
params: {
...document
}
},
upsert: {
...document
}
};
await client.update({
index: alias_name,
id: docId,
body: doc
});
What can be the issue here? My thinking was if only one of them is marked as write index, it would just write to that and writes should not fail (https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-aliases.html#aliases-write-index). Am I understanding wrong?
The problem is that an update is a combo operation where the document is first retrieved using GET, then updated using the script and finally reindexed using the index operation.
What doesn't work in your case is the first operation as GET cannot be done over an alias that spans multiple indexes, even if only one of them as the is_write_index flag set to true, there's no guarantee that the document to be updated is actually in that index.
Related
Let's say I got the alias car-alias pointing to car-index-1. I now want car-alias to point to car-index-2.
I therefore perform the following POST request to the Aliases API:
{
"actions": [
{
"remove": {
"index": "car-index-1",
"alias": "car-alias"
}
},
{
"add": {
"index": "car-index-2",
"alias": "car-alias"
}
}
]
}
I receive the following response:
{
"acknowledged": true
}
Can I now immediately index data into the car-alias and it ends up in the car-index-2?
Or does the "acknowledged": true response not guarantee that write operations point to the right index immediately?
Yes, the alias is changed atomically and will point to car-index-2 immediately when the call returns.
As stated in the documentation "...during the swap, the alias has no downtime and never points to both streams at the same time."
In addition to #Val's answer:
Since in your case, the alias only points to one index, the "next" index is automatically set as the write index.
From the docs regarding the is_write_index option of the add action:
is_write_index (Optional, Boolean)
If true, sets the write index or data stream for the alias.
If an alias points to multiple indices or data streams and
is_write_index isn’t set, the alias rejects write requests. If an
index alias points to one index and is_write_index isn’t set, the
index automatically acts as the write index. [...]
Only the add action supports this parameter.
I'm facing a problem in ElasticSearch 6.1 that I cannot solve and I don't know why. I have read the docs several times and maybe I'm missing something.
I have a scripted query that needs to do some calculation before decides if a record is available or not.
Here is the following script:
https://gist.github.com/dunice/a3a8a431140ec004fdc6969f77356fdf
What I'm doing is trying to loop though an array field with the following source:
"unavailability": [
{
"starts_at": "2018-11-27T18:00:00+00:00",
"local_ends_at": "2018-11-27T15:04:00",
"local_starts_at": "2018-11-27T13:00:00",
"ends_at": "2018-11-27T20:04:00+00:00"
},
{
"starts_at": "2018-12-04T18:00:00+00:00",
"local_ends_at": "2018-12-04T15:04:00",
"local_starts_at": "2018-12-04T13:00:00",
"ends_at": "2018-12-04T20:04:00+00:00"
},
]
When the script is executed it throws the error: No field found for [unavailability] in mapping with types [aircraft]
Is there any clue to make it work?
Thanks
UPDATE
Query:
https://gist.github.com/dunice/3ccd7d83ca6ddaa63c11013b84e659aa
UPDATE 2
Mapping:
https://gist.github.com/dunice/f8caee114bbd917115a21b8b9175a439
Data example:
https://gist.github.com/dunice/8ad0602bc282b4ca19bce8ae849117ad
You cannot access an array present in the source document via doc_values (i.e. doc). You need to directly access the source document via the _source variable instead, like this:
for(int i = 0; i < params._source['unavailability'].length; i++) {
Note that depending on your ES version, you might want to try ctx._source or just _source instead of params._source
I solve my use-case in a different approach.
Instead having a field as array of object like unavailability was I decided to create two fields as array of datetime:
unavailable_from
unavailable_to
My script walks through the first field then checks the second with the same position.
UPDATE
The direct access to _source is disabled by default:
https://github.com/elastic/elasticsearch/issues/17558
I need to populate data from one field to multiple fields on the same collection. For example:
Currently I have document like below:
{ _id: 1, temp_data: {temp1: [1,2,3], temp2: "foo bar"} }
I want to populate into two different fields on the same collection as like below:
{ _id: 1, temp1: [1,2,3], temp2: "foo bar" }
I have one billion documents to migrate. Please suggest me the efficient way to update all one billion documents?
In your favorite language, write a tool that runs through all documents, migrates them, and store them in a new database.
Some hints:
When iterating the results, make sure they are sorted (e.g. on the _id) so you can implement resume should your migration code crash at 90%...
Do batch inserts: read, say, 1000 items, migrate them, then write 1000 items in a single batch to the new database. Reads are automatically batched.
Create indexes after the migration, not before. That will be faster and lead to less fragmentation
Here I made a query for you, use following query to migrate your data
db.collection.find().forEach(function(myDoc) {
db.collection_new.update(
{_id: myDoc._id},
{
$unset: {'temp_data': 1},
$set: {
'temp1': myDoc.temp_data.temp1,
'temp2': myDoc.temp_data.temp2
}
},
{ upsert: true }
)
});
To learn more about foreach cursor please visit link
Need $limit and $skip operator to migrate data in batches. In update query i have used upsert beacuse there if already exist it will update otherwise inserted entry wiil be new.
Thanks
The ElasticSearch Docs reads:
An alias can also be mapped to more than one index, and when specifying it, the alias will automatically expand to the aliases indices.
But when I try to add an alias to 2 indices and write to both, neither seem to get updated with the document. If I remove one of the aliases, it will write correctly to the alias that still exists.
Fails with multiple write aliases:
$ curl -XGET 'http://localhost:9200/_aliases'
result:
{
"dev_01": {
"aliases": {
"dev_read": {},
"dev_write": {}
}
},
"dev": {
"aliases": {
"dev_write": {}
}
}
}
Works with single alias:
$ curl -XGET 'http://localhost:9200/_aliases'
result:
{
"dev_01": {
"aliases": {
"dev_read": {},
"dev_write": {}
}
},
"dev": {
"aliases": {}
}
}
Does elasticsearch support writing to multiple indices? Are aliases Read-Only if pointed at multiple indices?
the answer is No
So it appears I should have triaged this a beep deeper, but the response my client gets from es is:
ElasticSearchIllegalArgumentException[Alias [dev_write] has more than one indices associated with it [[dev_01, dev]], can't execute a single index op
Just wish the docs were a little more explicit up front, as they confused me a bit
At first seems to imply you can:
The index aliases API allow to alias an index with a name, with all APIs automatically converting the alias name to the actual index name. An alias can also be mapped to more than one index...
Associating an alias with more than one index are simply several add actions...
Further down the page lets you know you can not:
It is an error to index to an alias which points to more than one index.
I have an index in ElasticSearch with the following mapping:
mappings: {
feed: {
properties: {
html_url: {
index: not_analyzed
omit_norms: true
index_options: docs
type: string
}
title: {
index_options: offsets
type: string
}
created: {
store: true
format: yyyy-MM-dd HH:mm:ss
type: date
}
description: {
type: string
}
}
}
getting the following error when performing phrase search ("video games"):
IllegalStateException[field \"title\" was indexed without position data; cannot run PhraseQuery (term=video)];
Single word searches work fine. Tried "index_options: positions" as well but with no luck. Title field contains text in multiple languages, sometimes empty. Interesting that it seems to fail randomly, for example it would fail with 200K documents or 800K using the same dataset. Is there a reason some titles wouldn't get indexed with positions?
Elastic search version 0.90.5
Just in case someone else has the same issue. There was another type/table (feed2) in the same index with the same "title" field that was set to "not_analyzed".
For some reason even if you specify the type: http://elasticsearchhost.com:9200/index_name/feed/_search the other type is still being searched as well. Changing the mapping for feed2 type fixed the problem.
You probably have another field named 'title' with a different mapping in another type but in the same index.
Basically if you have 2 fields with the same name in the same index - even if they are in different types - they cannot have different mappings: to be more precise, even if they have the same type (eg: "string") but one of them is "analyzed" and the other is "not analyzed", problems will arise.
I mean, yeah, you can try to setup 2 different mappings, and ElasticSearch will not complain, but when searching you get strange result and everything will go bananas.
You can read more about this issue here where they say:
[...] In the end, we opted to enforce the rule that all fields with the same name in the same index must have the same mapping [...]
And yeah, considering how the promise of ElasticSearch has always been "it just works" this little detail took a lot of people by surprise.