In ElasticSearch, is it possible to make a partial update to an index template?
For example, I tried changing only the default analyzer with the following (using Sense):
PUT /_template/testtemplate/_update
{
"settings": {
"index":{
"analysis": {
"analyzer": {
"default": {
"type": "whitespace"
}
}
}
}
}
}
Bu that didn't work, I get an error that says Invalid index name [_template], must not start with '_'.
Do I have to pass the full template again using a PUT or there's some other way to do a partial update?
Partial updates only work for documents (and to some extent to mapping types), not for index templates. So, I confirm you'll need to store the full index template again and not only the small bit of it that you want to update.
As you can see in the source code for RestUpdateAction.java, the _update REST endpoint expects an {index} name, a {type} name and an {id}. So in your example above, that endpoint thinks that _template is an index name and complains.
Similarly, in the REST endpoint for creating index templates, RestPutIndexTemplateAction.java, you can see that the _template path doesn't support the _update endpoint at all.
Related
I am trying to set all mapped fields to string ie if a json message comes with following:
{
"logDate": "2012-04-23T18:25:43.511Z",
"logId": 123131,
"message": {
"username": "pera",
"password": "pera123"
}
}
I need to log every value as string ie. logId should be logged as "logId": "123131".
Is there a way to tell fluent bit what index mapping to use of maybe there is another setting that changes dynamic type to string?
Maybe can try adding an index template.
https://www.elastic.co/guide/en/elasticsearch/reference/current/index-templates.html
I am using https://github.com/babenkoivan/scout-elasticsearch-driver to implement Elasticsearch with Laravel Scout. Ivan mentions this on Github:
Indices created in Elasticsearch 6.0.0 or later may only contain a single mapping type. Indices created in 5.x with multiple mapping types will continue to function as before in Elasticsearch 6.x. Mapping types will be completely removed in Elasticsearch 7.0.0.
If I understood right here: https://www.elastic.co/guide/en/elasticsearch/reference/master/removal-of-types.html I either need to use:
PUT index?include_type_name=true
or, better:
2)
PUT index/_doc/1
{
"foo": "baz"
}
I am stuck since I have no idea how to use either 1) or 2)
How can I add the parameter include_type_name=true?
How can I create the right mapping without using the include_type_name parameter?
class TestIndexConfigurator extends IndexConfigurator
{
use Migratable;
/**
* #var array
*/
protected $settings = [
];
protected $name = 'test';
}
Earlier versions of Elasticsearch (<= 5) supported multiple types per index. That meant that you could have different data mappings for each type. With Elasticsearch 6, this was removed and you can only have single mapping type.
Therefore, for Elasticsearch 7 (latest release), you can add an index, setup mappings and add document like this:
Create an index
PUT user
Add mapping
PUT user/_mapping
{
"properties": {
"name": {
"type": "keyword"
},
"loginCount": {
"type": "long"
}
}
}
Add document(s)
PUT user/_doc/1
{
"name": "John",
"loginCount": 4
}
Check data in the index
GET user/_search
Now, regarding the scout-elasticsearch-driver that you use, after reading the documentation you mentioned, it is simply saying that you need to create separate index configurator for each searchable model, as multiple models cannot be stored inside the same index.
So to create the index, run
php artisan make:index-configurator MyIndexConfigurator
and then
php artisan elastic:create-index App\\MyIndexConfigurator
which will create the index in the elasticsearch for you.
To learn more about elasticsearch, I suggest you install both elasticsearch and kibana to your development machine and then play around with it in kibana - the interface is quite nice and supports autocomplete to ease the learning curve.
When I tried GET product/default/_mapping in Kibana console.
I kept getting this error.
"Types cannot be provided in get mapping requests, unless
include_type_name is set to true"
This is happening in elastic search 7.3.0.
Looks like the above command is no longer supported in latest versions of elastic search.
It worked for me when I remove the default from the above command.
GET product/_mapping
I getting same error like "Types cannot be provided in put mapping requests, unless the include_type_name parameter is set to true"
you have to add "include_type_name:true" inside the object
fix this problem above code
let type = true
return await esClient.indices.putMapping({
index:indexName,
type:mappingType,
body:mapping,
include_type_name:type
});
PUT busloggw4/_doc/_mapping?include_type_name=true
{
"properties": {
"log_flag": {
"type":"long"
}
}
}
I have 2 loggers from 2 different clusters logging into my elasticsearch. logger1 uses indices mydata-cluster1-YYYY.MM.DD and logger2 uses indices mydata-cluster2-YYYY.MM.DD.
I have no way of touching the loggers. So i would like to add a field on the ES side when the data is indexed to show which cluster the data belongs to. Can i use mappings to do this?
Thanks
What if you use the PUT mapping API, in order to add a field to your index:
PUT mydata-cluster1-YYYY.MM.DD/_mapping/mappingtype <-- change the mapping type according to yours
{
"properties": {
"your_field": {
"type": "text" <--- type of the field
}
}
}
This SO could come in handy. Hope it helps!
I use multi-fields in a lot of my mappings. In the doc of Elastic Search there is an indication that multi-fields should be replaced with the "fields" parameter. See http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/_multi_fields.html#_multi_fields
This works fine. However, to access a multi-field as a single field the documentation recommends to specify the copy_to parameter instead of the path parameter (see http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/mapping-core-types.html#_accessing_fields)
Can somebody provide an example of such a mapping definition (thus using the "fields" parameter combined with "copy_to").
I have the impression that if you use the fields parameter you still need to specify the path parameter. And if you use copy_to, you no longer need to use a multi-fields approach; the fields just become separate fields and data of one field is copied to another at index time.
Hope somebody can help.
thx
Marc
I think that the copy_to option can be viewed as a cleaner variant of the Multi-fields feature (that is, the fields option). Both of these are easy to use when you want to "copy" values of a field to one or more other fields (to apply different mapping rules). However, if you need to "copy" values from multiple fields to the same field (that is, when you want a custom _all field), you must add the path option to the mapping, if you're using Multi-fields. On the other hand, with the copy_to option, you can simply point multiple source fields to the same destination field.
See this: https://www.elastic.co/guide/en/elasticsearch/reference/1.6/_multi_fields.html
copy_to would allow you to merge different fields like first_name and last_name into full_name
while multi field is used when you want to define several ways to index your field. For example
// Document mapping
{
"properties": {
"name": {
"fields": {
"name_metaphone": {
"type": "string",
"analyzer": "mf_analyzer"
},
"name_exact": {
"index": "not_analyzed",
"type": "string"
}
},
"type": "multi_field"
}
}
}
The thing is that I already defined a field "myvalue" as INTEGER. Now I think was a mistake and I want to store in the same field an string, so I want to change it, without loosing data, to STRING. is there any way of making it?, or I need to re-create the index and re-index the whole data?
I already tried running:
{
"mappings": {
"myvalue": {
"type":"string"
}
}
}
But if I get the mapping again from the server still appear as Integer
There is not any way to change the mapping on a core field type for existing data. You will need to re-create the index with the myvalue field defined as a string and re-index your data.