I'm using kibana-4. Following the documentation here I should be able to create an index by putting this in my elasticsearch.yaml file:
PUT .kibana
{
"index.mapper.dynamic": true
}
I'm not sure I understand how to do this, because a yaml file should not take values formatted like the above block, right?
I noticed that .kibana was a default index, so after inputting it into the kibana console, I was asked to input a time field for the default index. However, the input HTML element is a dropdown that contained no options. Without selecting a time-field option I am not allowed to create a default index. What am I supposed to do? Has anyone else run into a similar problem?
I understand the problem faced by you. Even i faced the same while using Kibana 4 for first time.
Here are 2 possible solutions to your problem:-
1. Input data into elasticsearch which contains a timestamped field. So upon inputting data that field will be directly recognized by Kibana & would be showed to you in the dropdown menu (where you are currently seeing empty).
It is empty because Kibana couldn't recognize the timestamped field from the data inserted by you in elasticsearch.
2. Untick the option of Index contains time-based events which will allow you to just enter your index name & access Kibana.
Note:- while using Option 2 & specifying index name as .kibana you would notice that it doesn't contain any field or data because .kibana doesnt store any data.
I would suggest you to create an index using curl command and insert data in it with or without timestamped field. If inserted data without timestamped field use Option 2 otherwise use Option 1.
Related
I am using Elasticsearch and Kibana as plugin to view the data in the indices. I am using Kibana's DevTools to send commands for adding/deleting/updating indices etc.
I want to add a field to a certain text property so it will have a keyword field to be able to both make a full text searches and aggregate using this property.
1) Does a change like that means I need to update Kibana's index pattern as well?
2) I have read the ElasticSearch's docs on PUT Mappings and know how to use it to update the indices themselves, but I don't know how to update the index patterns.. I read the same API should be used to update it, but I don't know how to see the index pattern's original mapping in order to update it.
Yes, if you change the index mapping in ES, then you need to go in Kibana and refresh the related index patterns.
Right now, you need to go inside Kibana (Management > Index patterns), select the index pattern, and press the "Refresh" button at the top right of the window in order to pick up the mapping changes.
Also note that if you updated some text fields in order to have a keyword sub-field, you'll also need to call the _update_by_query API on your index in order to reindex the changed field in all your documents
My index settings in Kibana tell me that I have fields with mapping conflicts in my logstash-* index patterns.
What is the easiest way to find out which fields have a conflicting mapping and/or in which indices the conflict occurs?
As of at least Kibana 5.2, you can type "conflict" into the Filter field, which will filter all fields down to only those which have a conflict. At the far right there is a column named "controls", and for each field it has a button with a pencil icon. Clicking that will tell you which indices have which mapping.
Fields filtered to only those with conflicts:
Indices in which field mapping conflicts:
You can easily find how fields are mapped using the mapping API in Kibana.
If you know you have a mapping conflict, I will assume you know the field name that has the conflict. These will be listed under Management/Index Patterns/index_pattern
If you have indices that are created daily, such as production-2020.06.16, you can search across all the indices with production*.
Go to Dev Tools and enter this query, changing the index pattern (production*) and conflictedFieldname to suit your needs.
GET production*/_mapping/field/conflictedFieldname
This will pull all indices that match the production* pattern and will list the mapping for conflictedFieldname for each index. Scroll through and see which one is not like the other one.
You can also check out the Elasticsearch documentation here: Elasticsearch documentation: Get Field Mapping API
The reason you're getting a conflict is because the first value that goes into the index is used by Elasticsearch to make its best guess as to what data type it should be. You can ensure it is always the same type by placing a template for the index pattern you are concerned with.
Elasticsearch documentation: Put Index Template
In Elasticsearch 5.5.2, you can click on the dropdown on the right of the Filter search box and select "conflict". This is in the Index Patterns page.
It should be easy to spot those in the list of fields, when defining the pattern. Something like this:
Since I couldn't locate the mapping conflict in the gui. I went down the hard path analysed my config for missing/conflicting field type found the offender and reindexed my data.
If you click the type column on the index patterns page where the warning is displayed, it should sort the indexes by type. Conflicted fields will have type 'conflict'.
I have an problem with the unique count feature.
I get data from elasticsearch for example an computer name (PC-01) in a field.
When i want to use a visualisation unique count then kibana makes from "DESKTOP-2D562R2" -> "DESKTOP" and "2D562R2" as a entery.
See this splitted field:
The data kibana gets from elastic search looks like this entery data:
The problem with this is that 2d562r2 and desktop two different "enterys" are in a kibana table or with unique count.
Your field is being analyzed (split into tokens). Change the mapping (or template, depending on how you're creating the indexes) to make this field not_analyzed.
Note that, as a hack, logstash's default template creates a ".raw" version of string fields that is not analyzed. You could refer to enterys.raw.
I have accidentally created some wrong scripted field in kibana-4 and now my elasticsearch is not working and throwing exceptions. I am looking for a command to list all the scripted fields currently used by the elasticsearch and delete the one that is causing trouble to me.
You can easily/add delete a scripted field in Kibana 4.
Just go to Settings, select Indices tab, click on your index name , selected scripted fields tab beside the fields tab & add/delete accordingly :)
I have just updated a website, the update adds new fields to elasticsearch.
In my dev environment, it all works fine. but on the live site, the new fields are not being found.
Eg. I have added a new field with the value : 1
However, when adding a filtered query of
{"field":1}
It does not find any matching results.
When I look in the documents, I can see docs with the field set to 1
Would the reason for this be that the new field was added after the mappings was set? I am not all that familiar with elasticsearch, So I am not really sure where to start looking to fix it.
Any help would be appreciated.
Update:
querying from URL shows nothing either
_search/?pretty=true&size=50&q=field1:*
however there is another field that was added at the same time which I can search on.
I can see field1 in the result set but it just wont allow me to search on it.
Only difference i see in the mapping is that the one that is working is set to type:long whereas the one not working is set as type:string
Is it a length issue on the ngram? what was your "min_gram" settings?
When you check on your index settings like this:
GET <host>/<index_name>/_settings
Does it work when you filter for a two digit field?
Are all the field values one digit?
It's OK to add a field after the mapping was set. ElasticSearch will guess the mapping for you. (in fact, it's one of their selling features --- no need to define the mapping, just throw the data at it)
There are a few things that can go wrong:
Verify that data is actually in the index. To do that, just navigate to the _search url with no parameters, you should see the field if it is indexed.
Look at your mapping. Could it be that the field is explicitly set not to be indexed?
Another possibility is that your query is wrong (but that is unlikely, since you're saying it works in the development environment)