Kibana's UI allows the user to create a scripted field which is stored as part of the index (screenshot below). How can that be done programatically? In particular, using either the NEST client or the Elasticsearch low level client.
Kibana UI for the Indice with the Scripted Fields tab highlighted
Note that I am not asking how to create add an expression/script field as part of a query, I'm specifically looking for how to add it as part of the Index when the mapping is created so that queries can reference it without having to explicitly include it.
Kibana dashboards are stored in the .kibana index. To export dashboards, you can query the Kibana index as you would any other index. For example, curl -XGET http://localhost:9200/.kibana/_search?type=dashboard&pretty would show the JSON for your dashboards. You could export the template, add the scripted field to the JSON, and then POST it again. Since Kibana uses a standard Elasticsearch index, the normal Elasticsearch API would apply to modifying Kibana dashboards. This may provide a little more clarification.
At the time of writing, current version 5.2 does not have an official way to do this.
This is how I do it:
Get index fields: GET /.kibana/index-pattern/YOUR_INDEX
Add your scripted field to _source.fields (as string, notice scaped quotation marks)
"fields":"[...,{\"name\":\"test\",\"type\":\"number\",\"count\":0,\"scripted\":true,\"script\":\"doc['area_id'].value\",\"lang\":\"painless\",\"indexed\":false,\"analyzed\":false,\"doc_values\":false,\"searchable\":true,\"aggregatable\":true}]"
Post back _source json to /.kibana/index-pattern/YOUR_INDEX
{
"title":"YOUR_INDEX",
"timeFieldName":"time",
"fields":"[...,{\"name\":\"test\",...}]"
}
Related
In Kibana (ElasticSearch v6.8) I'm storing documents containing a date field and a LaunchTime field, and I have a scripted field uptime as their difference (in seconds):
(doc['date'].value.millis - doc['LaunchTime'].value.millis) / 1000 / 60
I'm trying to create a monitor (under alerting) on the max value of this field of the index, but the field 'Uptime' doesn't show up in the list of fields I can do a max query on. Its type is number and in visualisations I can do max/min etc. displays of this field.
Is this a limitation of Kibana alerting - that I can't use a scripted field? Or is there some way I can make it available to use?
I'm afraid it is a limitation of kibana's scripted fields. See this post about the same subject referring to the scripted field official documentation. I believe that the watcher are handled by ES itself while the scripted field are handled by kibana (they can be used in discovery and visualisations because kibana is handlind those too)
But have no fear! you already have the script for the calculation and you could just add it into logstash to add the field to you actual documents when you index them, which would enable you to use it for watchers AND would probably optimize the load at runtime, since the val is only calculated one, when you ingest it. Then you could run an update by query with a the script and add the field in you existing documents.
If you don't use logstash, you could look into ES's ingestion pipelines, but it's a rather advanced subject and i'm not sure if it was implemented in 5.x.
I have elasticsearch mapping which lists a field but when trying to visualize in kibana it doesnt list that field. Moreover i can see that field under 'popular' section of Discover page.
i see some note like:
"This field is present in your elasticsearch mapping but not in any documents in the search results. You may still be able to visualize or search on it."
What does this mean and how can i visualize a mapped field.
It's mean that you have a mapping but you don't have documents, you don't have data,so you need to index data to elasticsearch index
I am using elastic search to index my data and i was able to do it with the following POST request
http://localhost:9200/index/type/id
{
JSON data over here
}
Yesterday while i was going through some of the elastic tutorials i found one person mentioning about setting analyzer to those fields where we are planning to do full text search.I found during my googling that mapping API can be used to update datatypes and analyzer, but in my case i want to do it as i am creating index.
How can i do it?
You can create index with custom settings (and mappings) in the first request and then index your data with second request. In this case you can not do both at the same time.
However if you index your data first and index does not exist yet, it will be created automatically with default settings. You can then update your mappings.
Source: Index
I want to automate the creation of a set visualizations for new kibana/elasticsearch installations.
So I need to know if I can automate this, independent the programming language.
There are no APIs yet in Kibana to manage the searches, visualizations and dashboards. Some feature requests have been suggested (here and here) but they are still being discussed.
However, since Kibana visualizations are stored in the .kibana index with the visualization mapping type, you can definitely GET them, learn how they are built, modify them and PUT them again.
For a visualization named "Top consumers by country", you can get the visualization spec using
curl -XGET http://localhost:9200/.kibana/visualization/Top-consumers-by-country
You'll get a document containing the title of your visualization, another field called visState containing the specification of your visualization (obvisouly different for each visualization) and finally a field named kibanaSavedObjectMeta which contains the Elasticsearch query and index details.
You can also view/edit/export the same data in Settings > Objects > Visualizations
I have Kibana 4.0.1 running on top of elasticsearch 1.4.4. It was very smooth and virtually had no setup time. Suddenly I have run into a problem.
If I add a new field in my elasticsearch index, it's not visible in fields section. I can still query on that field in discover section. But, I can't make a graph based on the new field as it's not visible in fields list.
Kibana apparently fetches _mapping at the time of setup and stores it in elasticsearch index named .kibana. Once done, it never changes that. Deleting this index should load fresh _mapping from elasticsearch. But I don't want to lose all the saved dashboards and visualizations.
Is there a was to force Kibana to load fresh mapping at regular interval?
Yes in the settings tab you can refresh the index. Check the yellow refresh botton in the image below.