Grafana 4 Templating with Elasticsearch 5 - elasticsearch

Question

Edit: See below for the solution
Currently having an issue with the templating in Grafana - trying to get a dropdown of hostnames from some data I'm feeding in to Elasticsearch via Logstash's Graphite plugin, so I can build a dynamic template in Grafana.
Versions are
Grafana 4.1.2 + Elasticsearch/Logstash 5.2.1
The terms query in Grafana I'm trying to use is as follows as per docs on grafana website - http://docs.grafana.org/features/datasources/elasticsearch/ :
{"find": "terms", "field": "host_name"}
This works fine if the field is a numeric type field - eg I get results in the template for metric_value, but this doesn't seem to work for text/string fields. I'm wondering if this is maybe due to the way I'm constructing or ingesting the fields - You can see below how I"m trying to achieve this - note, I've tried "keyword" and "text" types for these fields, neither seem to work.
This is the Logstash input filter that I'm using - basically trying to split the graphite style metric into seperate fields -
input {
graphite {
type => graphite
port => 2003
id => "graphite_input"
}
}
filter {
if [type] == "graphite" {
grok {
match => [ "message", "\Aicinga2\.%{MONGO_WORDDASH:host_name:keyword}\.%{WORD:metric_type:keyword}\.%{NOTSPACE:metric_name:keyword}\.value%{SPACE}%{NUMBER:metric_value:float}%{SPACE}%{POSINT:timestamp:date}" ]
}
}
}
output {
if [type] == "graphite" {
elasticsearch {
index => "graphite-%{+YYYY.MM}"
hosts => ["localhost"]
}
}
}
And an example document I'm indexing (taken from kibana)
{
"_index": "graphite-2017.02",
"_type": "graphite",
"_id": "XYZdflksdf",
"_score": null,
"_source": {
"#timestamp": "2017-02-21T00:17:16.000Z",
"metric_name": "interface-eth0.snmp-interface.perfdata.eth0_in_discard",
"port": 37694,
"icinga2.XXXYYY.services.interface-eth0.snmp-interface.perfdata.eth0_in_discard.value": 357237,
"#version": "1",
"host": "192.168.1.1",
"metric_type": "services",
"metric_value": 357237,
"message": "icinga2.XXXYYY.services.interface-eth0.snmp-interface.perfdata.eth0_in_discard.value 357237 1487636236",
"type": "graphite",
"host_name": "XXXYYY",
"timestamp": "1487636236"
},
"fields": {
"#timestamp": [
1487636236000
]
},
"sort": [
1487636236000
]
}

Answer 1

I have now solved this problem myself. The string fields are required to be defined as not_analyzed in order to appear in the Grafana dashboard.
Here's an example Template you can use:
Note: you'll have to install this manually, it seems like logstash won't install it into elasticsearch for some reason (maybe a bug?)
Install like so (assuming path is /etc/logstash/graphite-new.json:
curl -XPUT 'http://localhost:9200/_template/graphite-*' -d#/etc/logstash/graphite-new.json
Template:
{
"template" : "graphite-*",
"settings" : { "index.refresh_interval" : "60s" },
"mappings" : {
"_default_" : {
"_all" : { "enabled" : false },
"dynamic_templates" : [{
"message_field" : {
"match" : "message",
"match_mapping_type" : "string",
"mapping" : { "type" : "string", "index" : "not_analyzed" }
}
}, {
"string_fields" : {
"match" : "*",
"match_mapping_type" : "string",
"mapping" : { "type" : "string", "index" : "not_analyzed" }
}
}],
"properties" : {
"#timestamp" : { "type" : "date", "format" : "dateOptionalTime" },
"#version" : { "type" : "integer", "index" : "not_analyzed" },
"metric_name" : { "type" : "string", "index" : "not_analyzed" },
"host" : { "type" : "string", "index" : "not_analyzed" },
"host_name" : { "type" : "string", "index" : "not_analyzed" },
"metric_type" : { "type" : "string", "index" : "not_analyzed" }
}
}
}
}
I've still got this defined in the logstash filter as well:
if [type] == "graphite" {
elasticsearch {
index => "graphite-%{+YYYY.MM}"
hosts => ["localhost"]
template => "/etc/logstash/graphite-new.json"
}
}