elasticsearch multiple aggregations not working - elasticsearch

Question

fighting with elasticsearch aggregations - might need some advice ...
elasticsearch version:
Version: 1.4.1, Build: 89d3241/2014-11-26T15:49:29Z, JVM: 1.7.0_72
sample dataset:
{
"_index": "logstash-2014.12.17",
"_type": "netflow",
"_id": "AUpaDdUVUcM5Us_C6x7Z",
"_score": 1,
"_source": {
"message": "<27>Dec 17 22:01:02 es01 nfcapd[29441]: expip=10.245.132.16 fweventtime=2014-12-17 22:01:02.793 fwevent=DENIED srcip=78.110.142.76 dstip=179.24.227.252 srcport=62327 dstport=41863 proto=UDP input=3 output=4 inbytes=0 outbytes=0 postnatsrcip=78.110.142.76 postnatdstip=179.24.227.252 postnatsrcport=62327 postnatdstport=41863 ingressacl=0x45b0635e/0x9872d678/0x724bf9a4 egressacl=0x0/0x0/0x0",
"#version": "1",
"#timestamp": "2014-12-17T21:01:02.794Z",
"type": "netflow",
"host": "127.0.0.1",
"timestamp": "Dec 17 22:01:02",
"hostname": "es01",
"expip": "10.245.132.16",
"time": "2014-12-17 22:01:02.793",
"fwevent": "DENIED",
"srcip": "78.110.142.76",
"dstip": "179.24.227.252",
"srcport": "62327",
"dstport": "41863",
"proto": "UDP",
"output": "4",
"inbytes": "0",
"outbytes": "0",
"postnatsrcip": "78.110.142.76",
"postnatdstip": "179.24.227.252",
"postnatsrcport": "62327",
"postnatdstport": "41863",
"ingressacl1": "0x45b0635e",
"ingressacl2": "0x9872d678",
"ingressacl3": "0x724bf9a4",
"egressacl1": "0x0",
"egressacl2": "0x0",
"egressacl3": "0x0",
"srcgeo": {
"country_code3": "CHE",
"latitude": 47,
"longitude": 8,
"location": [
8,
47
]
}
}
}
sample query:
GET _search
{
"size": 1,
"query": {
"filtered": {
"filter": {
"range": {
"#timestamp": {
"gt": "2014-12-17T21:00:00"
}
}
}
}
},
"aggs": {
"proto": {
"terms": {
"field": "proto"
},
"aggs": {
"traffic_sum": {
"sum": {
"field": "outbytes"
}
}
}
}
}
}
results in error:
{
"error": "SearchPhaseExecutionException[Failed to execute phase [query], all shards failed;
shardFailures {[jJZG3gX7QlujjG4ZXttyRA][logstash-2014.12.17][0]:
ClassCastException[org.elasticsearch.index.fielddata.plain.PagedBytesIndexFieldData cannot be cast to org.elasticsearch.index.fielddata.IndexNumericFieldData]}{[8Rz-FI7JSvebgBdGG9zOkA][logstash-2014.12.17][1]:
RemoteTransportException[[bigdata02][inet[/<snip>:9301]][indices:data/read/search[phase/query]]];
nested: ClassCastException[org.elasticsearch.index.fielddata.plain.PagedBytesIndexFieldData cannot be cast to org.elasticsearch.index.fielddata.IndexNumericFieldData]; }{[8Rz-FI7JSvebgBdGG9zOkA][logstash-2014.12.17][2]:
RemoteTransportException[[bigdata02][inet[/<snip>:9301]][indices:data/read/search[phase/query]]];
nested: ClassCastException[org.elasticsearch.index.fielddata.plain.PagedBytesIndexFieldData cannot be cast to org.elasticsearch.index.fielddata.IndexNumericFieldData]; }{[jJZG3gX7QlujjG4ZXttyRA][logstash-2014.12.17][3]:
ClassCastException[org.elasticsearch.index.fielddata.plain.PagedBytesIndexFieldData cannot be cast to org.elasticsearch.index.fielddata.IndexNumericFieldData]}{[jJZG3gX7QlujjG4ZXttyRA][logstash-2014.12.17][4]:
ClassCastException[org.elasticsearch.index.fielddata.plain.PagedBytesIndexFieldData cannot be cast to org.elasticsearch.index.fielddata.IndexNumericFieldData]}]",
"status": 500
}
* works fine with just one aggregation - fails if I insert second aggregation *
any idea?

Answer 1

this is the important part:
ClassCastException[org.elasticsearch.index.fielddata.plain.PagedBytesIndexFieldData
cannot be cast to org.elasticsearch.index.fielddata.IndexNumericFieldData]}]"
You're attempting to do a sum using a string field.
this field is the problem:
"outbytes": "0",
Either
delete your existing data and create a numeric field type by posting a document containing "outbytes": 0 (note the lack of quotation marks).
Delete your existing data and create an explicit mapping with field outbytes set to a number.
Keep your data but update the aggregation to call a script that does the string to number conversion.
My recommendation would be to go for option 2.