elasticsearch search request size limit error - elasticsearch

Question

me elasticsearch version 7.9.3 (running on ubuntu) holds an index of each day (logs)
so when a query needs to include for example data from 2020-01-01 until 2020-11-20
Search query will look like this: (which returns error 400)
http://localhost:9200/log_2020-02-14,log_2020-02-26,log_2020-02-27,log_2020-04-24,log_2020-04-25,log_2020-07-17,log_2020-08-01,log_2020-09-09,log_2020-09-21,log_2020-10-06,log_2020-10-07,log_2020-10-08,log_2020-10-16,log_2020-10-17,log_2020-10-18,log_2020-10-21,log_2020-10-22,log_2020-11-12/_search?pretty
I know I can split the request into two but I don't see why (4096 bytes over HTTP it's not so big)
any chance to config this issue ?
response:
{
"error": {
"root_cause": [
{
"type": "too_long_frame_exception",
"reason": "An HTTP line is larger than 4096 bytes."
}
],
"type": "too_long_frame_exception",
"reason": "An HTTP line is larger than 4096 bytes."
},
"status": 400
}

Answer 1

URLs cannot exceed a certain size depending on the medium. Elasticsearch limits that length to 4096 bytes.
Since you seem to be willing to query all indexes of 2020 since January 1st until today (Nov 20), you can use a wildcard like this:
http://localhost:9200/log_2020*/_search?pretty
Another way is by leveraging aliases and put all your 2020 indexes behind the log_2020 alias:
POST /_aliases
{
"actions" : [
{ "add" : { "index" : "log_2020*", "alias" : "log_2020" } }
]
}
After running that you can query the alias directly
http://localhost:9200/log_2020/_search?pretty
If you want to make sure that all your daily indexes get the alias upon creation you can add an index template
PUT _index_template/my-logs
{
"index_patterns" : ["log_2020*"],
"template": {
"aliases" : {
"log_2020" : {}
}
}
}
UPDATE
If you need to query between 2020-03-04 and 2020-09-21, you can query the log_2020 alias with a range query on your date field
POST log_2020/_search
{
"query": {
"range": {
"#timestamp": {
"gte": "2020-03-04",
"lt": "2020-09-22"
}
}
}
}