Indexing the last word of a string in Elasticsearch - elasticsearch

Question

I'm looking for a way to index the last word (or more generally: the last token) of a field into a separate sub-field. I've looked into the Predicate Script token filter but the painless script API in that context only provides the absolute position of the toekn from the start of the original input string so I could find the first token like this:
GET /_analyze
{
"tokenizer": "whitespace",
"filter": [
{
"type": "predicate_token_filter",
"script": {
"source": """
token.position == 0
"""
}
}
],
"text": "the fox jumps the lazy dog"
}
This works and results in:
{
"tokens" : [
{
"token" : "the",
"start_offset" : 0,
"end_offset" : 3,
"type" : "<ALPHANUM>",
"position" : 0
}
]
}
But I need the last token, not the first. Is there any way to achieve this without preparing a separate field pre-indexing, outside of Elasticsearch?

Answer 1

You're on the right path!! The solution is not that far from what you have... When you know you can easily fetch the first token, but what you need is the last... just reverse the string...
The following analyzer will output just the token you need, i.e. dog.
We first start by reversing the whole string, then we split by token, use your predicate script to only select the first one and reverse that token again. Voilà!
POST test/_analyze
{
"text": "the fox jumps the lazy dog",
"tokenizer": "keyword",
"filter": [
"reverse",
"word_delimiter",
{
"type": "predicate_token_filter",
"script": {
"source": """
token.position == 0
"""
}
},
"reverse"
]
}
Result:
{
"tokens" : [
{
"token" : "dog",
"start_offset" : 0,
"end_offset" : 3,
"type" : "word",
"position" : 0
}
]
}