Creating custom fields based on analysis output in ealsticsearch - elasticsearch

Question

I am having document where value is raw string :
{ "content" : "field1=1 , field2=foo"}
My intention is to, query by field1, field2 values.
Closest thing I can think of is to use custom analyser which will create tokens based on comma separator, and then I can search with matching exact values like "field1=1" or "field2=foo" . However, ideally I like to search by range values for field1, pattern matching for field2 etc.
Is there any way to achieve this? I could not find any way to store result of analysis which I can query in this way.

Answer 1

How are you ingesting the documents? If you are doing via logstash ,then you can apply the transformation there using a filter processor

Answer 2

I'm having a little difficulty understanding your question. However, I think you are asking if there's a way to make the type of Field1 numeric and the type of Field2 searchable?
Hopefully you are running Kibana so you can use the Dev console to test this out. If you just let Elastic import data it will create aggregateable and searchable fields for both field1 and field2 because they are both set to string values:
PUT /content_default/type/1 {"field1":"1" , "field2":"foo"}
If you instead omit the quotes around the 1, Elastic will create the field as a long (assuming you haven't already imported a document with a string in the same field) - this allows you to search by range. Here I'm creating a new field3 and setting the value to 1, if you query you should see it's a long
PUT /content_default/type/2 {"field1":"1" , "field2":"foo", "field3":1}
You can pre-load a template to allow you to define types up-front before loading any data - that way Elastic doesn't have to guess what types your fields should be. With strings you can also define whether you want them to be just keywords, searchable or both.
Something like this should do the trick for you:
PUT _template\with_template
{
   "template":"content_with_template",
   "mappings":{
      "content_with_template":{
         "properties":{
            "field2":{
               "analyzer":"simple",
               "type":"text"
            },
            "field1":{
               "type":"keyword"
            },
            "field3":{
               "type":"long"
            }
         }
      }
   }
}
Then put a document in the new 'content_with_template' index like this, at this point it doesn't matter if field3 is in quotes or not - as long as it's parses to a number it'll save
PUT /content_with_template/type/1
{ "field1":"a1d" , "field2":"foo", "field3":1}
https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping.html
https://www.elastic.co/guide/en/elasticsearch/reference/current/indices-templates.html