Elasticsearch - Enable fulltext search of field - elasticsearch

Question

I have run into a brick wall considering searching in my logged events. I am using an elasticsearch solution, filebeat to load messages from logs to elasticsearch, and Kibana front end.
I currently log the messages into a field message and exception stacktrace (if present) into error.message. So the logged event's snippet may look like:
{
"message": "Thrown exception: CustomException (Exception for testing purposes)"
"error" : {
"message" : "com.press.controller.CustomException: Exception for testing purposes\n at
com.press.controller....<you get the idea at this point>"
}
}
Of course there are other fields like timestamp, but those are not important. What is important is this:
When I search message : customException, I can find the events I logged. When I search error.message : customException, I do not get the events. I need to be able to fulltext search all fields.
Is there a way how to tell elasticsearch to enable the fulltext search in the fields?
And why has the "message" field enabled it by default? None of my colleagues are aware that any indexing command was run on the field in the console after deployment and our privileges do not allow me or other team members to run indexing or analysis commands on any field. So it has to be in the config somewhere.
So far I was unable to find the solution. Please push me in the right direction.
Edit:
The config of fields is as follows:
We use a modified ECS, and both messages are declared as
level: core
type: text
in file fields.yml.
in filebeat, the config snippet is as such:
filebeat.inputs:
- type: log
enabled: true
paths: .....
...
...
processors:
- rename:
fields:
- from: "msg"
to: "message"
- from: "filepath"
to: "log.file.name"
- from: "ex"
to: "error.message"
ignore_missing: true
fail_on_error: true
logging.level: debug
logging.to_files: true
For security requirements, I cannot disclose full files. Also, I need to write all the snippets by hand, so misspells are probably my fault.
Thanks

Answer 1

Problem is with the analyzer associated with your field, by default for text fields in ES, standard analyzer is used which doesn't create separate tokens if text contains . for ex: foo.bar would result in just 1 token as foo.bar while if you want both foo and bar should match in foo.bar then you need to genrate 2 tokens as foo and bar.
What you need is a custom analyzer which creates token as above as your error.message text contains . which I explained in my example:
PUT /my_index
{
"settings": {
"analysis": {
"analyzer": {
"my_analyzer": {
"tokenizer": "standard",
"char_filter": ["replace_dots"]
}
},
"char_filter": {
"replace_dots": {
"type": "mapping",
"mappings": [
". => \\u0020"
]
}
}
}
}
}
POST /my_index/_analyze
{
"analyzer": "my_analyzer",
"text": "foo.bar"
}
The above example creates 2 tokens as foo and bar and same should happen with you when you create and test it with these API.
Let me know if you face any issue with it.

Answer 2

Elastic Search indexes all fields by default, here you did not define the mapping hence all fields should be indexed by default.
Also for your case I doubt if the data is properly going in elastic search as the log doesn't seem to be proper json.
Do you see proper logs in Kibana if yes please send a sample log/screenshot