ELK configuration for my application logs forward to elastic search using log stash - elasticsearch

Question

I am new in ELK configuration.
https://www.digitalocean.com/community/tutorials/how-to-install-elasticsearch-logstash-and-kibana-elastic-stack-on-ubuntu-16-04
I have configure in my local machine and it is work fine.
I want to forward my application logs file to elastic search using log-stash of file beats.
When I have configure all things working fine for system logs.
but I am not able to store my application log to elastic search.
Please help me.
This is my log file:
service.log
{"name":"service name", "hostname":"abc", "pid":4474, "userId":"123", "school_id":"123", "role":"student", "username":"mahi123", "serviceName":"loginService", "level":40, "msg":"successFully fetch trail log", "time":"2019-06-01T10:55:46.482Z","v":0}

Answer 1

Some troubleshooting steps to take care of when logs do not reach Elastisearch:
Check your log parsing configuration file(usually made with the extension .conf). Make sure it's having the right path to scan logs from, right set of filters etc. To see if this .conf file is actually working, one can try:
logstash -f <elasticsearch.conf file path> If this doesn't throw any error on console, that means you are good at this point and will have to move to next step.
Check if Kibana indices are getting created. Run
curl http://<hostipaddress or localhost>:9200/_cat/indices?v.
If yes, go to Kibana Management and create index patterns.
If not, check if your system has enough available memory to serve logstash and elastisearch. free -m would be helpful once you start logstash and elasticsearch services.
Many a times, I have seen people trying ELK setup on a machine which has insufficient RAM(4GB sounds good for a standalone setup).
Check your logstash and Elasticsearch services are up and running. If Elasticsearch is getting down or getting restarted during log parsing or indices creation, that's most probably due to lack of system resources.
-bash-4.2# systemctl status elasticsearch
�� elasticsearch.service - Elasticsearch
Loaded: loaded (/usr/lib/systemd/system/elasticsearch.service; enabled; vendor preset: disabled)
Active: active (running) since Wed 2019-06-05 14:08:26 UTC; 1 weeks 0 days ago
Docs: http://www.elastic.co
Main PID: 1396 (java)
CGroup: /system.slice/elasticsearch.service
������1396 /bin/java -Xms1g -Xmx1g -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMS...
Jun 05 14:08:26 cue-bldsvr4 systemd[1]: Started Elasticsearch.
Jun 05 14:08:26 cue-bldsvr4 systemd[1]: Starting Elasticsearch...
-bash-4.2# systemctl status logstash
�� logstash.service - logstash
Loaded: loaded (/etc/systemd/system/logstash.service; enabled; vendor preset: disabled)
Active: active (running) since Wed 2019-06-05 14:50:52 UTC; 1 weeks 0 days ago
Main PID: 4320 (java)
CGroup: /system.slice/logstash.service
������4320 /bin/java -Xms256m -Xmx1g -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFrac...
Jun 05 14:50:52 cue-bldsvr4 systemd[1]: Started logstash.
Jun 05 14:50:52 cue-bldsvr4 systemd[1]: Starting logstash...
Jun 05 14:51:08 cue-bldsvr4 logstash[4320]: Sending Logstash's logs to /var/log/logstash which is now configur...rties
Hint: Some lines were ellipsized, use -l to show in full.
-bash-4.2#