What makes Kibana to not show docker container logs in APM "Transactions" page under "Logs" tab.
I verified the logs are successfully being generated with the "trace.id" associated for proper linking.
I have the exact same environment and configs (7.16.2) up via docker-compose and it works perfectly.
Could not figure out why this feature works locally but does not show in Elastic Cloud deploy.
UPDATE with Solution:
I just solved the problem.
It's related to the Filebeat version.
From 7.16.0 and ON, the transaction/logs linking stops working.
Reverted Filebeat back to version 7.15.2 and it started working again.
If you are not using file beats, for example - We rolled our own logging implementation to send logs from a queue in batches using the Bulk API.
We have our own "ElasticLog" class and then use Attributes to match the logs-* Schema for the Log Stream.
In particular we had to make sure that trace.id was the same as the the actual Traces, trace.id property. Then the logs started to show up here (It does take a few minutes sometimes)
Some more info on how to get the ID's
We use OpenTelemetry exporter for Traces and ILoggerProvider for Logs. The fire off batches independently of each other.
We populate the Trace Id's at the time of instantiation of the class as a default value. This way you in the context of the Activity. Also helps set the timestamp exactly when the log was created.
This LogEntry then gets passed into the ElasticLogger processor and mapped as displayed above to the ElasticLog entry with the Attributes needed for ES
Related
I upload some logs into elastic via filebeat, but there is some other information added to my original logs like the host name ,os kernel and other information about host..., and the main message become unformatted, i want to delete all the field that are unnecessary and only keep my original message in the initial form.
I have tried to delete add_host_metadata from filebeat.yml but the problem still persist.
I'm working with elk on windows.
You could use the include_fields processor enter link description here or what you could do is use the drop_fields for the fields you don’t need. Filebeat will sometimes add in fields such as host, or log, which can be dropped. There are some
That can’t be dropped though.
I am using prometheus as datasource for the grafana dashboard. I am adding the Mesh IP as the URL of the default datasource. Whenever the grafana runs, it creates grafana.db which contains all the information related to datasource. I need to work in such a way that user can change the default URL of the datasource. Till now, everything works very well.
Now my problem is, when I try to change the IP of default datasource, and when I run the container again, it again picks the default URL instead of last saved URL in the grafana.db file. I want it to work in such a way that it should read default datasource IP from grafana.db if the file is available otherwise read it from default Mesh IP.
I can think of two different approaches for this:
Calling some queries using Postgres.
Get notified from GUI whenever URL is changed by the user and update that URL in the variable.
I am completely lost how to solve this problem. Anyone please help me how I can solve this problem using above mentioned approaches or any other one.
Thanks in advance.
The grafana.db resorts to the old default URL because the data is not being persisted across restarts.
For data persistence, you need to map Grafana to an external DB. Install another db outside docker and use the following link to map it to Grafana: database_configuration
Also look at provisioning
I jave a JAVA application in which I am using GCP to create VM instances from images.
In this application, I would like to allow the user to view the vm creation logs in order to be updated on the status of the creation, and to be able to see failure points in detail.
I am sure such logs exist in GCP, but have been unable to find specific APIOs which let me see a specific action, for example creation of instance "X".
Thanks for the help
When you create a VM, the answer that you have is a JobID (because the creation take time and the Compute Engine API answer immediately). To know the status of the VM start (and creation) you have to poll regularly this JobID.
In the logs, you can also filter with this JobID to select and view only the logs that you want on the Compute API side (create/start errors).
If you want to see the logs of the VM, filter the logs not with the JobID but with the name of the VM, and its zone.
In Java, you have client libraries that help you to achieve this
I have managed to process log files using the ELK kit and I can now see my logs on Kibana.
I have scoured the internet and can't seem to find a way to remove all the old logs, viewable in Kibana, from months ago. (Well an explaination that I understand). I just want to clear my Kibana and start a fresh by loading new logs and them being the only ones displayed. Does anyone know how I would do that?
Note: Even if I remove all the Index Patterns (in Management section), the processed logs are still there.
Context: I have been looking at using ELK to analyse testing logs in my work. For that reason, I am using ElasticSearch, Kibana and Logstatsh v5.4, and I am unable to download a newer version due to company restrictions.
Any help would be much appreciated!
Kibana screenshot displaying logs
Update:
I've typed "GET /_cat/indices/*?v&s=index" into the Dev Tools>Console and got a list of indices.
I initially used the "DELETE" function, and it didn't appear to be working. However, after restarting everything, it worked the seond time and I was able to remove all the existing indices which subsiquently removed all logs being displayed in Kibana.
SUCCESS!
Kibana is just the visualization part of the elastic stack, your data is stored in elasticsearch, to get rid of it you need to delete your index.
The 5.4 version is very old and already passed the EOL date, it does not have any UI to delete the index, you will need to use the elasticsearch REST API to delete it.
You can do it from kibana, just click in Dev Tools, first you will need to list your index using the cat indices endpoint.
GET "/_cat/indices?v&s=index&pretty"
After that you will need to use the delete api endpoint to delete your index.
DELETE /name-of-your-index
On the newer versions you can do it using the Index Management UI, you should try to talk with your company to get the new version.
I am currently using Elastic Cloud to store my AWS CloudWatch logs. Everything seems to work fine as I'm already able to display charts and to query ElasticSearch correctly. Yet, I got a strange behavior I can't explain.
I am logging some events from my app. Let's say request_start and request_end. They are both available on Kibana. Yet, I'm also logging another event, let's say request_middle. I can see it on CloudWatch.
When checking in the Discover tab of Kibana, I don't see this event. I tried event:"request_middle" query, in vain. And if I display a list of all events under this same tab, I get a full list, except request_middle.
I tried to query directly Elastic Search, in case of. But no results as well.
Have some of you already encountered such a case? If so, how did you fix it?