If want to see, all the ran queries, from Kibana Dev tool, how to see it? Is there any query, which shows all the ran queries may be for last 30 days?
I guess there is no Api for showing history of executed queries how ever you can find the related log files in var/log/elasticsearch file path More over if you want to always keep record of all executed queries and events in your Dev tools you can create and index for it and use Logstash to insert your operation logs(which previously said where is stored) into that index.
Related
Is it possible to save a bunch of queries into a single JSON file to import in Kibana Console?
I know there's an option to save a single query[2] and the Kibana console is based on local storage, but I would like to load up the queries based on parameters, such that changing the params(e.g load_from=filename.json) should load up a different set of queries.
For example, when I open http://localhost:5601/app/kibana#/dev_tools/console?load_from=filename.json, it should open the Kibana console with ES queries from the file.
EDIT: As a workaround, it's possible to do this with Postman API Client or similar API clients.
Solution:
EDIT 2 on 22/02/2022: Kibana Spaces is the answer. It lets you organize dashboards and other saved objects into meaningful categories[3]. Whenever you load http://localhost:5601/ it lets you choose the space you want to work with. Having multiple browser tabs with different saved spaces should work for most cases.
[2] https://www.elastic.co/guide/en/kibana/master/save-load-delete-query.html
[3] https://www.elastic.co/guide/en/kibana/master/xpack-spaces.html
Unfortunately, that's not possible yet.
Elastic is (supposedly) working on a new Kibana feature (tabbed console panes #10095) that will provide support for better organizing the code in the Dev Tools application. The issue has been opened for a while and not much seems to be happening, so we'll see.
The release date of that feature is not known yet.
I have managed to process log files using the ELK kit and I can now see my logs on Kibana.
I have scoured the internet and can't seem to find a way to remove all the old logs, viewable in Kibana, from months ago. (Well an explaination that I understand). I just want to clear my Kibana and start a fresh by loading new logs and them being the only ones displayed. Does anyone know how I would do that?
Note: Even if I remove all the Index Patterns (in Management section), the processed logs are still there.
Context: I have been looking at using ELK to analyse testing logs in my work. For that reason, I am using ElasticSearch, Kibana and Logstatsh v5.4, and I am unable to download a newer version due to company restrictions.
Any help would be much appreciated!
Kibana screenshot displaying logs
Update:
I've typed "GET /_cat/indices/*?v&s=index" into the Dev Tools>Console and got a list of indices.
I initially used the "DELETE" function, and it didn't appear to be working. However, after restarting everything, it worked the seond time and I was able to remove all the existing indices which subsiquently removed all logs being displayed in Kibana.
SUCCESS!
Kibana is just the visualization part of the elastic stack, your data is stored in elasticsearch, to get rid of it you need to delete your index.
The 5.4 version is very old and already passed the EOL date, it does not have any UI to delete the index, you will need to use the elasticsearch REST API to delete it.
You can do it from kibana, just click in Dev Tools, first you will need to list your index using the cat indices endpoint.
GET "/_cat/indices?v&s=index&pretty"
After that you will need to use the delete api endpoint to delete your index.
DELETE /name-of-your-index
On the newer versions you can do it using the Index Management UI, you should try to talk with your company to get the new version.
Im using Drill in my hadoop cluster, having drillbits across all nodes. Currently im using Drill to query Hive tables and i need to know the exact query execution time. For a more friendy approach, i started to use the Drill Web UI available on port 8047 and i run the queries and then i cant access the profile part to see how much time it took.
The problem is that sometimes the queries result dont show on the list of the profile page.
Does anyone knows why this happens?
Thanks in advance
I believe this is because each Drillbit only has the profile for the queries it was the Foreman for. Going into a bit more detail, when you run a Drill query from the command line the following happens:
You enter Query A on the cli.
The cli picks a Drillbit to be the Foreman for Query A. The Foreman is the Drillbit that coordinates running the query. Note: each query can have a different Foreman.
Query A runs and Query A's Foreman stores the profile locally. This means Query A's profile will only be available on it's Foreman, if you go to a different Drillbit it will not have a profile for Query A.
The only real solution for this is that you will have to check each Drillbit in the cluster to see which one has the profile for your query, or to have one Drillbit be the designated Foreman for all queries.
I have an application running CockroachDB on AWS and want to log all of the queries it executes. However, I can’t find anything in my cluster’s logs that seems to contain the queries. I tried grepping through them for my application’s SELECT statements but can’t find them.
Is there a way to enable query logging?
The other answer is outdated. There are two mechanisms:
for regular inspection/troubleshooting, one should use SQL statement execution logging as documented here: https://www.cockroachlabs.com/docs/stable/query-behavior-troubleshooting.html#cluster-wide-execution-logs
for access audits (when required for e.g. regulation) use audit logging: https://www.cockroachlabs.com/docs/stable/sql-audit-logging.html
When starting a node with cockroach start, include the --vmodule=executor=2 flag to log all SQL queries. For example:
cockroach start --background --vmodule=executor=2
You can access the logs then from the log files stored in cockroach-data/logs
If you’re curious about logging queries that run “too long”, set the env var COCKROACH_TRACE_SQL to a duration to log all queries longer than that duration.
One can also activate this logging after starting the server by accessing the URL http://[admin UI URL]/debug/vmodule/executor=2 using the same host/port as the admin UI.
I'm using Logstash, Elasticsearch and Kibana to process, store and visualize my logs.
My setup works fine but now I'm looking for a new tool : before ELK I was used to read my logs on Notepad++ or Glogg (I'm on Windows) and now I'm using only kibana discover tab.
Do you think I can find a native application that looks like a read-only Notepad++ that query Elasticsearch and display my logs like before ?
The three features I actually need are :
querying multiple sources logs,
for a specified date range,
and display it quickly to a concise and fast viewer.
I don't think it's very complicated to implement, so that's why i'm wondering if it already exists :)