Im using Drill in my hadoop cluster, having drillbits across all nodes. Currently im using Drill to query Hive tables and i need to know the exact query execution time. For a more friendy approach, i started to use the Drill Web UI available on port 8047 and i run the queries and then i cant access the profile part to see how much time it took.
The problem is that sometimes the queries result dont show on the list of the profile page.
Does anyone knows why this happens?
Thanks in advance
I believe this is because each Drillbit only has the profile for the queries it was the Foreman for. Going into a bit more detail, when you run a Drill query from the command line the following happens:
You enter Query A on the cli.
The cli picks a Drillbit to be the Foreman for Query A. The Foreman is the Drillbit that coordinates running the query. Note: each query can have a different Foreman.
Query A runs and Query A's Foreman stores the profile locally. This means Query A's profile will only be available on it's Foreman, if you go to a different Drillbit it will not have a profile for Query A.
The only real solution for this is that you will have to check each Drillbit in the cluster to see which one has the profile for your query, or to have one Drillbit be the designated Foreman for all queries.
Related
If want to see, all the ran queries, from Kibana Dev tool, how to see it? Is there any query, which shows all the ran queries may be for last 30 days?
I guess there is no Api for showing history of executed queries how ever you can find the related log files in var/log/elasticsearch file path More over if you want to always keep record of all executed queries and events in your Dev tools you can create and index for it and use Logstash to insert your operation logs(which previously said where is stored) into that index.
enter image description hereOur group members used beeswax to do the hive queries.
When few people do the query,everything is okay.
But when lots people do the query at the same time or in a short time,something strange happened. The database list keep loading and the whole browser will be no response.At this time, the cores of hive servers and metastore servers were always more than one core.
So I had to wait for a long time or restart the whole hive server and everything going okay again.
I had checked the log of hue and hive server,but found nothing useful.(so sad of it)
I had worked for this question for a every log time but could not get the solution of this problem.
Because of the limit of network,I can not paste the screenshots and any logs here.
Are there anyone can help me.
I search this picture from internet.I made a mark on it.When the problem happened,there is a loading icon on the it and it was impossible to change database.
I have an application running CockroachDB on AWS and want to log all of the queries it executes. However, I can’t find anything in my cluster’s logs that seems to contain the queries. I tried grepping through them for my application’s SELECT statements but can’t find them.
Is there a way to enable query logging?
The other answer is outdated. There are two mechanisms:
for regular inspection/troubleshooting, one should use SQL statement execution logging as documented here: https://www.cockroachlabs.com/docs/stable/query-behavior-troubleshooting.html#cluster-wide-execution-logs
for access audits (when required for e.g. regulation) use audit logging: https://www.cockroachlabs.com/docs/stable/sql-audit-logging.html
When starting a node with cockroach start, include the --vmodule=executor=2 flag to log all SQL queries. For example:
cockroach start --background --vmodule=executor=2
You can access the logs then from the log files stored in cockroach-data/logs
If you’re curious about logging queries that run “too long”, set the env var COCKROACH_TRACE_SQL to a duration to log all queries longer than that duration.
One can also activate this logging after starting the server by accessing the URL http://[admin UI URL]/debug/vmodule/executor=2 using the same host/port as the admin UI.
Good Day.
I've been developing with meteorJS which uses mongodb. No problems there. I've been using the mongo shell to access the database on my dev machine (osx 10.11). This is my first project with mongo and when the shell would load, it would connect to db.test and I'd always show dbs and get the list of database, then use myApp.
Yesterday whenever I go into the shell and I type show dbs the only one shown is local 0.078GB. However my app is still working and pulling and pushing data to the database.
I've checked the dbpath in the mongod.conf and that seems ok. I'm not entirely sure about the exact order of things, but two things where different (I'm not sure if these happened prior to the show dbs not showing everything or after, and I'm not sure which came first):
when loading the mongo shell I was getting this error:
WARNING: soft rlimits too low. Number of files is 256, should be at least 1000"
I followed these directions which seemed to stop that error from appearing (https://github.com/basho/basho_docs/issues/1402 )
I use Meteor Toys and for the first time I update user.profile.companyName (which is a custom field within the standard profile from within the Meteor Toys widget.
Just odd that the app can still access the database and collections, but that the mongo shell doesn't show. I've update mongod via brew upgrade mongodb from 3.0.2 to 3.0.7 to no avail.
Any ideas?
If you want to use the regular mongo console you have to specify the port to be 3001 for meteor apps instead of the default 27017. Otherwise it's much simpler to just type meteor mongo and connect that way. Then you can type 'show collections' and it will show them all just like normal.
MongoDB do not show the database unless if there is minimum of one collection with a document in it.
Refer to this link
I am using JMeter to test performance of the following Server Infrastructure. The Code Base uses ICEfaces framework and hence generates dynamic ID's each time there is a new build.
I record the scripts and run them for different variants of load (10 Users, 20 Users, 30 Users and so on). Whenever a new code base is deployed, because of change in ID's, I have to re-record the scripts before I perform Test runs again.
As of now I am able to satisfactorily get my job done.
I wish to take my job to a whole new level by trying to test performance on the following Server Infrastructure.
The following are my issues -
Because of two different nodes (Node1 and Node2) each node has a unique set of dynamic ID's associated with it and when I record a script on a particular login session, I cannot be sure of the Node my session is pinned on and as a result the recorded script is tailor made for a single node and not a cluster.
When "Load Balancer" gets in action, I cannot be sure of the Node JMeter hits during performance run and for obvious reasons the run fails to generate results.
I want a cleaver way to record script which can successfully run on a multiple server configuration.
How to perform performance Testing on this configuration?