Whenever hive table is locked in exclusive mode query to describe the same table hangs until lock is released. Is there a way to introduce a timeout after which describe query would just error out?
Before executing DESC <TABLE_NAME> command, check lock status SHOW LOCKS <TABLE_NAME>;
If table is un-locked then
DESC <TABLE_NAME>
Related
I ran an ALTER sql on Table A. Then table A is stuck, system.mutations is not responding. I also wait 3 hours. I tried below actions,
restart Clickhouse service, reboot this Clickhouse server machine. reboot my client computer.
checked log, and found this, "Current max source part size for mutation is 0 but part size 1513. Will not mutate part all_24_24_0. "
The sql I used is like "alter table A update columnA=columnA/(select sum()....) where ...."
No other data is inserting. No recoreds in system.merges or system.replication_queue.
Table A has around 1000 records
I just want to drop this table A, then I can recreate it with the history records, but table A is not responding. Also system.mutations.
We have a daily partitioned table with retention of about 180 days. We have created a view with a group by .., to_char(DATE_COL,'YYYYMM');
The users started to extract the data from the views for each month; and at one moment of executing the view for 201510, it had failed with "Object no longer exists" even though the view and the underlying table exists.
I am suspecting that Unix issued a new partition creation statement for the next day which is automated process(verified in data dictionary);
The question is if the query on the view is running, there would be read lock on the table, thereby the partition couldn't have been created as it needs exclusive lock;
If the view wasn't running, then the ALTER table statement to create partition would have been completed and then if there was any query on the view, it wouldn't have been failed.
Did the query on the view fired at almost the same time when alter table statement to add partition was being executed; if so, as there is an exclusive lock on the table through alter table statement, the query on the view would have waited for read lock as currently there is exclusive lock. Why did I see this error, can you please elucidate.
I am running an mload script on a table that is cleared at the beginning of a job. When the script fails, the error and log tables prevent the job from running a second time. How does one reset mload cleanly after a failure?
You need to drop the work/error/log table
DROP TABLE UV_mytable;
DROP TABLE ET_mytable;
DROP TABLE WT_mytable;
DROP TABLE LT_mytable;
And then release the load lock:
RELEASE MLOAD mytable;,
If this fails
RELEASE MLOAD mytable IN APPLY;
But why does the job fail at all?
HIVE 0.13 will SHARED lock the entire database(I see a node like LOCK-0000000000 as a child of the database node in Zookeeper) when running a select statement on any table in the database. HIVE creates a shared lock on the entire schema even when running a select statement - this results in a freeze on CREATE/DELETE statements on other tables in the database until the original query finishes and the lock is released.
Does anybody know a way around this? Following link suggests concurrency to be turned off but we can't do that as we are replacing the entire table and we have to make sure that no select statement is accessing the table before we replace the entire contents.
http://mail-archives.apache.org/mod_mbox/hive-user/201408.mbox/%3C0eba01cfc035$3501e4f0$9f05aed0$#com%3E
use mydatabase;
select count(*) from large_table limit 1; # this table is very large and hive.support.concurrency=true`
In another hive shell, meanwhile the 1st query is executing:
use mydatabase;
create table sometable (id string) ROW FORMAT DELIMITED FIELDS TERMINATED BY '\t' STORED AS TEXTFILE ;
The problem is that the “create table” does not execute untill the first query (select) has finished.
Update:
We are using Cloudera's distribution of Hive CDH-5.2.1-1 and we are seeing this issue.
I think they never made such that in Hive 0.13. Please verify your Resource manager and see that you have enough memory when you are executing multiple Hive queries.
As you know each Hive query will trigger a map reduce job and if YARN doesn't have enough resources it will wait till the previous running job completes. Please approach your issue from memory point of view.
All the best !!
I am using Postgres 8.4
I need to execute an ALTER statement on a running database with ~4M data on the relevant table. My sql is like:
ALTER TABLE some_table ALTER a_row bigint;
Now, relevant row type is int
But what i wonder is data consistency, Nearly 3-4 records are written to that table and some more are being read per second.
What i need to do for avoiding data consistency and such other problems.
When you execute and ALTER TABLE sql, table will be locked and you shouldn't have any problems except some possible performance issues in INSERT sqls in your case. But if you are going to do this once, there is no reason to hesitate.