My hive table name is in the below format:
schema_name.hive_table_name
eg: schema1.abc;
Now when I try to do MSCK repair table on the above hive table it throws below error.
Logging initialized using configuration in file:/etc/hive/conf.dist/hive-log4j.properties
FAILED: ParseException line 1:28 missing EOF at '.' near 'schema_name'
Below is the command I used:
hive -e "MSCK repair table schema_name.hive_table_name"
Could any one help on this?
I tried the below statement:
hive -e "use schema_name;MSCK repair table hive_table_name"
This allows to add partition to hive with the specific schema mentioned .
It worked for me.
Thanks
Related
Is there a way to catch all schema + table name info in a single command through Hive in a similar way to
SELECT * FROM information_schema.tables
from the PostgreSQL world?
show databases and show tables combined in a loop [here an example] is an answer, but I'm looking for a more compact way to have the same result in a single command.
It's been long I have worked on Hive Queries but as far as I remember you can probably use
hive> desc formatted tableName;
or
hive> describe formatted tableName;
It will give you all the relevant information related to the Table like the Schema, Partition info, Table Type like Managed Table, etc
I am not sure If you are particularly looking for this ??
There is another way to query Hive Tables, is writing Hive Scripts which can be called from Hadoop Terminal rather than from Hive Terminal itself.
std]$ cat sample.hql or vi sample.hql
use dbName;
select * from tableName;
desc formatted tableName;
# this hql script can be called from outside the hive terminal
std]$ hive -f sample.hql
or, without even have to write script file you can probably query hive as
std]$ hive -e "use dbName; select * from emp;" > text.txt or >> to append
On the Database level, you can probably query as :
hive> use dbName;
hive> set hive.cli.print.current.db=true;
hive(dbName)> describe database dbName;
it will bring metadata from MySQL(metastore) about the Database.
I am trying to load csv file into an hbase table using shell command Dimporttsv.
The csv files reside in a dir in my hdfs (/csvFiles)
the csv file was generated from a mysql table with the following feilds:
+-------------+
Field
+-------------+
tweet_id
user_id
screen_name
description
created_at
+-------------+
I created a table in hbase with a single family name as shown below:
create 'dummyTable', 'cf1'
the command I am using:
ImportTsv -Dimporttsv.separator=',' -Dimporttsv.columns=HBASE_ROW_KEY,cf1:user_id,cf1:tweet_id,cf1:screen_name,cf1:description,cf1:created_at dummyTable /csvFiles/all_users.csv
however I am getting this syntax error:
SyntaxError: (hbase):8: syntax error, unexpected tSYMBEG
I've looked at the following posts and followed the recommendations in them but to no avail. I would appreciate your help.
Import TSV file into hbase table
https://community.hortonworks.com/articles/4942/import-csv-data-into-hbase-using-importtsv.html
http://hbase.apache.org/book.html#importtsv
Exit from Hbase shell and try by adding single quotes to importtsv.columns
bash$ hbase org.apache.hadoop.hbase.mapreduce.ImportTsv -Dimporttsv.separator=',' -Dimporttsv.columns='HBASE_ROW_KEY,cf1:user_id,cf1:tweet_id,cf1:screen_name,cf1:description,cf1:created_at' dummyTable hdfs://<your_name_node_addr>/csvFiles/all_users.csv
(or)
From Hbase Shell:
hbase(main):001:0> ImportTsv -Dimporttsv.separator=',' -Dimporttsv.columns='HBASE_ROW_KEY,cf1:user_id,cf1:tweet_id,cf1:screen_name,cf1:description,cf1:created_at' dummyTable hdfs://<your_name_node_addr>/csvFiles/all_users.csv
I have a Hive table partitioned on an 'int' column.
I want to export the Hive table to MySql using Sqoop export tool.
sqoop export --connect jdbc:mysql://XXXX:3306/temp --username root --password root --table emp --hcatalog-database temp --hcatalog-table emp
I tried the above sqoop command but it failed with below exception.
ERROR tool.ExportTool: Encountered IOException running export job: java.io.IOException:
The table provided temp.emp uses unsupported partitioning key type for column mth_id : int.
Only string fields are allowed in partition columns in HCatalog
I understand that the partition on int column is not supported.
But would like to check whether this issue is fixed in any of the latest releases with an extra config/option.
As a workaround, I can create another table without a partition before exporting.But I would like to check whether there is a better way to achieve this?
Thanks in advance.
I am trying to change the partition location of my external hive table.
Command that I try to run:
ALTER TALBE sl_uploads PARTITION (hivetimestamp='2016-07-26 15:00:00') SET LOCATION '/data/dev/event/uploads/hivetimestamp=2016-07-26 15:00:00'
Error I get :
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. java.net.URISyntaxException: Illegal character in path
My data for a particular partition exists at the path:
/data/dev/event/uploads/hivetimestamp=date time/actual_data
I think space is creating an issue. But any help on this would be great.
your hdfs in path is right?
add /actual_data/?
Hive is unable to read the full hdfs path due to space in "2016-07-26 15:00:00";
you can use below commands;
hive> set part=2016-07-26 15:00:00;
hive>ALTER TALBE sl_uploads PARTITION (hivetimestamp='2016-07-26 15:00:00') SET LOCATION '/data/dev/event/uploads/hivetimestamp=#part';
First thing i saw that, you wrote TALBE instead of TABLE.
I am not sure what I am doing wrong here:
hive> CREATE TABLE default.testtbl(int1 INT,string1 STRING)
stored as orc
tblproperties ("orc.compress"="NONE")
LOCATION "/user/hive/test_table";
FAILED: ParseException line 1:107 missing EOF at 'LOCATION' near ')'
while the following query works perfectly fine:
hive> CREATE TABLE default.testtbl(int1 INT,string1 STRING)
stored as orc
tblproperties ("orc.compress"="NONE");
OK
Time taken: 0.106 seconds
Am I missing something here. Any pointers will help. Thanks!
Try put the "LOCATION" in front of "tblproperties" like below, worked for me.
CREATE TABLE default.testtbl(int1 INT,string1 STRING)
stored as orc
LOCATION "/user/hive/test_table"
tblproperties ("orc.compress"="NONE");
It seems even the sample SQL from book "Programming Hive" got the order wrong. Please reference to the official definition of create table command:
https://cwiki.apache.org/confluence/display/Hive/LanguageManual+DDL#LanguageManualDDL-CreateTable
#Haiying Wang pointed out that LOCATION is to be put in front of tblproperties.
But I think the error also occurs when location is specified above stored as.
Its better to stick to the correct order:
CREATE [TEMPORARY] [EXTERNAL] TABLE [IF NOT EXISTS] [db_name.]table_name -- (Note: TEMPORARY available in Hive 0.14.0 and later)
[(col_name data_type [COMMENT col_comment], ... [constraint_specification])]
[COMMENT table_comment]
[PARTITIONED BY (col_name data_type [COMMENT col_comment], ...)]
[CLUSTERED BY (col_name, col_name, ...) [SORTED BY (col_name [ASC|DESC], ...)] INTO num_buckets BUCKETS]
[SKEWED BY (col_name, col_name, ...) -- (Note: Available in Hive 0.10.0 and later)]
ON ((col_value, col_value, ...), (col_value, col_value, ...), ...)
[STORED AS DIRECTORIES]
[
[ROW FORMAT row_format]
[STORED AS file_format]
| STORED BY 'storage.handler.class.name' [WITH SERDEPROPERTIES (...)] -- (Note: Available in Hive 0.6.0 and later)
]
[LOCATION hdfs_path]
[TBLPROPERTIES (property_name=property_value, ...)] -- (Note: Available in Hive 0.6.0 and later)
[AS select_statement]; -- (Note: Available in Hive 0.5.0 and later; not supported for external tables)
Refer: Hive Create Table
Check this post:
Loading Data from a .txt file to Table Stored as ORC in Hive
And check your source files present at the specified directory /user/hive/test_table. Incase the files are in .txt or some other non ORC format then you can follow the steps in the above post to come out of the error.
ParseException line lineNumber missing EOF at '.' near 'schemaName':
Got the above error while trying to execute the following command from linux script to truncate a hive table
dse -u username -p password hive -e "truncate table keyspace.tablename;"
Fix:
Need to separate the commands within the script line as follows -
dse -u username -p password hive -e "use keyspace; truncate table keyspace.tablename;"
Happy coding!
Got the same error while creating a table in hive.
I used the drop command to drop the table and then run the create table command that I had again.
Worked for me.
If you see this error when running the HiveQL from a file with the command "hive -f file.hql". And that it points the first line of your query most definitely this is because of a forgotten semicolon(;) for a previous query.
Since parser looks for semicolon(;) as a terminator for each query.
for example:
DROP TABLE IF EXISTS default.emp
create table default.emp (
field1 type,
field2 type)
ROW FORMAT DELIMITED FIELDS TERMINATED BY '|'
STORED AS TEXTFILE
LOCATION 's3://gts-promocube/source-data/Lowes/POS/';
If you save the above in a file and execute it with hive -f, then you'll get the error:
FAILED: ParseException line 2:0 missing EOF at 'CREATE' near emp.
Solution: Put a semicolon(;) for the DROP TABLE command above.