how to overwrite the data in hive using sqoop - hadoop

Question

I am trying to load data into an already existing table in hive via sqoop from mysql database. I am referring to the below guide for reference:-
http://sqoop.apache.org/docs/1.4.5/SqoopUserGuide.html#_importing_data_into_hive
--hive-import has been tried and tested successfully.
I created a hive table as below:-
create table sqoophive (id int, name string, location string)
row format delimited
fields terminated by '\t'
lines terminated by '\n'
stored as textfile;
Loaded the data as required.
I want to use --hive-overwrite option to overwrite the content in the above table. As per the guide mentioned above - "--hive-overwrite Overwrite existing data in the Hive table."
"If the Hive table already exists, you can specify the --hive-overwrite option to indicate that existing table in hive must be replaced."
So I tried the below queries separately to get the result:-
sqoop import --connect jdbc:mysql://localhost/test --username root --password 'hr' --table sample --hive-import --hive-overwrite --hive-table sqoophive -m 1 --fields-terminated-by '\t' --lines-terminated-by '\n'
sqoop import --connect jdbc:mysql://localhost/test --username root --password 'hr' --table sample --hive-overwrite --hive-table sqoophive -m 1 --fields-terminated-by '\t' --lines-terminated-by '\n'
but rather than replacing the content in the existing table it just created a file in the below path /user/<username>/<mysqltablename>
Can somebody please explain me where I am going wrong?

Answer 1

the first query should work fine. I didn't give fields terminated and lines terminated as the schema already exists.
the keywords --hive-import and --hive-overwrite should be there.
if only --hive-overwrite is there, it doesn't load data to the table. just copies to hdfs.

Answer 2

It's putting the _SUCCESS file in
/user/<username>/<mysqltablename>
You can change where that goes with --warehouse-dir
ex: --warehouse-dir /tmp
One would think that hive-overwrite would handle this, meaning remove that directory first. But for good reason Hive doesn't want to start removing dirs in HDFS. What if something else was put in there?
hive-overwrite is saying, "I'm going to overwrite the rows in Hive, not just add to the table." Thus you will not have duplicates.
You have to remove that directory and the _SUCCESS file first; or better yet, right after the import is successful.
hadoop fs -rm -R /user/<username>/<mysqltablename>

Answer 3

sqoop import with out --target-dir OR --warehouse-dir (for --hive-import) will import /user/<username>/<mysqltablename>:
By default, Sqoop will import a table named foo to a directory named
foo inside your home directory in HDFS. For example, if your username
is someuser, then the import tool will write to
/user/someuser/foo/(files). You can adjust the parent directory of the
import with the --warehouse-dir argument.
You can also explicitly choose the target directory with --target-dir param
but as #hrobertv said that --hive-overwrite does not delete existing dir but it overwrites the HDFS data location of hive table. if you want to save new data at same location as origin than you would have to delete the existing table dir first and then run sqoop import with specifying --target-dir OR --warehouse-dir for --hive-overwrite to store data at specific location as per your requirement...