See table and control file in the bottom. It works if my LOAD_CLOB column data type is varchar2 instead of clob. If the column is clob datatype, it throws an error.
SQL*Loader-309: No SQL string allowed as part of HEBE_AHRA_DETAILS field specification
But I need the datatype to be clob because the string will be long.
Create table Table_load
(Row_id varchar2(20),
Load_clob clob);
Control File
OPTIONS (SKIP=1, DIRECT=true)
UNRECOVERABLE
load data
infile 'LOAD.txt'
badfile 'BAD.bad'
discardfile 'Discard.dsc'
replace
into table Table_load
fields terminated by "|" TRAILING NULLCOLS
( ROW_ID ":ROW_ID",
LOAD_CLOB "'Current Row ID '||:ROW_ID"
)
Related
I have to load data from a text file into a table. My data in text file is delimited by ',' and each item is present in double quotes (i.e., "").
For example, data in the text file is like below:
"1009","John","NY","USA"
"1010","Ron","AZ","USA"
How should I write my control file in order not to include the double quotes (i.e., "") while loading data into the table.
Assuming that the table structure is like the following:
create table someTable(
colA number,
colB varchar2(100),
colC varchar2(100),
colD varchar2(100)
)
You can use the SQLLoader with a control file like:
OPTIONS(skip=0)
load data
infile "data.txt"
append into TABLE someTable
fields
terminated by ','
enclosed by '"'
(
colA "to_number(:colA)", /* here you can use a format for numbers, if any */
colB,
colC,
colD
)
I'm trying to load the data from an xml file to a table. I get the below errors, please help me out.
Table:
CREATE TABLE TEST_XML
(FILL CHAR(30)
XMLDATA CLOB);
Here is my control file
LOAD DATA
INFILE *
TRUNCATE INTO TABLE TEST_XML XMLType(XMLDATA)
FIELDS ( FILL FILLER CHAR(100), XMLDATA LOBFILE(CONSTANT test_file.xml) TERMINATED BY EOF )
BEGINDATA 0
I get the below error:
Table TEST_XML, loaded from every logical record. Insert option in
effect for this table: TRUNCATE
Column Name Position Len Term Encl Datatype
------------------------------ ---------- ----- ---- ---- --------------------- FILL FIRST 100 CHARACTER (FILLER FIELD) XMLDATA
DERIVED * EOF CHARACTER
Static LOBFILE. Filename is test_file.xml
Record 1: Rejected - Error on table TEST_XML. ORA-01008: not all
variables bound
For me it is invalid syntax in control file. Oder of key word is relevant. Also like after begindata
LOAD DATA
INFILE *
INTO TABLE TEST_XML
truncate
FIELDS
( FILL FILLER CHAR(100)
,XMLDATA LOBFILE(CONSTANT test_file.xml) TERMINATED BY EOF )
BEGINDATA
0
I need to load a CSV file into table using sql loader but the lenth of a column is exceeding the Max length of varchar2 data type.
Create table statement :
CREATE TABLE TEST_PIPE_SEP (FILE_NM VARCHAR2(3000), KEY_COL VARCHAr2(4000), DESCR VARCHAR2(4000), RUN_DATE DATE );
CTL file :
load data
into table test_pipe_sep
append
fields terminated by ','
TRAILING NULLCOLS
(
FILE_NM char(4000) "trim(:FILE_NM)",
KEY_COL char(4000) "trim(:KEY_COL)",
DESCR "trim(:DESCR)",
RUN_DATE "to_date(sysdate, 'dd-mon-yyyy hh24:mi:ss')"
)
CSV sample record :
sample_file_name.csv,"B"|"STRESS_TESTING_SCENARIO_ID"|"TRANCHE_COLLATERAL_TYPE"|"TRANCHE_GUARANTEE_TYPE"|"BS_TYPE"|"CONTRACT_REFERENCE"|"CONTRACT_TYPE"|,Not Present in file2
I just not paste the full text here as it will become lengthy but you can append the length of middle column in CSV or KEY_COL field value to more than 4000 for testing.
SQLLDR :
sqlldr userid=$SQL_CREDENTIALS control=new_test_3.ctl data=data/$filename log=logs/$filename.log bad=logs/$filename.bad skip=1
Please suggest if there is any way where we can load a string of length more than 4000 using SQL LOADER or is there any workaround for this kind of problem.
I am on :
Oracle Database 11g Enterprise Edition Release 11.2.0.4.0 - 64bit Production
I did it ! :)
Change the column data type to CLOB as below :
CREATE TABLE TEST_PIPE_SEP (FILE_NM VARCHAR2(3000), KEY_COL CLOB, DESCR VARCHAR2(4000), RUN_DATE DATE );
Then the CTL I used to load data into CLOB column is as below :
load data
into table test_pipe_sep
append
fields terminated by ','
TRAILING NULLCOLS
(
FILE_NM char(4000) "trim(:FILE_NM)",
KEY_COL CHAR(30000) optionally ENCLOSED BY '<' AND '>',
DESCR "trim(:DESCR)",
RUN_DATE "to_date(sysdate, 'dd-mon-yyyy hh24:mi:ss')"
)
I have a raw external table with four columns-
Table 1 :
create external table external_partitioned_rawtable (age_bucket
String,country_destination String,gender
string,population_in_thousandsyear int) row format delimited
fields terminated by '\t' lines terminated by '\n' location
'/user/HadoopUser/hive'
I want a external table with partitions from Country_destination and gender.Table -2
create external table external_partitioned (age_bucket
String,population_in_thousandsyear int) partitioned
by(country_destination String,gender String) row format delimited
fields terminated by '\t' lines terminated by '\n';
Insert Overwrite is failing with null pointer exception-
insert overwrite table external_partitioned partition(country_destination,gender) <br>
select (age_bucket,population_in_thousandsyear,country_destination,gender) <br>
from external_partitioned_rawtable;
FAILED: NullPointerException null
For dynamic partition insertion, before executing the INSERT statement you have to execute two properties of hive:
set hive.exec.dynamic.partition=true;
set hive.exec.dynamic.partition.mode=nonstrict;
then execute insert statement(which I have modified)
insert overwrite table external_partitioned partition(country_destination,gender)
select age_bucket,population_in_thousandsyear,country_destination,gender
from external_partitioned_rawtable;
I hope this help you!!!
I am trying to load data into hive tables which is delimited by double pipe(||). When I try this :
Sample I/P:
1405983600000||111.111.82.41||806065581||session-id
Creating table in hive:
create table test_hive(k1 string, k2 string, k3 string, k4 string,) row format delimited fields terminated by '||' stored as textfile;
Loading data from text file:
load data local inpath '/Desktop/input.txt' into table test_hive;
When I do this it is storing data in the below format:
1405983600000 tabspace-as-second-column 111.111.82.41 tabspace-as-fourth-column
Where as I am expecting the data in table to be
1405983600000 111.111.82.41 806065581 session-id
Kindly help me out I have tried different options on this but unable to resolve it
Multicharater delimiter eg. || is not supported in Hive till ver 0.13 . So fields terminated by || won't work out.There is an alter native for this.
CREATE EXTERNAL TABLE page_view(viewTime INT, userid BIGINT,
page_url STRING, referrer_url STRING,
ip STRING COMMENT 'IP Address of the User',
country STRING COMMENT 'country of origination')
COMMENT 'This is the staging page view table'
ROW FORMAT DELIMITED FIELDS TERMINATED BY '\054'
SERDE serde_name WITH SERDEPROPERTIES (field.delim='||')
STORED AS TEXTFILE
LOCATION '<hdfs_location>';
The default serde can be used. Multi character delimiters can be used for fields , line , escape characters by specifying them in the serde properties.
This issue has been resolved in hive 14 with the use of multidelimiter serde. Please find documentation here.
https://cwiki.apache.org/confluence/display/Hive/MultiDelimitSerDe
You could do this if you don't want to use alternate serde or have earlier version of hive:
create external table my_table (line string) location /path/file;
Then create view on top:
create view my_view as select split(line,'\\|\\|')[0] as column_1
, split(line,'\\|\\|')[1] as column_2
, split(line,'\\|\\|')[2] as column_3
from my_table;
Query the view. Good luck.