I'm trying to load the data from an xml file to a table. I get the below errors, please help me out.
Table:
CREATE TABLE TEST_XML
(FILL CHAR(30)
XMLDATA CLOB);
Here is my control file
LOAD DATA
INFILE *
TRUNCATE INTO TABLE TEST_XML XMLType(XMLDATA)
FIELDS ( FILL FILLER CHAR(100), XMLDATA LOBFILE(CONSTANT test_file.xml) TERMINATED BY EOF )
BEGINDATA 0
I get the below error:
Table TEST_XML, loaded from every logical record. Insert option in
effect for this table: TRUNCATE
Column Name Position Len Term Encl Datatype
------------------------------ ---------- ----- ---- ---- --------------------- FILL FIRST 100 CHARACTER (FILLER FIELD) XMLDATA
DERIVED * EOF CHARACTER
Static LOBFILE. Filename is test_file.xml
Record 1: Rejected - Error on table TEST_XML. ORA-01008: not all
variables bound
For me it is invalid syntax in control file. Oder of key word is relevant. Also like after begindata
LOAD DATA
INFILE *
INTO TABLE TEST_XML
truncate
FIELDS
( FILL FILLER CHAR(100)
,XMLDATA LOBFILE(CONSTANT test_file.xml) TERMINATED BY EOF )
BEGINDATA
0
Related
using the control file below to load data
LOAD DATA
INFILE '/c/Transaction.txt'
INTO TABLE tab1 APPEND WHEN (1:1) = 'D'
(RUN_ID "RUN_ID_SEQ.NEXTVAL"
,RUN_DATE_TIME "SYSDATE"
)
INTO TABLE tab2 TRUNCATE WHEN(1:1) <> 'D'
FIELDS TERMINATED BY '|' TRAILING NULLCOLS
(
DEALER_NUMBER
,TRAN_CODE
,TRAN_AMOUNT "TO_NUMBER(:PL818_TRAN_AMOUNT,'999999.99')"
,TRAN_DATE "TO_DATE(:PL818_TRAN_DATE,'DD-MM-YYYY')"
)
This is the data set
DEALER_ID|TRAN_TYPE_CODE|TRAN_AMOUNT|TRAN_DATE
203113|34|1000.50|12-07-2022
No errors during load. Data loads correctly into the first table but first two characters are skipped when loading data into second table, table data looks like this. What could be causing this?
3113 34 1000.5 12-JUL-22
Skips "20"
Sample tables:
SQL> desc tab1
Name Null? Type
----------------------------------------- -------- ----------------------------
RUN_ID NUMBER
RUN_DATE_TIME DATE
SQL> desc tab2
Name Null? Type
----------------------------------------- -------- ----------------------------
DEALER_NUMBER NUMBER
TRAN_CODE NUMBER
TRAN_AMOUNT NUMBER
TRAN_DATE DATE
SQL>
Control file; note position(1) when loading into tab2. You must use it when loading data into different tables, using the same control file:
LOAD DATA
INFILE *
INTO TABLE tab1 APPEND WHEN (1:1) = 'D'
(RUN_ID "RUN_ID_SEQ.NEXTVAL"
,RUN_DATE_TIME "SYSDATE"
)
INTO TABLE tab2 TRUNCATE WHEN(1:1) <> 'D'
FIELDS TERMINATED BY '|' TRAILING NULLCOLS
(
DEALER_NUMBER position(1) --> here
,TRAN_CODE
,TRAN_AMOUNT "TO_NUMBER(:TRAN_AMOUNT,'999999.99')"
,TRAN_DATE "TO_DATE(:TRAN_DATE,'DD-MM-YYYY')"
)
BEGINDATA
203113|34|1000.50|12-07-2022
Loading session and the result:
SQL> $sqlldr scott/tiger#orcl control=test42.ctl log=test42.log
SQL*Loader: Release 18.0.0.0.0 - Production on Sri Srp 13 08:28:19 2022
Version 18.5.0.0.0
Copyright (c) 1982, 2018, Oracle and/or its affiliates. All rights reserved.
Path used: Conventional
Commit point reached - logical record count 1
Table TAB1:
0 Rows successfully loaded.
Table TAB2:
1 Row successfully loaded.
Check the log file:
test42.log
for more information about the load.
SQL> select * from tab2;
DEALER_NUMBER TRAN_CODE TRAN_AMOUNT TRAN_DATE
------------- ---------- ----------- ----------
203113 34 1000,5 12-07-2022
--
here's your missing "20"
SQL>
See table and control file in the bottom. It works if my LOAD_CLOB column data type is varchar2 instead of clob. If the column is clob datatype, it throws an error.
SQL*Loader-309: No SQL string allowed as part of HEBE_AHRA_DETAILS field specification
But I need the datatype to be clob because the string will be long.
Create table Table_load
(Row_id varchar2(20),
Load_clob clob);
Control File
OPTIONS (SKIP=1, DIRECT=true)
UNRECOVERABLE
load data
infile 'LOAD.txt'
badfile 'BAD.bad'
discardfile 'Discard.dsc'
replace
into table Table_load
fields terminated by "|" TRAILING NULLCOLS
( ROW_ID ":ROW_ID",
LOAD_CLOB "'Current Row ID '||:ROW_ID"
)
i created a table using hive i want to partition the data based on location
create table student(
id bigint
,name string
,location string
, course array<string>)
ROW FORMAT DELIMiTED fields terminated by '\t'
collection items terminated by ','
stored as textfile;
and data like
100 student1 ongole java,.net,hadoop
101 student2 hyderabad .net,hadoop
102 student3 vizag java,hadoop
103 student4 ongole .net,hadoop
104 student5 vizag java,.net
105 student6 ongole java,.net,hadoop
106 student7 neollre .net,hadoop
creating partition table:
create table student_partition(
id bigint
,name string
,course array<string>)
PARTITIONED BY (address string)
ROW FORMAT DELIMiTED fields terminated by '\t'
collection items terminated by ','
stored as textfile;
INSERT OVERWRITE TABLE student_partition PARTITION(address) select *
from student;
i'm trying to partition the data based on location but it shows below error:
FAILED: SemanticException [Error 10044]: Line 1:23 Cannot insert into
target table because column number/types are different 'address':
Cannot convert column 2 from string to array.
please anyone help me.
The columns of the source and the target should match
Option 1: adjust the source to the target. The partition column goes last
insert into student_partition partition (address)
select id,name,course,location
from student
;
Option 2: adjust the target to the source
insert into student_partition partition (address) (id,name,address,course)
select *
from student
;
P.s.
You might need this -
set hive.exec.dynamic.partition.mode=nonstrict
;
I am trying to import some data with sql*loader but I can't import a latitude/longitude. On the table those columns are FLOAT(126) and on the data file is just text. I tried with FLOAT EXTERNAL on the sqlldr control file but it doesn't work. I got ORA-01722: invalid number.
Describe my_table;
Name Null Type
------------------------- -------- ------------
PRE_ID NOT NULL NUMBER(38)
PRE_DH NOT NULL TIMESTAMP(6)
PRE_PRO NOT NULL NUMBER(38)
PRE_INF NOT NULL NUMBER(38)
PRE_TPL NOT NULL NUMBER(38)
PRE_LAT NOT NULL FLOAT(126)
PRE_LNG NOT NULL FLOAT(126)
data file:
55831;08/12/2016 16:48:07;1;-128;2;-22.4741249084473;-50.55194854736336
55831;09/12/2016 08:02:06;1;-128;2;-22.5002975463867;-50.8194427490234
55831;09/12/2016 19:12:06;1;-128;2;-22.5002975463867;-50.8194427490234
and sqlldr control file:
load data
infile 'my_file.csv' "str '\r\n'"
append
into table my_table
fields terminated by ';'
trailing nullcols
( PRE_ID CHAR(4000),
PRE_DH TIMESTAMP "DD/MM/YYYY HH24:MI:SS",
PRE_PRO CHAR(4000),
PRE_INF CHAR(4000),
PRE_TPL CHAR(4000),
PRE_LAT FLOAT EXTERNAL,
PRE_LNG FLOAT EXTERNAL,
)
log file:
Table MY_TABLE, loaded from every logical record.
Insert option in effect for this table: APPEND
TRAILING NULLCOLS option in effect
Column Name Position Len Term Encl Datatype
------------------------------ ---------- ----- ---- ---- ---------------------
PRE_ID FIRST 4000 ; CHARACTER
PRE_DH NEXT * ; DATETIME DD/MM/YYYY HH24:MI:SS
PRE_PRO NEXT 4000 ; CHARACTER
PRE_INF NEXT 4000 ; CHARACTER
PRE_TPL NEXT 4000 ; CHARACTER
PRE_LAT NEXT * ; CHARACTER
PRE_LNG NEXT * ; CHARACTER
value used for ROWS parameter changed from 64 to 1
Record 1: Rejected - Error on table MY_TABLE, column PRE_LAT.
ORA-01722: invalid number
Record 2: Rejected - Error on table MY_TABLE, column PRE_LAT.
ORA-01722: invalid number
Record 3: Rejected - Error on table MY_TABLE, column PRE_LAT.
ORA-01722: invalid number
You're seeing this because the operating system environment is set up in a way that causes Oracle to treat a comma as a decimal separator and a period as a group separator. Your error messages are in English, interestingly, so not sure exactly what you have that set to, but you can see the same thing with something like NLS_LANG="FRENCH_FRANCE.WE8ISO8859P1".
From the log you can see that the field in your CSV file is being read as character data. The target column is a float (any type of number column would have the same issue), which means an implicit conversion is happening, and is using your NLS settings. You can see the same thing more simply with:
alter session set NLS_NUMERIC_CHARACTERS='.,';
select to_number('-22.4741249084473') from dual;
TO_NUMBER('-22.4741249084473')
------------------------------
-22.4741249084473
alter session set NLS_NUMERIC_CHARACTERS=',.';
select to_number('-22.4741249084473') from dual;
Error report -
ORA-01722: invalid number
Same conversion, but the alter session is swapping the meaning of the comma and period.
You can either explicitly set your environment to something with the right NLS numeric characters via NLS_LANG:
export NLS_LANG="ENGLISH_UNITED KINGDOM.WE8ISO8859P1"
or just that specific setting:
export NLS_NUMERIC_CHARACTERS='.,'
... before running SQL*Loader.
In order to load data (from a CSV file) into an Oracle database, I use SQL*Loader.
In the table that receives these data, there is a varchar2(500) column, called COMMENTS.
For some reasons, I want to ignore this information from the CSV file.
Thus, I wrote this control file:
Options (BindSize=10000000,Readsize=10000000,Rows=5000,Errors=100)
Load Data
Infile 'XXX.txt'
Append into table T_XXX
Fields Terminated By ';'
TRAILING NULLCOLS
(
...
COMMENTS FILLER,
...
)
This code seems to work correctly, as the COMMENTS field in database is always set to null.
However, if in my CSV file I have a record where the corresponding COMMENTS field exceeds the 500 characters limit, I get an error from SQL*Loader:
Record 2: Rejected - Error on table T_XXX, column COMMENTS.
Field in data file exceeds maximum length
Is there a way to really exclude the processing of my COMMENTS fields?
I can't reproduce your problem. I'm using Oracle 10.2.0.3.0 with SQL*Loader 10.2.0.1.
Here is my test case:
SQL> CREATE TABLE test_sqlldr (
2 ID NUMBER,
3 comments VARCHAR2(20),
4 id2 NUMBER
5 );
Table created
Control file:
LOAD DATA
INFILE test.data
INTO TABLE test_sqlldr
APPEND
FIELDS TERMINATED BY ';'
TRAILING NULLCOLS
( id,
comments filler,
id2
)
data file:
1;aaa;2
3;abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz;4
5;bbb;6
I'm using the command sqlldr userid=xxx/yyy#zzz control=test.ctl and I'm getting all the rows without errors:
SQL> select * from test_sqlldr;
ID COMMENTS ID2
---------- -------------------- ----------
1 2
3 4
5 6
You may try another approach, I'm getting the same desired result with the following control file:
LOAD DATA
INFILE test.data
INTO TABLE test_sqlldr
APPEND
FIELDS TERMINATED BY ';'
TRAILING NULLCOLS
( id,
comments "substr(:comments,1,0)",
id2
)
Update following Romaintaz's comment: I looked into it again and managed to get the same error as you when the size of the column exceeded 255 characters. This is because the default datatype of SQL*Loader is char(255). If you have a column with more data you will have to specify the length. The following control file solved the problem for a column with 300 characters:
LOAD DATA
INFILE test.data
INTO TABLE test_sqlldr
APPEND
FIELDS TERMINATED BY ';'
TRAILING NULLCOLS
( id,
comments filler char(4000),
id2
)
Hope this Helps,
--
Vincent
Just to suggest a tiny improvement, you might try something like:
LOAD DATA
IN FILE test.data INTO TABLE test_sqlldr
APPEND
FIELDS TERMINATED BY ';'TRAILING NULLCOLS
(
id,
comments char(4000) "substr(:comments, 1, 200)",
id2)
Now you'll grab the first 200 characters (or any number you specify in it's place) of all comments - unless some of your input records have values for the comments field that exceed 4000 characters, in which they'll be rejected by loader with the 'exceeds max length' error noted earlier. But assuming that's rare or not the case, all the records will load with some of the comments truncated to 200 chars.
If you go over char(4000) you'll get a SQL Loader error - there's a limit to how far you can push the beast.