I am trying to load a tsv file with variable length and blanks but I cannot load it.
My TSV file has:
code name information surname
1234 Peter Peter
1111 Carl exampleexample example Jhon
I'm trying with:
OPTIONS (SKIP=1)
LOAD DATA
INFILE 'EXAMPLE.TSV'
INTO TABLE PERSON
FIELDS TERMINATED BY ','
OPTIONALLY ENCLOSED BY '"'
(
CODE,
NAME,
INFORMATION,
SURNAME
)
Here's an example.
SQL> desc person
Name Null? Type
----------------------------------------- -------- ----------------------------
CODE NUMBER
NAME VARCHAR2(10)
INFORMATION VARCHAR2(30)
SURNAME VARCHAR2(10)
SQL>
Control file; note fields terminated by x'09'. I included sample data into the control file itself, for simplicity.
options (skip=1)
load data
infile *
replace
into table person
fields terminated by x'09'
(
code,
name,
information,
surname
)
begindata
code name information surname
1234 Peter Peter
1111 Carl example exa John
Testing:
SQL> $sqlldr scott/tiger control=test18.ctl log=test18.log
SQL*Loader: Release 11.2.0.2.0 - Production on Sri Tra 29 20:26:06 2020
Copyright (c) 1982, 2009, Oracle and/or its affiliates. All rights reserved.
Commit point reached - logical record count 2
SQL> select * From person;
CODE NAME INFORMATION SURNAME
---------- ---------- ------------------------------ ----------
1234 Peter Peter
1111 Carl example exa John
Related
I am trying to load a '|' delimited file, but it fails because some columns have crlf values.
I converted the text files to xlsx and imported them successfully using SQL Developer.
I noticed that SQL Developer using a Line Terminator option set to "standard: CR LF, CR or LF".
I suspect that I need to set that in my ctl file, but have been unable to find the correct syntax.
Any assistance would be appreciated.
Here is a screenshot from SQL Developer:
As usual, it helps if you post what you have (in this case, control file, table description and sample data). Without it, we have to guess and that doesn't have to reflect reality.
Sample table:
SQL> create table test
2 (id number,
3 description varchar2(50));
Table created.
Control file (contains sample data):
load data
infile *
replace
continueif next preserve(1:1) != "|"
into table test
fields terminated by '|'
trailing nullcols
(
dummy filler,
id,
description
)
begindata
|1|this is some text
with no meaning
at all
|2|some
more text
Loading session:
SQL> $sqlldr scott/tiger#pdb1 control=test11.ctl log=test11.log
SQL*Loader: Release 21.0.0.0.0 - Production on Sat Nov 26 21:51:53 2022
Version 21.3.0.0.0
Copyright (c) 1982, 2021, Oracle and/or its affiliates. All rights reserved.
Path used: Conventional
Commit point reached - logical record count 1
Commit point reached - logical record count 2
Table TEST:
2 Rows successfully loaded.
Check the log file:
test11.log
for more information about the load.
Result:
SQL> select * from test;
ID DESCRIPTION
---------- --------------------------------------------------
1 this is some text with no meaning at all
2 some more text
SQL>
[EDIT] Using piece of code #p3consulting suggested: no difference (at least, not in my 21XE database):
Control file (modified line #2, the infile one):
load data
infile * "STR X'220D0A'"
replace
continueif next preserve(1:1) != "|"
into table test
fields terminated by '|'
trailing nullcols
(
dummy filler,
id,
description
)
begindata
|1|this is some text
with no meaning
at all
|2|some
more text
Loading session:
SQL> $sqlldr scott/tiger#pdb1 control=test11.ctl log=test11.log
SQL*Loader: Release 21.0.0.0.0 - Production on Sun Nov 27 12:34:04 2022
Version 21.3.0.0.0
Copyright (c) 1982, 2021, Oracle and/or its affiliates. All rights reserved.
SQL*Loader-283: file processing string "STR X'220D0A'" ignored for INFILE *
Path used: Conventional
Commit point reached - logical record count 1
Commit point reached - logical record count 2
Table TEST:
2 Rows successfully loaded.
Check the log file:
test11.log
for more information about the load.
Result: no difference:
SQL> select * from test;
ID DESCRIPTION
---------- --------------------------------------------------
1 this is some text with no meaning at all
2 some more text
SQL>
[EDIT #2: removed CONTINUEIF]
load data
infile * "STR X'220D0A'"
replace
into table test
<snip>
begindata
<snip>
Loading session:
SQL> $sqlldr scott/tiger#pdb1 control=test11.ctl log=test11.log
SQL*Loader-283: file processing string "STR X'220D0A'" ignored for INFILE *
<snip>
Ah! I should've read the message, Oracle says it all:
file processing string "STR X'220D0A'" ignored for INFILE
So: with a new control file:
load data
infile "c:\temp\test11.txt" "STR X'220D0A'"
replace
into table test
fields terminated by '|'
trailing nullcols
(
dummy filler,
id,
description
)
and data in a separate, test11.txt file:
|1|this is some text
with no meaning
at all
|2|some
more text
Loading session:
<snip>
Commit point reached - logical record count 1
Table TEST:
1 Row successfully loaded.
Check the log file:
test11.log
for more information about the load.
Result:
SQL> select * from test;
ID DESCRIPTION
---------- --------------------------------------------------
1 this is some text
with no meaning
at all
SQL>
That looks A LOT better now!
Though, row#2 is missing. Log file doesn't say anything about it:
SQL> $type test11.log
SQL*Loader: Release 21.0.0.0.0 - Production on Sun Nov 27 15:30:39 2022
Version 21.3.0.0.0
Copyright (c) 1982, 2021, Oracle and/or its affiliates. All rights reserved.
Control File: test11.ctl
Data File: c:\temp\test11.txt
File processing option string: "STR X'220D0A'"
Bad File: test11.bad
Discard File: none specified
(Allow all discards)
Number to load: ALL
Number to skip: 0
Errors allowed: 50
Bind array: 250 rows, maximum of 1048576 bytes
Continuation: none specified
Path used: Conventional
Table TEST, loaded from every logical record.
Insert option in effect for this table: REPLACE
TRAILING NULLCOLS option in effect
Column Name Position Len Term Encl Datatype
------------------------------ ---------- ----- ---- ---- ---------------------
DUMMY FIRST * | CHARACTER
(FILLER FIELD)
ID NEXT * | CHARACTER
DESCRIPTION NEXT * | CHARACTER
Table TEST:
1 Row successfully loaded.
0 Rows not loaded due to data errors.
0 Rows not loaded because all WHEN clauses were failed.
0 Rows not loaded because all fields were null.
Space allocated for bind array: 129000 bytes(250 rows)
Read buffer bytes: 1048576
Total logical records skipped: 0
Total logical records read: 1
Total logical records rejected: 0
Total logical records discarded: 0
Run began on Sun Nov 27 15:30:39 2022
Run ended on Sun Nov 27 15:30:39 2022
Elapsed time was: 00:00:00.17
CPU time was: 00:00:00.11
SQL>
No bad nor discard file:
SQL> $type test11.bad
The system cannot find the file specified.
SQL> $del test11.dsc
Could Not Find c:\temp\test11.dsc
SQL>
The input file looks like this (when all characters are displayed):
[FINALLY]
p3consulting: 22 is not the hex for | ... it's 7C... but you need to finish the record by |CRLF... your rows end with only CRLF.
Bingo!
Control file:
load data
infile "c:\temp\test11.txt" "STR X'7C0D0A'"
replace
into table test
fields terminated by '|'
trailing nullcols
(
dummy filler,
id,
description
)
Test11.txt:
|1|this is some text
with no meaning
at all|
|2|some
more text|
Loading:
SQL> $sqlldr scott/tiger#pdb1 control=test11.ctl log=test11.log
SQL*Loader: Release 21.0.0.0.0 - Production on Sun Nov 27 17:11:21 2022
Version 21.3.0.0.0
Copyright (c) 1982, 2021, Oracle and/or its affiliates. All rights reserved.
Path used: Conventional
Commit point reached - logical record count 2
Table TEST:
2 Rows successfully loaded.
Check the log file:
test11.log
for more information about the load.
Result:
SQL> select * From test;
ID DESCRIPTION
---------- --------------------------------------------------
1 this is some text
with no meaning
at all
2 some
more text
SQL>
Try
INFILE 'target.dat' "STR X'220D0A'"
The terminator_string is specified as either 'char_string' or X'hex_string' where:
'char_string' is a string of characters enclosed in single or double quotation marks
X'hex_string' is a byte string in hexadecimal format that you can get with the cast_to_raw function.
So here X'220D0A' is "|CRLF" and this will keep the "CRLF" in the imported column.
using the control file below to load data
LOAD DATA
INFILE '/c/Transaction.txt'
INTO TABLE tab1 APPEND WHEN (1:1) = 'D'
(RUN_ID "RUN_ID_SEQ.NEXTVAL"
,RUN_DATE_TIME "SYSDATE"
)
INTO TABLE tab2 TRUNCATE WHEN(1:1) <> 'D'
FIELDS TERMINATED BY '|' TRAILING NULLCOLS
(
DEALER_NUMBER
,TRAN_CODE
,TRAN_AMOUNT "TO_NUMBER(:PL818_TRAN_AMOUNT,'999999.99')"
,TRAN_DATE "TO_DATE(:PL818_TRAN_DATE,'DD-MM-YYYY')"
)
This is the data set
DEALER_ID|TRAN_TYPE_CODE|TRAN_AMOUNT|TRAN_DATE
203113|34|1000.50|12-07-2022
No errors during load. Data loads correctly into the first table but first two characters are skipped when loading data into second table, table data looks like this. What could be causing this?
3113 34 1000.5 12-JUL-22
Skips "20"
Sample tables:
SQL> desc tab1
Name Null? Type
----------------------------------------- -------- ----------------------------
RUN_ID NUMBER
RUN_DATE_TIME DATE
SQL> desc tab2
Name Null? Type
----------------------------------------- -------- ----------------------------
DEALER_NUMBER NUMBER
TRAN_CODE NUMBER
TRAN_AMOUNT NUMBER
TRAN_DATE DATE
SQL>
Control file; note position(1) when loading into tab2. You must use it when loading data into different tables, using the same control file:
LOAD DATA
INFILE *
INTO TABLE tab1 APPEND WHEN (1:1) = 'D'
(RUN_ID "RUN_ID_SEQ.NEXTVAL"
,RUN_DATE_TIME "SYSDATE"
)
INTO TABLE tab2 TRUNCATE WHEN(1:1) <> 'D'
FIELDS TERMINATED BY '|' TRAILING NULLCOLS
(
DEALER_NUMBER position(1) --> here
,TRAN_CODE
,TRAN_AMOUNT "TO_NUMBER(:TRAN_AMOUNT,'999999.99')"
,TRAN_DATE "TO_DATE(:TRAN_DATE,'DD-MM-YYYY')"
)
BEGINDATA
203113|34|1000.50|12-07-2022
Loading session and the result:
SQL> $sqlldr scott/tiger#orcl control=test42.ctl log=test42.log
SQL*Loader: Release 18.0.0.0.0 - Production on Sri Srp 13 08:28:19 2022
Version 18.5.0.0.0
Copyright (c) 1982, 2018, Oracle and/or its affiliates. All rights reserved.
Path used: Conventional
Commit point reached - logical record count 1
Table TAB1:
0 Rows successfully loaded.
Table TAB2:
1 Row successfully loaded.
Check the log file:
test42.log
for more information about the load.
SQL> select * from tab2;
DEALER_NUMBER TRAN_CODE TRAN_AMOUNT TRAN_DATE
------------- ---------- ----------- ----------
203113 34 1000,5 12-07-2022
--
here's your missing "20"
SQL>
I want to insert the rejected records in a log and to the temp table that i created . but this is my problem on codes , i'm new in creating CTL and Shell Script .
here's my code :
LOAD DATA
INFILE 'open_account .csv'
BADFILE 'open_account.bad'
DISCARDFILE 'open_account.dsc'
APPEND
INTO TABLE accountrequest_temp
FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '"' | TRAILING NULLCOLS
(Branch,
Account_Type,
Title,
FirstName,
Lastname,
Birthday,
WorkPhone,
HomePhone,
Address,
State,
Zip,
Email,);
here's my error :
$ sqlldr hr/password, control=loaddata.ctl
SQL*Loader: Release 11.2.0.2.0 - Production on Tue Aug 17 11:20:06 2021
Copyright (c) 1982, 2009, Oracle and/or its affiliates. All rights reserved.
SQL*Loader-350: Syntax error at line 7.
Illegal combination of non-alphanumeric characters
FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '"' | TRAILING NULLCO
Quite a few errors. To mention some of them:
don't terminate column list with a semi-colon
that's why you got the "Illegal combination ..." error
remove pipe sign (|) in front of trailing nullcolls
remove space in infile file name
When fixed, it works. I re-used table accountRrequest (with a typo in its name) we created yesterday, just renamed it. That's why my column list differs from yours a little bit.
Control file:
LOAD DATA
INFILE 'open_account.csv'
BADFILE 'open_account.bad'
DISCARDFILE 'open_account.dsc'
APPEND
INTO TABLE accountrequest_temp
FIELDS TERMINATED BY ','
OPTIONALLY ENCLOSED BY '"'
TRAILING NULLCOLS
(
requestid,
Branch,
Account_Type,
Title,
FirstName,
Lastname,
Birthday,
WorkPhone,
HomePhone,
Address,
State,
Zip,
Email,
status
)
Sample data (in the open_account.csv file):
1,Brnch, type1,Mr.,Little,Foot,,123456,543123,Ilica 20,Croatia,10000,little#foot.com,Entered
Loading session:
SQL> $sqlldr scott/tiger#kc11gt control=test39.ctl log=test39.log
SQL*Loader: Release 11.2.0.1.0 - Production on Uto Kol 17 07:18:34 2021
Copyright (c) 1982, 2009, Oracle and/or its affiliates. All rights reserved.
Commit point reached - logical record count 1
Result:
SQL> select * from accountrequest_temp;
REQUESTID BRANCH ACCOUNT_TYPE TITL FIRSTNAME LASTNAME BIRTHDAY
---------- --------------- --------------- ---- --------------- --------------- -------------------
WORKPHONE HOMEPHONE ADDRESS STATE ZIP
---------- ---------- ------------------------------ --------------- ----------
EMAIL STATUS
------------------------------ ----------
1 Brnch type1 Mr. Little Foot
123456 543123 Ilica 20 Croatia 10000
little#foot.com Entered
SQL>
My raw file like below
bbbb 1000 Open 1000 Dep 12/03/2010
dddd 1001 Open 2000 Loan 13/01/2019
eeee 1003 Dor 3000 Dep 11/04/1965
Sqlldr control file
load data
infile *
truncate into table Mytab
fields
trailing nullcols
(
Name position (1:4),
Acc position(6:9),
Status Position(11:14),
Amt position(15:18),
type position(20:23),
Date position(24:33)
)
begin data
I need date column with hyphen(-) symbol not for slash(/) symbol.
12/03/2010 -- 12-03-2010
Date position(24:33) ---> What I have to mention here
It would help if you also provided mytab description. Because, "Date" column (whose name is invalid; it is reserved for datatype name. I renamed it to "Datum") should be of date datatype, while it seems you use varchar2. In either case, you'd use appropriate function, enclosed into double quotes. I used replace as it does what you asked for.
Here's an example:
Control file (yours is invalid as it doesn't follow data format; I fixed it):
load data
infile *
truncate into table Mytab
fields
trailing nullcols
(Name position(1:4),
Acc position(6:9),
Status position(11:14),
Amt position(16:19),
type position(21:24),
Datum position(26:35) "replace(:datum, '/', '-')"
)
begindata
bbbb 1000 Open 1000 Dep 12/03/2010
dddd 1001 Open 2000 Loan 13/01/2019
eeee 1003 Dor 3000 Dep 11/04/1965
Loading session & result:
SQL> $sqlldr scott/tiger#xe control=test34.ctl log=test34.log
SQL*Loader: Release 11.2.0.1.0 - Production on Sri O×u 24 07:54:46 2021
Copyright (c) 1982, 2009, Oracle and/or its affiliates. All rights reserved.
Commit point reached - logical record count 3
SQL> select * from mytab;
NAME ACC STATUS AMT TYPE DATUM
---------- ---------- ---------- ---------- ---------- ----------
bbbb 1000 Open 1000 Dep 12-03-2010
dddd 1001 Open 2000 Loan 13-01-2019
eeee 1003 Dor 3000 Dep 11-04-1965
SQL>
I am loading CSV file data into table called EMPLOYEE using SQL*Loader.
My CSV file data is separated by pipe (|):
EMPID|EMPNAME_ADDRESS|SALARY|GRADE
123|Rams Hyd|1000|A1
124|Sand MUM|2,000|A2
125|"PRASANNA qwer trasf\"501 advv vvd, qee ggg\trfe \411005 THE|3,00,000|A3
and my control file is:
LOAD DATA
Insert INTO TABLE EMPLOYEE
Fields terminated by "|" Optionally enclosed by '"' TRAILING NULLCOLS
(
EMPID,
EMPNAME,
SALARY,
GRADE
)
When I load the data using above control file, the first two record are loading fine and for the third record i am getting error like as mentioned below.
no terminator found after TERMINATED and ENCLOSED field
Please suggest changes to be done to load the data properly.
Here's a way to load such data.
First, a target table (I hope it makes sense; you should have provided it):
SQL> create table employee
2 (empid number,
3 empname varchar2(200),
4 salary number,
5 grade varchar2(10)
6 );
Table created.
SQL>
Control file: note the SKIP option (which skips the header line), as well as REPLACE function call for the SALARY column (to remove superfluous commas which are formatting stuff; add them later, in the presentation layer).
options (skip = 1)
load data
infile *
replace
into table employee
fields terminated by "|" TRAILING NULLCOLS
(
empid,
empname,
salary "replace(:salary, ',', null)",
grade)
begindata
EMPID|EMPNAME_ADDRESS|SALARY|GRADE
123|Rams Hyd|1000|A1
124|Sand MUM|2,000|A2
125|"PRASANNA qwer trasf\"501 advv vvd, qee ggg\trfe \411005 THE|3,00,000|A3
Loading session:
SQL> $sqlldr scott/tiger control=test06.ctl log=test06.log
SQL*Loader: Release 11.2.0.2.0 - Production on Uto Tra 9 19:37:46 2019
Copyright (c) 1982, 2009, Oracle and/or its affiliates. All rights reserved.
Commit point reached - logical record count 2
Commit point reached - logical record count 3
SQL> select * From employee;
EMPID EMPNAME SALARY GRADE
---------- ---------------------------------------- ---------- ----------
123 Rams Hyd 1000 A1
124 Sand MUM 2000 A2
125 "PRASANNA qwer trasf\"501 advv vvd, qee 300000 A3
ggg\trfe \411005 THE
SQL>