I would like to load data in Oracle via the sqlldr, however, it is always loading the data in another format.
So does my data from my file look like:
2018-11-27 13 Vienna 1 66.90 1
This is the result after having loaded the data:
27-Nov-17 1443443505 ienna 909510961 0.9 3377
All columns except the date column are wrong
This is my table structure:
BOOKINGDATE DATE
CUSTOMERID NUMBER(38,0)
LOCATIONID VARCHAR(255 BYTE)
NUMBEROFPARKINGTICKET NUMBER(38,0)
CHARGETICKET NUMBER(18,2)
DURATIONINMINUTES NUMBER(38)
This is my table definition in my file:
LOAD DATA
APPEND
INTO TABLE ROTH.PARKSCHEIN_ROTH
FIELDS TERMINATED BY '\t'
OPTIONALLY ENCLOSED BY '"'
(
BOOKINGDATE DATE 'YYYY-MM-DD',
CUSTOMERID INTEGER,
LOCATIONID CHAR(255),
NUMBEROFPARKINGTICKET INTEGER,
CHARGETICKET DECIMAL EXTERNAL,
DURATIONINMINUTES INTEGER
)
Can someone please tell me which datatypes do I have to use?
I thought Oracle is find all the types by itself except the date?
Thank you very much in advance for you help.
It's generally easiest to leave it to the default and let conversion happen in the database:
load data
replace
into table parkschein_roth
fields terminated by '\t'
optionally enclosed by '"'
( bookingdate date 'YYYY-MM-DD'
, customerid
, locationid
, numberofparkingticket
, chargeticket
, durationinminutes )
The log then shows it did this:
Column Name Position Len Term Encl Datatype
------------------------------ ---------- ----- ---- ---- ---------------------
BOOKINGDATE FIRST * WHT O(") DATE YYYY-MM-DD
CUSTOMERID NEXT * WHT O(") CHARACTER
LOCATIONID NEXT * WHT O(") CHARACTER
NUMBEROFPARKINGTICKET NEXT * WHT O(") CHARACTER
CHARGETICKET NEXT * WHT O(") CHARACTER
DURATIONINMINUTES NEXT * WHT O(") CHARACTER
Note that specifying a numeric datatype without the EXTERNAL keyword instructs SQL*Loader to read the binary data in the file directly, not its character representation: for example, what's displayed as 1 in a text editor is character 49 (that is, the symbol representing the bit sequence 00110001), and not an actual numeric 1. (I have never seen a data file formatted with binary encoded numbers but I suppose they must exist.) See Numeric EXTERNAL in the SQL*Loader Field List Reference.
Note following comments: it seems the line actually ended with 00110001 (the character '1') followed by 00001101 (Windows carriage return) before the linefeed. Looking at the result you got, it must have read those two bytes and interpreted them as 0000110100110001 to get decimal 3377.
I'm entering values from a CSV file to an Oracle table using a SQL*Loader script. In this table there are fields with NOT NULL constraints. In my CSV file the corresponding field is "" and I would like put a blank string into the Oracle table when that happens.
This is my control file:
LOAD DATA
infile 'F:\tar.csv'
REPLACE
INTO TABLE tar
fields terminated by ',' optionally enclosed by '"'
TRAILING NULLCOLS
(IDTAR ,
DATABACKUP DATE "YYYY-MM-DD",
PAESE ,
R_ELEM NULLIF (R_ELEM=BLANKS))
and this is the error in the log file:
ORA-01400: cannot insert NULL into ("MY_SCHEMA"."TAR"."PAESE")
How can I avoid the error by supplying a different value?
You can apply an SQL operator, such as NVL(:PAESE, 'XXX'). Notice the colon before the reference to the field name. In situ:
LOAD DATA
infile 'gian.csv'
REPLACE
INTO TABLE tar
fields terminated by ',' optionally enclosed by '"'
TRAILING NULLCOLS
(
IDTAR,
DATABACKUP DATE "YYYY-MM-DD",
PAESE "NVL(:PAESE, 'XXX')",
R_ELEM NULLIF (R_ELEM=BLANKS)
)
With a dummy table:
create table tar (
idtar number,
databackup date,
paese varchar2(10) not null,
r_elem varchar2(10)
);
and CSV, where the 3rd and 4th lines have trailing spaces for the nullif() clause:
1,2017-08-01,A,B
2,2017-08-02,C,
3,2017-08-03,,
4,2017-08-04,"",
then running with that control file gets:
SQL*Loader: Release 11.2.0.4.0 - Production on Fri Aug 4 19:39:23 2017
Copyright (c) 1982, 2011, Oracle and/or its affiliates. All rights reserved.
Commit point reached - logical record count 4
and the log says:
...
Column Name Position Len Term Encl Datatype
------------------------------ ---------- ----- ---- ---- ---------------------
IDTAR FIRST * , O(") CHARACTER
DATABACKUP NEXT * , O(") DATE YYYY-MM-DD
PAESE NEXT * , O(") CHARACTER
SQL string for column : "NVL(:PAESE, 'XXX')"
R_ELEM NEXT * , O(") CHARACTER
NULL if R_ELEM = BLANKS
Table TAR:
4 Rows successfully loaded.
0 Rows not loaded due to data errors.
0 Rows not loaded because all WHEN clauses were failed.
0 Rows not loaded because all fields were null.
...
Querying the table shows all four rows were loaded:
set null "<null>"
select * from tar;
IDTAR DATABACKU PAESE R_ELEM
---------- --------- ---------- ----------
1 01-AUG-17 A B
2 02-AUG-17 C <null>
3 03-AUG-17 XXX <null>
4 04-AUG-17 XXX <null>
Obviously replace 'XXX' with the actual default string you want to use. You said 'a blank string', so you could use "NVL(:PAESE, ' ')" to insert a single space character for instance. You can't use an empty string though, as that is the same as null as far as Oracle is concerned.
I have an external table which reads from a CSV file and is failing on certain rows.
External table definition:
E_ID NUMBER
A_IND VARCHAR2 (3 Byte)
B_IND VARCHAR2 (3 Byte)
E_DATE DATE
E_AMT NUMBER
F_DATE DATE
D_E_DATE DATE
I see the following info from a log file generated when I select * from the external table.
KUP-05004: Warning: Intra source concurrency disabled because parallel select was not requested.
Field Definitions for table EXTERNAL_TABLE_XTL
Record format DELIMITED BY NEWLINE
Data in file has same endianness as the platform
Rows with all null fields are accepted
Fields in Data Source:
E_ID CHAR (255)
Terminated by ","
Enclosed by """ and """
Trim whitespace same as SQL Loader
A_IND CHAR (255)
Terminated by ","
Enclosed by """ and """
Trim whitespace same as SQL Loader
B_IND CHAR (255)
Terminated by ","
Enclosed by """ and """
Trim whitespace same as SQL Loader
E_DATE CHAR (10)
Date datatype DATE, date mask MM/DD/YYYY
Terminated by ","
Enclosed by """ and """
Trim whitespace same as SQL Loader
E_AMT CHAR (255)
Terminated by ","
Enclosed by """ and """
Trim whitespace same as SQL Loader
F_DATE CHAR (10)
Date datatype DATE, date mask MM/DD/YYYY
Terminated by ","
Enclosed by """ and """
Trim whitespace same as SQL Loader
D_E_DATE CHAR (10)
Date datatype DATE, date mask MM/DD/YYYY
Terminated by ","
Enclosed by """ and """
Trim whitespace same as SQL Loader
KUP-04021: field formatting error for field D_E_DATE
KUP-04026: field too long for datatype
KUP-04101: record 56 rejected in file /home/TEST.csv
KUP-04021: field formatting error for field D_E_DATE
KUP-04026: field too long for datatype
KUP-04101: record 61 rejected in file /home/TEST.csv
KUP-04021: field formatting error for field D_E_DATE
KUP-04026: field too long for datatype
KUP-04101: record 70 rejected in file /home/TEST.csv
The file was transferred to the server via FileZilla. From reading other posts I thought maybe it was because the file was transferred in binary mode (it was originally on Auto setting) and maybe some non-printed characters have came in. So I tried to transfer using ASCII setting but that did not work. Then I tried to delete one of the lines that caused an error and retype it in manually. That did not work either.
Failed sample data:
5560000,N,Y,,24950,10/12/2011,10/27/2011
5550001,Y,Y,11/26/2013,73813,11/18/2013,11/29/2013
5560002,Y,Y,11/6/2015,22041.28,11/6/2015,11/18/2015
5560003,Y,Y,10/10/2012,2768.66,10/10/2012,10/24/2012
5560004,N,Y,,29750,9/30/2013,10/15/2013
5560005,Y,Y,10/8/2015,76474.84,10/8/2015,10/21/2015
5560006,N,Y,,63879.28,11/16/2011,11/30/2011
5560007,N,Y,,100000,11/14/2013,11/21/2013
Successful sample data:
5560008,Y,N,11/1/2010,,,
5550009,Y,N,,,,
5550010,N,N,,,,
5550011,N,N,,,,
5560012,Y,Y,2/12/2016,50000,2/12/2016,2/23/2016
5560013,Y,N,7/22/2011,,,
My first assumption is for some reason double digit months are not being accepted for the field D_E_DATE. Please note this is successful in the dev environment but not production and both are the same database version.
The following is working fine for me.
Table Definition:
CREATE TABLE my_data (
E_ID NUMBER,
A_IND VARCHAR2 (3 Byte),
B_IND VARCHAR2 (3 Byte),
E_DATE DATE,
E_AMT NUMBER,
F_DATE DATE,
D_E_DATE DATE
)
ORGANIZATION EXTERNAL (
TYPE ORACLE_LOADER
DEFAULT DIRECTORY MY_DIR
ACCESS PARAMETERS (
RECORDS DELIMITED BY NEWLINE
FIELDS TERMINATED BY ','
MISSING FIELD VALUES ARE NULL
(
E_ID,
A_IND,
B_IND,
E_DATE date 'MM/DD/YYYY',
E_AMT,
F_DATE date 'MM/DD/YYYY',
D_E_DATE date 'MM/DD/YYYY'
)
)
LOCATION ('data.txt')
);
Sample Data:
[oracle#ora12c Desktop]$ cat data.txt
5560000,N,Y,,24950,10/12/2011,10/27/2011
5550001,Y,Y,11/26/2013,73813,11/18/2013,11/29/2013
5560002,Y,Y,11/6/2015,22041.28,11/6/2015,11/18/2015
5560003,Y,Y,10/10/2012,2768.66,10/10/2012,10/24/2012
5560004,N,Y,,29750,9/30/2013,10/15/2013
5560005,Y,Y,10/8/2015,76474.84,10/8/2015,10/21/2015
5560006,N,Y,,63879.28,11/16/2011,11/30/2011
5560007,N,Y,,100000,11/14/2013,11/21/2013
Output:
SQL> select * from my_date;
E_ID A_I B_I E_DATE E_AMT F_DATE D_E_DATE
---------- --- --- --------- ---------- --------- ---------
5560000 N Y 24950 12-OCT-11 27-OCT-11
5550001 Y Y 26-NOV-13 73813 18-NOV-13 29-NOV-13
5560002 Y Y 06-NOV-15 22041.28 06-NOV-15 18-NOV-15
5560003 Y Y 10-OCT-12 2768.66 10-OCT-12 24-OCT-12
5560004 N Y 29750 30-SEP-13 15-OCT-13
5560005 Y Y 08-OCT-15 76474.84 08-OCT-15 21-OCT-15
5560006 N Y 63879.28 16-NOV-11 30-NOV-11
5560007 N Y 100000 14-NOV-13 21-NOV-13
8 rows selected.
The answer to this question was found in the following thread:
Oracle external table date field - works in one DB and not in another
Transferring the same file from the dev server to prod server seemed to have resolved the issue. Weird, I wish I knew better exactly why this issue occurred and how to resolve it.
LOAD DATA
INFILE 'file.txt'
TRUNCATE
INTO TABLE SOURCE_TABLE
FIELDS TERMINATED BY '|'
TRAILING NULLCOLS
(
merchant_no INTEGER EXTERNAL(7),
MCHTYP char(1),
ACC_BPAC_TYP char(3),
ACC_DT_EFF DATE "DD/MM/YYYY",
DT_EFF DATE "DD/MM/YYYY",
ACC_SORT_CD INTEGER EXTERNAL(9),
ACC_ACCTNO INTEGER EXTERNAL(9),
DT_IF DATE "DD/MM/YYYY",
PURGE_IND char(1)
)
The problem is, Sql Loader is terminated on 'ACC_SORT_CD'.
I am facing the problem in every column after specifying DATE datatype.
It is not able to read the column which is coming after DATE datatype.
Insert option in effect for this table: TRUNCATE
TRAILING NULLCOLS option in effect
Column Name Position Len Term Encl Datatype
------------------------------ ---------- ----- ---- ---- ---------------------
MCHNO FIRST 7 CHARACTER
Terminator string : '|'
MCHTYP NEXT 1 CHARACTER
Terminator string : '|'
ACC_BPAC_TYP NEXT 3 CHARACTER
Terminator string : '|'
ACC_DT_EFF NEXT * DATE DD/MM/YYYY
Terminator string : '|'
DT_EFF NEXT * DATE DD/MM/YYYY
Terminator string : '|'
ACC_SORT_CD NEXT 9 CHARACTER
Terminator string : '|'
ACC_ACCTNO NEXT 9 CHARACTER
Terminator string : '|'
DT_IF NEXT * DATE DD/MM/YYYY
Terminator string : '|'
PURGE_IND NEXT 1 CHARACTER
Terminator string : '|'
SQL*Loader-466: Column ACC_SORT_CD does not exist in table SOURCE_TABLE.
I have tried with another file as well but same problem is present.
Please let me know what I am missing?
I have a SQL Loader Control file,
LOAD DATA
INFILE 'test.txt'
INTO TABLE TEST replace
fields terminated "|" optionally enclosed by '"' TRAILING NULLCOLS
( DOCUMENTID INTEGER(10),
CUSTID INTEGER(10),
USERID INTEGER(10),
FILENAME VARCHAR(255),
LABEL VARCHAR(50),
DESCRIPTION VARCHAR(2000),
POSTDATE DATE "YYYY-MM-DD HH24:MI:SS" NULLIF POSTDATE="",
USERFILENAME VARCHAR(50),
STORAGEPATH VARCHAR(255)
)
and it's giving me an error when I run SQL Loader on it,
Record 1: Rejected - Error on table TEST, column FILENAME.
Variable length field exceeds maximum length.
Here's that row.. the length of that column is way under 255..
1|5001572|2|/Storage/Test/5001572/test.pdf|test.pdf||2005-01-13 11:47:49||
And here's an oddity I noticed within the log file
Column Name | Position | Len | Term | Encl | Datatype
FILENAME | NEXT | 257 | | | VARCHAR
I define the length as 255 in both my table and control file. Yet the log spits it out as 257? I've tried knocking down the length in the control file to 253, so it appears as 255 in the log file, but the same issue.
Any help? This has bugged me for two days now.
Thanks.
Don't define your data fields as VARCHAR2 and INTEGER. Use CHAR. Most of the time, when loading data from a text file, you want to use CHAR, or perhaps DATE, although even that is converted from a text form. Most of the time you don't even need a length specifier. The default length for a CHAR field is 255. Your control file should look something like:
LOAD DATA
INFILE "test.txt"
INTO TABLE TEST replace
fields terminated "|" optionally enclosed by '"' TRAILING NULLCOLS
(
DOCUMENTID,
CUSTID,
USERID ,
FILENAME,
LABEL,
DESCRIPTION CHAR(2000),
POSTDATE DATE "YYYY-MM-DD HH24:MI:SS" NULLIF POSTDATE=BLANKS,
USERFILENAME,
STORAGEPATH
)
+1 for DCookie, but to expand on that it's important to distinguish between data types as specified in a table and data types in a SQL*loader control file as they mean rather different things, confusingly.
Start with a look at the the documentation, and note that when loading regular text files you need to be using the "portable" data types.
Varchar is a "non-portable" type, in which:
... consists of a binary length subfield followed by a character string of the specified length
So as DCookie says, CHAR is the thing to go for, and INTEGER EXTERNAL is a very commonly used SQL*Loader data type which you'd probably want to specify for DOCUMENTID etc.