I have a weird problem with sqloader..In my database I created a sequence like this:
CREATE SEQUENCE TEST_ID_SEQ
MINVALUE 1
START WITH 1
INCREMENT BY 1
CACHE 20;
And my control-file looks like this:
load data
INFILE 'C:\Users\\test.csv'
into table TABLE_NAME
append
fields terminated by ','
OPTIONALLY ENCLOSED BY '"' AND '"'
trailing nullcols
(
TEST_COLUMN_ID "TEST_ID_SEQ.NEXTVAL",
INSERT_DATE EXPRESSION "current_timestamp(3)",
COLUMN_1 CHAR(4000),
COLUMN_2 CHAR(4000),
......
)
So during the import I just want to insert a generated number in the column "TEST_COLUMN_ID".
The problem now is that if i write this line in my control-file:
TEST_COLUMN_ID "TEST_ID_SEQ.NEXTVAL",
then the wother values wont be imported correctly. Which means, all the values will be shifted to the right. For example the content of the value in COLUMN_1 will be imported in COLUMN_2 and so on..when i delete the line than it works correctly.....can someone help me?
ok the solution is to put the SEQ_NUMBER field as the last field in the control
file. If someone is interested he can see this link for more details: http://www.mathcs.emory.edu/~cheung/Courses/377/Oracle/SQL-loader/FAQ.html
Related
I'm trying to load a table with oracle sqlldr but get ORA-19032: Expected XML tag, got no content on (allowed!) null fields.
If the table is populated by inserts, or the column is updated to null, everthing is fine. But loading via sqlldr doesn't work.
Loader call:
sqlldr $DBCS control=$TABLE.ctl data=$TABLE.csv bad=$LOGDIR/$TABLE.bad log=$LOGDIR/$TABLE.log rows=10000 bindsize=20000000 readsize=20000000 silent=header,feedback
CTL-File:
LOAD DATA
INFILE MY_TABLE.csv "STR '|\n'"
INTO TABLE MY_TABLE APPEND
FIELDS TERMINATED BY ';'
OPTIONALLY ENCLOSED BY '"'
TRAILING NULLCOLS
(
"NORMAL_COLUMN1" CHAR(40)
, "NORMAL_COLUMN2" CHAR(250)
, "NORMAL_COLUMN3" CHAR(250)
, "XML_COLUMN" CHAR(16000)
)
Table Definition:
CREATE TABLE MY_TABLE
(
NORMAL_COLUMN1 NUMBER(14) not null,
NORMAL_COLUMN2 VARCHAR2(250) not null,
NORMAL_COLUMN3 VARCHAR2(250),
XML_COLUMN SYS.XMLTYPE
);
csv line - not working:
21001;"lulul";"lalal";|
csv line - working:
21001;"lulul";;"<a>ala</a>"|
Note: In general, i'd create "correct" datatypes for the "normal columns", but the ctl-file is created by an unload-script...and it works ;)
Important: It works for non-null values for the xml column.
I hope there is a way to modify the corresponding line in the ctl-File, but i didn't find a solution yet.
The issue appears to be that at some point within that loader execution something similar to the following is occurring where the NULL value is being passed as a parameter to the XmlType(value) constructor or the createXML(value) function:
select XmlType('') as xml_value from dual;
I have tried a few ways to work around it with no success. I can continue to research it when I have more time, but I wanted to at least post this as an explanation. You could (hackily) work around this by doing something like:
LOAD DATA
INFILE MY_TABLE.csv "STR '|\n'"
INTO TABLE MY_TABLE APPEND
FIELDS TERMINATED BY ';'
OPTIONALLY ENCLOSED BY '"'
TRAILING NULLCOLS
(
"NORMAL_COLUMN1" CHAR(40)
, "NORMAL_COLUMN2" CHAR(250)
, "NORMAL_COLUMN3" CHAR(250)
, "Nvl(XML_COLUMN, '<empty />')" CHAR(16000)
)
Not the ideal solution, but it should get you moving forward. Let me know if you end up just going this route, otherwise I will see if I can find something better.
Use nval function with XMLTYPE to provide dummy xml in case of null value.
EXTRACTVALUE(XMLTYPE(nvl(XML_COLUMN, '1')), '/XPATh/')
I'm trying to load data from a Datafile in different tables, I read a lot about field declaration and delimitation(Position(n:n), terminated by ). The point is than I'm not sure how to do what I need to do. Let me explain this with an example.
I have two tables (person, phone):
person_table( person_id_pk, person_name) - phone_table(person_id_pk, phone)
I have a datafile with:
$ datafile.txt
1,jack pierson,+13526985442
2,Katherine McLaren,+15264586548
My point is, when I'm declaring my ConfigFile.ctl, how do I specify than the field number 3 (phone field) should be insert or append into "phone_table", and the others two fields (person_id, person_name) should be insert or append into "person_table"
Considering than the fields are not fixed length, my reference is the field position. (Field datafile position)
I was thinking to try something like
$configfile.ctl
LOAD DATA
INFILE datafile.txt
APPEND
INTO TABLE person_table
(
person_id_pk POSITION (*) INTEGER EXTERNAL TERMINATED BY "," ,
person_name POSITION(*+1) CHAR(30) TERMINATED BY ","
)
INTO TABLE phone_table
(
person_id_fk POSITION (*) INTEGER EXTERNAL TERMINATED BY ","
phone ------> Right here is my point, how can I specify to SQL Loader than here
should be the field number 3 from datafile
)
I hope you guys get my point. it is a HUGE issue for me, because i'm dealing with CSV files which contains 60, 80, even 100 fields (columns based on Excel File). And every fields or group of fields could be in different tables.
I really appreciate the guide and help you could grant me. I'm probably wrong about my example and controlfile declarations, I haven't implemented anything yet. So I'm open to every suggest you could give me.
Your control file should look like this. The second "INTO TABLE" Uses POSITION(1) to move the logical "pointer" back to the start of the current line so it can be read again. then the name is skipped by defining it as a FILLER.
LOAD DATA
INFILE datafile.txt
APPEND
INTO TABLE person_table
FIELDS TERMINATED BY "," TRAILING NULLCOLS
(
person_id_pk INTEGER EXTERNAL,
person_name CHAR(30)
)
INTO TABLE phone_table
FIELDS TERMINATED BY "," TRAILING NULLCOLS
(
person_id_fk POSITION(1) INTEGER EXTERNAL,
x_name FILLER,
phone CHAR(12)
)
I'm loading data into my table through SQL Loader
data loading is successful but i''m getting garbage(repetitive) value in a particular column for all rows
After inserting :
column TERM_AGREEMENT is getting value '806158336' for every record
My csv file contains atmost 3 digit data for that column,but i'm forced to set my column definition to Number(10).
LOAD DATA
infile '/ipoapplication/utl_file/LBR_HE_Mar16.csv'
REPLACE
INTO TABLE LOAN_BALANCE_MASTER_INT
fields terminated by ',' optionally enclosed by '"'
(
ACCOUNT_NO,
CUSTOMER_NAME,
LIMIT,
REGION,
**TERM_AGREEMENT INTEGER**
)
create table LOAN_BALANCE_MASTER_INT
(
ACCOUNT_NO NUMBER(30),
CUSTOMER_NAME VARCHAR2(70),
LIMIT NUMBER(30),
PRODUCT_DESC VARCHAR2(30),
SUBPRODUCT_CODE NUMBER,
ARREARS_INT NUMBER(20,2),
IRREGULARITY NUMBER(20,2),
PRINCIPLE_IRREGULARITY NUMBER(20,2),
**TERM_AGREEMENT NUMBER(10)**
)
INTEGER is for binary data type. If you're importing a csv file, I suppose the numbers are stored as plain text, so you should use INTEGER EXTERNAL. The EXTERNAL clause specifies character data that represents a number.
Edit:
The issue seems to be the termination character of the file. You should be able to solve this issue by editing the INFILE line this way:
INFILE'/ipoapplication/utl_file/LBR_HE_Mar16.csv' "STR X'5E204D'"
Where '5E204D' is the hexadecimal for '^ M'. To get the hexadecimal value you can use the following query:
SELECT utl_raw.cast_to_raw ('^ M') AS hexadecimal FROM dual;
Hope this helps.
I actually solved this issue on my own.
Firstly, thanks to #Gary_W AND #Alessandro for their inputs.Really appreciate your help guys,learned some new things in the process.
Here's the new fragment which worked and i got the correct data for the last column
LOAD DATA
infile '/ipoapplication/utl_file/LBR_HE_Mar16.csv'
REPLACE
INTO TABLE LOAN_BALANCE_MASTER_INT
fields terminated by ',' optionally enclosed by '"'
(
ACCOUNT_NO,
CUSTOMER_NAME,
LIMIT,
REGION,
**TERM_AGREEMENT INTEGER Terminated by Whitspace**
)
'Terminated by whitespace' - I went through some threads of SQL Loader and i used 'terminated by whitespace' in the last column of his ctl file. it worked ,this time i didn't even had to use 'INTEGER' or 'EXTERNAL' or EXPRESSION '..' for conversion.
Just one thing, now can you guys let me now what could possibly be creating issue ?what was there in my csv file in that column and how by adding this thing solved the issue ?
Thanks.
As mentioned in the title, i wish to have a control file to handle this case. The scenario is i have to insert record into different table. For example, WHEN (1:3) is HEA, it need to Append into table header. WHEN (1:3) is DTL it need replace into table Detail. is that possible to do this?
I have a situation where data from one file goes to three tables depending on the first field in the file. The WHEN clause looks at the first field and takes action based on that. Notice that when a 'WHEN' is met, the first field is then skipped by declaring it a filler. To answer your question, I believe you can put the APPEND or REPLACE after the INTO TABLE clause. Give it a try and let us know.
OPTIONS (DIRECT=TRUE)
UNRECOVERABLE
LOAD DATA
APPEND
INTO TABLE TABLE_A
WHEN (01) = 'CLM'
FIELDS TERMINATED BY '|' TRAILING NULLCOLS
( rec_skip filler POSITION(1)
,CLM_CLAIM_ID CHAR NULLIF(CLM_CLAIM_ID=BLANKS)
...
)
INTO TABLE TABLE_B
WHEN (01) = 'SLN'
FIELDS TERMINATED BY '|' TRAILING NULLCOLS
( rec_skip filler POSITION(1)
,SL_CLAIM_ID CHAR NULLIF(SL_CLAIM_ID=BLANKS)
...
)
INTO TABLE TABLE_C
WHEN (01) = 'COB'
FIELDS TERMINATED BY '|' TRAILING NULLCOLS
( rec_skip filler POSITION(1)
,COB_CLAIM
...
)
More info: http://docs.oracle.com/cd/B28359_01/server.111/b28319/ldr_control_file.htm#i1005657
In order to load data (from a CSV file) into an Oracle database, I use SQL*Loader.
In the table that receives these data, there is a varchar2(500) column, called COMMENTS.
For some reasons, I want to ignore this information from the CSV file.
Thus, I wrote this control file:
Options (BindSize=10000000,Readsize=10000000,Rows=5000,Errors=100)
Load Data
Infile 'XXX.txt'
Append into table T_XXX
Fields Terminated By ';'
TRAILING NULLCOLS
(
...
COMMENTS FILLER,
...
)
This code seems to work correctly, as the COMMENTS field in database is always set to null.
However, if in my CSV file I have a record where the corresponding COMMENTS field exceeds the 500 characters limit, I get an error from SQL*Loader:
Record 2: Rejected - Error on table T_XXX, column COMMENTS.
Field in data file exceeds maximum length
Is there a way to really exclude the processing of my COMMENTS fields?
I can't reproduce your problem. I'm using Oracle 10.2.0.3.0 with SQL*Loader 10.2.0.1.
Here is my test case:
SQL> CREATE TABLE test_sqlldr (
2 ID NUMBER,
3 comments VARCHAR2(20),
4 id2 NUMBER
5 );
Table created
Control file:
LOAD DATA
INFILE test.data
INTO TABLE test_sqlldr
APPEND
FIELDS TERMINATED BY ';'
TRAILING NULLCOLS
( id,
comments filler,
id2
)
data file:
1;aaa;2
3;abcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyzabcdefghijklmnopqrstuvwxyz;4
5;bbb;6
I'm using the command sqlldr userid=xxx/yyy#zzz control=test.ctl and I'm getting all the rows without errors:
SQL> select * from test_sqlldr;
ID COMMENTS ID2
---------- -------------------- ----------
1 2
3 4
5 6
You may try another approach, I'm getting the same desired result with the following control file:
LOAD DATA
INFILE test.data
INTO TABLE test_sqlldr
APPEND
FIELDS TERMINATED BY ';'
TRAILING NULLCOLS
( id,
comments "substr(:comments,1,0)",
id2
)
Update following Romaintaz's comment: I looked into it again and managed to get the same error as you when the size of the column exceeded 255 characters. This is because the default datatype of SQL*Loader is char(255). If you have a column with more data you will have to specify the length. The following control file solved the problem for a column with 300 characters:
LOAD DATA
INFILE test.data
INTO TABLE test_sqlldr
APPEND
FIELDS TERMINATED BY ';'
TRAILING NULLCOLS
( id,
comments filler char(4000),
id2
)
Hope this Helps,
--
Vincent
Just to suggest a tiny improvement, you might try something like:
LOAD DATA
IN FILE test.data INTO TABLE test_sqlldr
APPEND
FIELDS TERMINATED BY ';'TRAILING NULLCOLS
(
id,
comments char(4000) "substr(:comments, 1, 200)",
id2)
Now you'll grab the first 200 characters (or any number you specify in it's place) of all comments - unless some of your input records have values for the comments field that exceed 4000 characters, in which they'll be rejected by loader with the 'exceeds max length' error noted earlier. But assuming that's rare or not the case, all the records will load with some of the comments truncated to 200 chars.
If you go over char(4000) you'll get a SQL Loader error - there's a limit to how far you can push the beast.