I have an Oracle SQL Loader control file based on position in a text file. One particular field periodically gets an ellipsis '...' from the source, which causes a carriage return in the loading table. No matter how many times I request '...' to NOT be used by these users, there is eventually someone who forgets, or due to staff turnover, etc. Here is the current control file line for that field:
TRAN_DESC POSITION(153 : 202) Char,
Is there any command that can be added to this line in order to ignore special characters such as an ellipsis?
I'd think of REPLACE. Here's an example.
Sample table:
SQL> create table test (id number, tran_desc varchar2(10));
Table created.
Control file:
load data
infile *
into table test
(id position(1:2),
tran_desc position(3:12) char "replace(:tran_desc, '...', '')"
)
begindata
10LittleFoot
11Big...foot
Loading session and result:
SQL> $sqlldr scott/tiger control=test2.ctl log=test2.log
SQL*Loader: Release 11.2.0.2.0 - Production on Pon Tra 5 17:03:39 2021
Copyright (c) 1982, 2009, Oracle and/or its affiliates. All rights reserved.
Commit point reached - logical record count 2
SQL> select * from test;
ID TRAN_DESC
---------- ----------
10 LittleFoot
11 Bigfoot
SQL>
Related
I'm currently inserting data some columns from a CSV file into a table by using SQL loader and that data is validated and the remaining columns are filled automatically based on the inserted data.
But if I have an unnecessary extra data beyond my required columns, that is inserting into the other columns of the table which is supposed to null if the data is ok after validation.
I want to take only certain columns from the CSV file and need to insert into table.. no need of any extra data from other columns in CSV file to be load.
What should I do?
I'm wondering if there is any thing I need to include in this!
Options (ERRORS=100000,SKIP=1)
Load data
Append
Into table emp
Fields terminated by ',' optionally enclosed by '"'
Trailing Nullcols
(Emp_id char,
Dept char,
Class integer,
Subclass integer
)
You don't have to do anything special - just omit all these columns from the control file, here:
(Emp_id char, Dept char, Class integer, Subclass integer )
To illustrate it: sample table has only two columns:
SQL> create table test
2 (emp_id number,
3 dept_name varchar2(10));
Table created.
Control file's data contain some more data - talking about this:
1,A,some more data,123
2,B,not important,553
------------------
no columns for this in the table
Control file itself:
LOAD DATA
INFILE *
REPLACE
INTO TABLE test
FIELDS TERMINATED BY ','
OPTIONALLY ENCLOSED BY '"'
TRAILING NULLCOLS
(
emp_id,
dept_name
)
BEGINDATA
1,A,some more data,123
2,B,not important,553
Loading session:
SQL> $sqlldr scott/tiger#orcl control=test45.ctl log=test45.log
SQL*Loader: Release 18.0.0.0.0 - Production on Čet Lis 27 12:55:34 2022
Version 18.5.0.0.0
Copyright (c) 1982, 2018, Oracle and/or its affiliates. All rights reserved.
Path used: Conventional
Commit point reached - logical record count 1
Commit point reached - logical record count 2
Table TEST:
2 Rows successfully loaded.
Check the log file:
test45.log
for more information about the load.
Result:
SQL> select * from test;
EMP_ID DEPT_NAME
---------- ----------
1 A
2 B
SQL>
As you can see, everything is just fine.
If it were vice versa, i.e. you'd want to populate columns that don't exist in the CSV file, you'd use e.g. filler (but - as that's not your issue - forget it. Actually, remember it, maybe you'll need it later).
I have a csv file that is pipe delimited and I'm trying to use SQL Loader to import the data. The data type in the table is Date. I'd like to import just the MM/DD/YYYY but I'm having errors.
My control file code for this field is:
field_a char(1024),
field_in_question DATE'MM/DD/RRRR',
field_c,
Dates in Sample File:
5/28/2019 0:00
3/30/2020 0:00
12/16/2019 0:00
The error I'm currently receiving is:
ORA-01858: a non-numeric character was found where a numeric was
expected
Any help would be greatly appreciated.
An oracle DATE type includes a time component. Your input data also has a time component. So just adjust your input date mask to account for it.
field_in_question DATE'MM/DD/YYYY hh:mi'
Notice I've also changed your mask for 'years' to 'YYYY'. The 'RR' and "RRRR' construct was meant as a temporary band-aid to buy time in solving the Y2K bug. And that was twenty years ago. Long past time to no longer need temporary fixes.
Here's how.
Sample table:
SQL> create table test (name varchar2(10), datum date, colc number);
Table created.
Control file (sample data included):
load data
infile *
replace
into table test
fields terminated by '|'
trailing nullcols
(
name,
datum "to_date(:datum, 'mm/dd/yyyy hh24:mi')",
colc
)
begindata
Little|5/28/2019 0:00|1
Foot|3/30/2020 0:00|2
Bigfoot|12/16/2019 0:00|3
Loading session and the result:
SQL> $sqlldr scott/tiger control=test23.ctl log=test23.log
SQL*Loader: Release 11.2.0.2.0 - Production on Pon Stu 16 22:35:58 2020
Copyright (c) 1982, 2009, Oracle and/or its affiliates. All rights reserved.
Commit point reached - logical record count 2
Commit point reached - logical record count 3
SQL> select * from test;
NAME DATUM COLC
---------- ------------------- ----------
Little 28.05.2019 00:00:00 1
Foot 30.03.2020 00:00:00 2
Bigfoot 16.12.2019 00:00:00 3
SQL>
FIle name emp.txt - the text file contains data like this:
emp_no,emp_EXPIRY_DATE,STATUS
a123456,2020-07-12,A
a123457,2020-07-12,A
I want to insert data into a temp table using a stored procedure.
Which database do you use? "Oracle" SQL Developer looks like "Oracle" (of course), but - code you posted as a comment isn't Oracle.
Anyway, if it was, then doing what you plan to do would require UTL_FILE package. CSV file should be put into a directory (usually on the database server) which is a source for directory (as an Oracle object); user that is supposed to load data should have read (and write?) privileges on it.
Alternatively, you could use the CSV file as an external table. That option might be simpler as it allows you to write ordinary SELECT statements against it, i.e. read data from it and insert into the target table that resides in an Oracle database. This option also requires the "directory" stuff.
Or, if you want to do that locally, consider using SQL*Loader; create a control file and load data. This option might be extremely fast, way faster than previous options. You won't see any difference for small files, but - for a lot of data - this might be your choice.
A SQL*Loader example:
Test table:
SQL> create table test
2 (emp_no varchar2(10),
3 emp_expiry_date date,
4 status varchar2(1));
Table created.
Control file:
options (skip=1)
LOAD DATA
infile emp.txt
replace
INTO TABLE test
FIELDS TERMINATED BY ',' OPTIONALLY ENCLOSED BY '"'
TRAILING NULLCOLS
(
emp_no,
emp_expiry_date "to_date(:emp_expiry_date, 'yyyy-dd-mm')",
status
)
Loading session & the result:
SQL> alter session set nls_date_Format = 'yyyy-mm-dd';
Session altered.
SQL> $sqlldr scott/tiger control=test13.ctl log=test13.log
SQL*Loader: Release 11.2.0.2.0 - Production on Sri Pro 11 21:02:44 2019
Copyright (c) 1982, 2009, Oracle and/or its affiliates. All rights reserved.
Commit point reached - logical record count 1
Commit point reached - logical record count 2
SQL> select * from test;
EMP_NO EMP_EXPIRY S
---------- ---------- -
a123456 2020-12-07 A
a123457 2020-12-07 A
SQL>
Is it possible to load csv file with sqlldr and use sequence at the same time?
Say for example I would like to use command
sqlldr id/pass#'ip:1521/ORCL' data=path\csv_test.csv
control=path\control.ctl log=path\log.log bad=path\bad.csv
to load csv file into database but at the same time use sequence to create an extra column that increments for every csv file insert (so every bulk insert of a csv file)
Sure there's an option; it is called a sequence. More info about it in Field List Reference documentation.
Here's an example.
Data will be loaded into the TEST table:
SQL> create table test
2 (id number,
3 ename varchar2(20)
4 );
Table created.
SQL>
A sequence will be used for the ID column. Control file looks like this:
load data
infile *
replace
into table test
(
id sequence,
ename char terminated by whitespace
)
begindata
Little
Foot
Stack
Over
Flow
Loading session:
M:\a1_maknuto>sqlldr scott/tiger#orcl control=test21.ctl
SQL*Loader: Release 11.2.0.2.0 - Production on Pon Vel 19 07:20:29 2018
Copyright (c) 1982, 2009, Oracle and/or its affiliates. All rights reserved.
Commit point reached - logical record count 4
Commit point reached - logical record count 5
Results:
SQL> select * From test;
ID ENAME
---------- --------------------
1 Little
2 Foot
3 Stack
4 Over
5 Flow
SQL>
[EDIT: aha, all rows should share the same "sequence"]
OK then, try something like this (note expression used for the ID column):
load data
infile *
append
into table test
(
id expression "userenv('sessionid')",
ename char(30) terminated by whitespace
)
begindata
Little
Foot
Stack
Over
Flow
A few loading sessions:
SQL> $sqlldr scott/tiger#orcl control=test21.ctl
SQL*Loader: Release 11.2.0.2.0 - Production on Pon Vel 19 08:13:23 2018
Copyright (c) 1982, 2009, Oracle and/or its affiliates. All rights reserved.
Commit point reached - logical record count 4
Commit point reached - logical record count 5
SQL> select * From test;
ID ENAME
---------- --------------------
4530297 Little
4530297 Foot
4530297 Stack
4530297 Over
4530297 Flow
SQL> $sqlldr scott/tiger#orcl control=test21.ctl
SQL*Loader: Release 11.2.0.2.0 - Production on Pon Vel 19 08:13:30 2018
Copyright (c) 1982, 2009, Oracle and/or its affiliates. All rights reserved.
Commit point reached - logical record count 4
Commit point reached - logical record count 5
SQL> select * From test;
ID ENAME
---------- --------------------
4530297 Little
4530297 Foot
4530297 Stack
4530297 Over
4530297 Flow
4530301 Little
4530301 Foot
4530301 Stack
4530301 Over
4530301 Flow
10 rows selected.
SQL>
Alternatively, you could use a sequence (an Oracle object). That's a little bit more "complex" (you'd need a function too) but - if you need it, I can create an example. Say so.
I have 4 tables I am copying from one schema to another using sqldr. 3 of the tables gave me no issues and I was able to successfully copy them all over. The 4th is where my problem arises. I cant quite understand why, there is nothing special about this 4th table as far as data types go or anything else. When i do run the sqlldr command, all rows end up in the .bad file and none are copied over. I will list the code im using for better understanding.
> pico deptbb02.csv
UW PICO(tm) 4.10 File: deptbb02.csv
10,infield,Jade,Clairmont,Lets play two
20,outfield,House of Pasta,Santee,Alea iacta est
30,pitcher,Crab Shack,Pacific Beach,Semper paratus
40,staff,Burger King,Lakeside,Experientia docet
50,catchers,Pinnacle Peak,Santee,Non Bastardi Carborundum
UW PICO(tm) 4.10 File: deptload.ctl
LOAD DATA
infile 'deptbb02.csv'
replace into table deptbb02
fields terminated by ','
(DEPTNO,DNAME,RESTAURANT,LOCATION,MOTTO)
> sqlldr username/password#database
control = deptload.ctl
SQL*Loader: Release 10.2.0.1.0 - Production on Tue Jul 28 01:27:38 2015
Copyright (c) 1982, 2005, Oracle. All rights reserved.
Commit point reached - logical record count 6
TABLE deptbb02 defined as...
SQL> desc deptbb02
Name Null? Type
----------------------------------------- -------- ----------------------------
DEPTNO NUMBER(3)
DNAME VARCHAR2(8)
RESTAURANT VARCHAR2(15)
LOCATION VARCHAR2(15)
MOTTO VARCHAR2(30)
I think this should be everything needed to understand my question, but don't hesitate to ask if i missed something. Thanks!
I suspect the data has spaces or control characters at the end of the MOTTO column or the sample data you posted is not the same as what was attempted to load in the .log file. i.e.
Record 3: Rejected - Error on table DEPTBB02, column MOTTO. ORA-12899: value too large for column "ST101"."DEPTBB02"."MOTTO" (actual: 44, maximum: 30)
The MOTTO column is defined in the table as varchar2(30) but sqlldr sees 44 characters. The data as shown for the 3rd record is 14 characters. Open the data file in an editor that can show control characters and spaces.
Try putting a TRIM() call around the fields in the control file to remove leading and trailing spaces maybe:
LOAD DATA
infile 'deptbb02.csv'
replace into table deptbb02
fields terminated by ','
(DEPTNO,
DNAME,
RESTAURANT,
LOCATION,
MOTTO CHAR "TRIM(:MOTTO)"
)