SQL Loader issue loading decimal negative numbers - oracle

I have the following issue using SQL Loader to load a small table, my process populates a field which can be populated for possitive or negative decimal numbers, and the problem is that when the number is possitive, the process rounds with 2 positions, but if it's negative only use 1 position. I need 2 position, and this is the simple code:
UNRECOVERABLE LOAD DATA
TRUNCATE
INTO TABLE [SCHEMA].TABLE1
FIELDS TERMINATED BY '|'
OPTIONALLY ENCLOSED BY '"'
TRAILING NULLCOLS
(
FIELD1 "trim(:FIELD1 )"
)
Example:
Source: 12345,78 Target: 1234,78
Source: -12345,78 Target: -1234,8
Edit1: It just happend with number with 7 or more integer length (X < -1000000), but I can insert this numbers with regular insert instead of SQLLoader.
Edit2: I have noticed that it is not a problem with sql*loader because the .ctl is already wrong. This file is generated with python and SQL with this properties:
set colsep |
set headsep off
set pagesize 0
set trimspool on
set linesize 10000
set termout off
set feedback off
set arraysize 250
But I dont know where it is defined the lenght of each field, and this looks the error.

I'd say that you're not looking correctly. Data is, probably, stored OK, but you don't see it. Here's an example:
SQL> create table test (field1 number);
Table created.
SQL>
Control file:
load data
infile *
replace into table test
fields terminated by '|' trailing nullcols
(
field1
)
begindata
12345,78
-12345,78
987654321
-987654321
987654321,01234
-987654321,01234
111333444,887766
-112333444,887766
Loading session & the result:
SQL> $sqlldr scott/tiger#orcl control=test26.ctl log=test26.log
SQL*Loader: Release 11.2.0.2.0 - Production on Uto Kol 21 08:52:10 2018
Copyright (c) 1982, 2009, Oracle and/or its affiliates. All rights reserved.
Commit point reached - logical record count 7
Commit point reached - logical record count 8
SQL> select * from test;
FIELD1
----------
12345,78
-12345,78
987654321
-987654321
987654321
-987654321
111333445
-112333445
8 rows selected.
SQL>
Ooops! Seems to be wrong, eh? But, if you fix the column format ...
SQL> col field1 format 999g999g999g990d000000
SQL> select * from test;
FIELD1
-----------------------
12.345,780000
-12.345,780000
987.654.321,000000
-987.654.321,000000
987.654.321,012340
-987.654.321,012340
111.333.444,887766
-112.333.444,887766
8 rows selected.
SQL>
See? Everything is here.
I guess that, if you used a GUI, you wouldn't have such problems. The former sentence doesn't mean that you should NOT use SQL*Plus - on the contrary; just ... learn how to use it properly :)
[EDIT, after reading the comment]
It is SET NUMFORMAT you might be looking for:
SQL> select -12334.789123 from dual;
-12334.789123
-------------
-12334,789
SQL> set numformat 999g999g990d000000
SQL> select -12334.789123 from dual;
-12334.789123
-------------------
-12.334,789123
SQL>

Related

sqlloader skips characters during data load into table

using the control file below to load data
LOAD DATA
INFILE '/c/Transaction.txt'
INTO TABLE tab1 APPEND WHEN (1:1) = 'D'
(RUN_ID "RUN_ID_SEQ.NEXTVAL"
,RUN_DATE_TIME "SYSDATE"
)
INTO TABLE tab2 TRUNCATE WHEN(1:1) <> 'D'
FIELDS TERMINATED BY '|' TRAILING NULLCOLS
(
DEALER_NUMBER
,TRAN_CODE
,TRAN_AMOUNT "TO_NUMBER(:PL818_TRAN_AMOUNT,'999999.99')"
,TRAN_DATE "TO_DATE(:PL818_TRAN_DATE,'DD-MM-YYYY')"
)
This is the data set
DEALER_ID|TRAN_TYPE_CODE|TRAN_AMOUNT|TRAN_DATE
203113|34|1000.50|12-07-2022
No errors during load. Data loads correctly into the first table but first two characters are skipped when loading data into second table, table data looks like this. What could be causing this?
3113 34 1000.5 12-JUL-22
Skips "20"
Sample tables:
SQL> desc tab1
Name Null? Type
----------------------------------------- -------- ----------------------------
RUN_ID NUMBER
RUN_DATE_TIME DATE
SQL> desc tab2
Name Null? Type
----------------------------------------- -------- ----------------------------
DEALER_NUMBER NUMBER
TRAN_CODE NUMBER
TRAN_AMOUNT NUMBER
TRAN_DATE DATE
SQL>
Control file; note position(1) when loading into tab2. You must use it when loading data into different tables, using the same control file:
LOAD DATA
INFILE *
INTO TABLE tab1 APPEND WHEN (1:1) = 'D'
(RUN_ID "RUN_ID_SEQ.NEXTVAL"
,RUN_DATE_TIME "SYSDATE"
)
INTO TABLE tab2 TRUNCATE WHEN(1:1) <> 'D'
FIELDS TERMINATED BY '|' TRAILING NULLCOLS
(
DEALER_NUMBER position(1) --> here
,TRAN_CODE
,TRAN_AMOUNT "TO_NUMBER(:TRAN_AMOUNT,'999999.99')"
,TRAN_DATE "TO_DATE(:TRAN_DATE,'DD-MM-YYYY')"
)
BEGINDATA
203113|34|1000.50|12-07-2022
Loading session and the result:
SQL> $sqlldr scott/tiger#orcl control=test42.ctl log=test42.log
SQL*Loader: Release 18.0.0.0.0 - Production on Sri Srp 13 08:28:19 2022
Version 18.5.0.0.0
Copyright (c) 1982, 2018, Oracle and/or its affiliates. All rights reserved.
Path used: Conventional
Commit point reached - logical record count 1
Table TAB1:
0 Rows successfully loaded.
Table TAB2:
1 Row successfully loaded.
Check the log file:
test42.log
for more information about the load.
SQL> select * from tab2;
DEALER_NUMBER TRAN_CODE TRAN_AMOUNT TRAN_DATE
------------- ---------- ----------- ----------
203113 34 1000,5 12-07-2022
--
here's your missing "20"
SQL>

Remove Blank Line In Between Select Queries When Spooling to CSV File 2

I have query that is executed from SQLPLUS (11.2.0.4) and output is spooled to file. I am getting output file empty lines after each line + many empty spaces after the last column in the query. I have opened the file in notepad++ and i can see tons of empty spaces appended to the last column and ended with CRLF ( windows) , the last column is varchar 4000.
I have read tons of articles related to the same in internet but none of them help me. I have tried to set trimspool ON/OFF, trimout ON/OFF (btw - both commands are skipped ), page 0, heading off, pagesize 0, page 0, space 0, newpage nono
Example of SQL :
set termout off
set pagesize 0
set termout off
set pagesize 0
set heading off
set feedback off
set newpage none
set space 0
set linesize 8000
set longchunksize 200000
/*above was tried step by step - no help*/
spool "G:/gggg/fffff.csv"
PROMT COL1|COL2|COL3
select col1||';'||col2||';'||nvl(col3,'') abc
FROM transactions;
spool off;
output (notepad++):
COL1|COL2|COL3CRLF
col1|col2|col3 CRLF
col1|col2|col3 CRLF
col1|col2|col3 CRLF
output(linux)
COL1|COL2|COL3$
col1|col2|$
COL3 $
col1|col2|$
COL3 $
col1|col2|$
COL3 $
I have to use SQLPLUS .
it is requested to have output file without double quotes and with tilda as delimiter
e.g.
aaaa~bbbb~cccc~eeeeeeeeeeeeeeeeeeeeeeeeee
dddd~rrrr~bggggggg~rrrrrrrrrrrrrrrrrrrrrrrrrr
eeee~rrrrrrr~ttttttt~yyyyyyyyyyyyyyyyyyyyyyyyyy
Columns type :
col1 integer
col2 varchar2(4000 BYTE)
col3 varchar2(4000 BYTE)
Current output looks like
COL1NAME~COL2NAME~COL3NAME
aaaa~bbbb~cccc~eeeeeeeeeeeeeeeeeeeeeeeeee
dddd~rrrr~bggggggg~rrrrrrrrrrrrrrrrrrrrrrrrrr
eeee~rrrrrrr~ttttttt~yyyyyyyyyyyyyyyyyyyyyyyyyy
I would like to remove the empty lines without any cut of the characters in lines.
I can see in notepad++ :
COL1NAME~COL2NAME~COL3NAMECRLF
aaaa~bbbb~cccc~eeeeeeeeeeeeeeeeeeeeeeeeee..................................CRLF
dddd~rrrr~bggggggg~rrrrrrrrrrrrrrrrrrrrrrrrrr..............................CRLF
eeee~rrrrrrr~ttttttt~yyyyyyyyyyyyyyyyyyyyyyyyyy............................CRLF
I can see in linux (after less) :
COL1NAME~COL2NAME~COL3NAME$
aaaa~bbbb~cccc~eeeeeeeeeeeeeeeeeeee$
eeeeee $
$
dddd~rrrr~bggggggg~rrrrrrrrrrrrrrrr$
rrrrrrrrrr $
$
eeee~rrrrrrr~ttttttt~yyyyyyyyyyyyyy$
yyyyyyyyyyyy $
$
What's your table look like? Is COL3 defined as a CHAR or VARCHAR2? If a CHAR, that explains everything.
We need to see your data and your table, to be able to help you, otherwise you're asking us to guess.
Using the HR sample schema and the locations table, notice the varchar2's.
CREATE TABLE locations (
location_id NUMBER(4, 0),
street_address VARCHAR2(40),
postal_code VARCHAR2(12),
city VARCHAR2(30)
CONSTRAINT loc_city_nn NOT NULL ENABLE,
state_province VARCHAR2(25),
country_id CHAR(2),
CONSTRAINT loc_c_id_fk FOREIGN KEY ( country_id )
REFERENCES hr.countries ( country_id )
ENABLE
);
COUNTRY_ID is CHAR(2) - so you'll always see 2 spaces for that column, even if you only inserted '1' character.
Querying the table, with sqlformat set to CSV, I get CSV back, automatically.
SQLPlus 11.2.0.4 is approx from 2008, that's quite old.
Here's what you can do with Oracle SQLcl - smaller than SQLPlus, more powerful, and more user-friendly. Also, less code to get what you want.
SQLcl: Release 21.4 Production on Wed Feb 16 08:40:23 2022
Copyright (c) 1982, 2022, Oracle. All rights reserved.
Last Successful login time: Wed Feb 16 2022 08:40:24 -05:00
Connected to:
Oracle Database 19c Enterprise Edition Release 19.0.0.0.0 - Production
Version 19.3.0.0.0
login.sql found in the CWD. DB access is restricted for login.sql.
Adjust the SQLPATH to include the path to enable full functionality.
SQL> set pagesize 200
SQL> cd c:\users\jdsmith\desktop
SQL> spool peeps.csv
SQL> select * from locations;
"LOCATION_ID","STREET_ADDRESS","POSTAL_CODE","CITY","STATE_PROVINCE","COUNTRY_ID"
1000,"1297 Via Cola di Rie","00989","Roma","","IT"
1100,"93091 Calle della Testa","10934","Venice","","IT"
1200,"2017 Shinjuku-ku","1689","Tokyo","Tokyo Prefecture","JP"
1300,"9450 Kamiya-cho","6823","Hiroshima","","JP"
1400,"2014 Jabberwocky Rd","26192","Southlake","Texas","US"
1500,"2011 Interiors Blvd","99236","South San Francisco","California","US"
1600,"2007 Zagora St","50090","South Brunswick","New Jersey","US"
1700,"2004 Charade Rd","98199","Seattle","Washington","US"
1800,"147 Spadina Ave","M5V 2L7","Toronto","Ontario","CA"
1900,"6092 Boxwood St","YSW 9T2","Whitehorse","Yukon","CA"
2000,"40-5-12 Laogianggen","190518","Beijing","","CN"
2100,"1298 Vileparle (E)","490231","Bombay","Maharashtra","IN"
2200,"12-98 Victoria Street","2901","Sydney","New South Wales","AU"
2300,"198 Clementi North","540198","Singapore","","SG"
2400,"8204 Arthur St","","London","","UK"
2500,"Magdalen Centre, The Oxford Science Park","OX9 9ZB","Oxford","Oxford","UK"
2600,"9702 Chester Road","09629850293","Stretford","Manchester","UK"
2700,"Schwanthalerstr. 7031","80925","Munich","Bavaria","DE"
2800,"Rua Frei Caneca 1360 ","01307-002","Sao Paulo","Sao Paulo","BR"
2900,"20 Rue des Corps-Saints","1730","Geneva","Geneve","CH"
3000,"Murtenstrasse 921","3095","Bern","BE","CH"
3100,"Pieter Breughelstraat 837","3029SK","Utrecht","Utrecht","NL"
3200,"Mariano Escobedo 9991","11932","Mexico City","Distrito Federal,","MX"
23 rows selected.
SQL> spool off
SQL>
And here's what that looks like -
If you can update your question with sample DDL and data, we can give you a better answer.
Disclaimer: I'm a product manager at Oracle for our Database tools.

ignore space in column value while loading data using sql loader by oracle

I am trying to load data using oracle sql loader and using space as separator of columns but I am facing problem that one of the columns value including space, I need your support to avoid considering this space as column separator.
I tried to use regexp_replace and replace functions
DSTCOUNTRY " REGEXP_REPLACE(:DSTCOUNTRY,'dstcountry=','')",
the column value is: dstcountry="United States"
and the expecting value to be stored in the table is:
United States
The sql loader command is:
load data
infile 'in'
append
into table test_table
fields terminated by " " optionally enclosed by '"'
TRAILING NULLCOLS
DSTCOUNTRY " REPLACE(:DSTCOUNTRY,'dstcountry=','')",
I am using oracle 10G and 12C.
According to what you posted so far, it is optionally enclosed you're looking for. Here's an example.
Test table:
SQL> create table test (id number, dstcountry varchar2(20));
Table created.
Control file (contains sample data as well):
load data
infile *
replace
into table test
fields terminated by " "
optionally enclosed by '"'
trailing nullcols
(
id,
dstcountry)
begindata
123 "Croatia"
125 "United States"
Loading session and result:
SQL> $sqlldr scott/tiger control=test08.ctl log=test08.log
SQL*Loader: Release 11.2.0.2.0 - Production on Pon Srp 22 17:59:23 2019
Copyright (c) 1982, 2009, Oracle and/or its affiliates. All rights reserved.
Commit point reached - logical record count 1
Commit point reached - logical record count 2
SQL> select * from test;
ID DSTCOUNTRY
---------- --------------------
123 Croatia
125 United States
SQL>

Load data from CSV file (data seperated by pipe "|") into a table using SQL loader when the data present in double quotes

I am loading CSV file data into table called EMPLOYEE using SQL*Loader.
My CSV file data is separated by pipe (|):
EMPID|EMPNAME_ADDRESS|SALARY|GRADE
123|Rams Hyd|1000|A1
124|Sand MUM|2,000|A2
125|"PRASANNA qwer trasf\"501 advv vvd, qee ggg\trfe \411005 THE|3,00,000|A3
and my control file is:
LOAD DATA
Insert INTO TABLE EMPLOYEE
Fields terminated by "|" Optionally enclosed by '"' TRAILING NULLCOLS
(
EMPID,
EMPNAME,
SALARY,
GRADE
)
When I load the data using above control file, the first two record are loading fine and for the third record i am getting error like as mentioned below.
no terminator found after TERMINATED and ENCLOSED field
Please suggest changes to be done to load the data properly.
Here's a way to load such data.
First, a target table (I hope it makes sense; you should have provided it):
SQL> create table employee
2 (empid number,
3 empname varchar2(200),
4 salary number,
5 grade varchar2(10)
6 );
Table created.
SQL>
Control file: note the SKIP option (which skips the header line), as well as REPLACE function call for the SALARY column (to remove superfluous commas which are formatting stuff; add them later, in the presentation layer).
options (skip = 1)
load data
infile *
replace
into table employee
fields terminated by "|" TRAILING NULLCOLS
(
empid,
empname,
salary "replace(:salary, ',', null)",
grade)
begindata
EMPID|EMPNAME_ADDRESS|SALARY|GRADE
123|Rams Hyd|1000|A1
124|Sand MUM|2,000|A2
125|"PRASANNA qwer trasf\"501 advv vvd, qee ggg\trfe \411005 THE|3,00,000|A3
Loading session:
SQL> $sqlldr scott/tiger control=test06.ctl log=test06.log
SQL*Loader: Release 11.2.0.2.0 - Production on Uto Tra 9 19:37:46 2019
Copyright (c) 1982, 2009, Oracle and/or its affiliates. All rights reserved.
Commit point reached - logical record count 2
Commit point reached - logical record count 3
SQL> select * From employee;
EMPID EMPNAME SALARY GRADE
---------- ---------------------------------------- ---------- ----------
123 Rams Hyd 1000 A1
124 Sand MUM 2000 A2
125 "PRASANNA qwer trasf\"501 advv vvd, qee 300000 A3
ggg\trfe \411005 THE
SQL>

Are default values supported in Oracle for columns with char semantics?

I was trying to add a column to a table but got a surprising effect with the DEFAULT clause. In a table with existing rows, I added a new column like so:
alter table t add c char(1 char) default 'N' not null;
When I subsequently added a check constraint to the table, it failed:
alter table t add constraint chk check(c in ('N', 'Y'));
Which resulted in
ERROR at line 1:
ORA-02293: cannot validate (T.CHK) - check constraint violated.
Other info:
Because I'm setting the units explicitly (i.e., char(1 char) as opposed to just char(1)), I don't expect the value of nls_length_semanatics to be relevant.
After adding the column as char(1 char), the newly added "N"s are actually "N " and I'm not sure what the extra whitespace is.
Adding the column as char(1 byte) works as expected;
Adding the column without the "default 'N' not null" followed by updating all existing rows to 'N', followed by altering the column to 'not null' also works as expected.
NLS_CHARACTERSET is AL32UTF8, but I don't expect that to be relevant either.
Database is Oracle 11g; 11.2.0.1.0.
Thanks.
I believe that what you're seeing is a bug that relies on a couple different things interacting
First, the database character set has to be a variable width character set (i.e. AL32UTF8) so that a single character may require up to four bytes of storage.
Second, the column must be declared with character length semantics
Third, starting in 11.1, Oracle added an optimization so that if you add a column to the table that is declared NOT NULL and that has a DEFAULT that Oracle could do that simply by updating the data dictionary rather than actually storing the default value in every row of the table.
When both of those things are true, it appears that the value that is returned has a length of 4 and is padded with the CHR(0) character.
SQL> select * from v$version;
BANNER
--------------------------------------------------------------------------------
Oracle Database 11g Enterprise Edition Release 11.2.0.1.0 - 64bit Production
PL/SQL Release 11.2.0.1.0 - Production
CORE 11.2.0.1.0 Production
TNS for 64-bit Windows: Version 11.2.0.1.0 - Production
NLSRTL Version 11.2.0.1.0 - Production
SQL> create table foo( col1 number );
Table created.
SQL> insert into foo values( 1 );
1 row created.
SQL> commit;
Commit complete.
SQL> alter table foo add c char(1 char) default 'N' not null;
Table altered.
SQL> alter table foo add constraint chk_foo check( c in ('Y', 'N') );
alter table foo add constraint chk_foo check( c in ('Y', 'N') )
*
ERROR at line 1:
ORA-02293: cannot validate (SCOTT.CHK_FOO) - check constraint violated
SQL> select c, dump(c) from foo;
C DUMP(C)
---- ------------------------------
N Typ=1 Len=4: 78,0,0,0
If you actually force the value to be stored in the table, you'll get the expected behavior where there is no CHR(0) padding. So if I insert a new row into the table, it passes.
SQL> insert into foo(col1) values (2);
1 row created.
SQL> select c, dump(c) from foo;
C DUMP(C)
---- ------------------------------
N Typ=1 Len=4: 78,0,0,0
N Typ=1 Len=1: 78
You could also issue an UPDATE to update the rows that aren't actually storing the value in the rows of the table
SQL> update foo
2 set c = 'N'
3 where c != 'N';
1 row updated.
SQL> select c, dump(c) from foo;
C DUMP(C)
---- ------------------------------
N Typ=1 Len=1: 78
N Typ=1 Len=1: 78
You tagged oracle11g, but you didn't specify a version.
This works for me on 11.2.0.2 on Linux x86-64.
SQL*Plus: Release 11.2.0.2.0 Production on Mon Mar 26 13:13:52 2012
Copyright (c) 1982, 2010, Oracle. All rights reserved.
Connected to:
Oracle Database 11g Enterprise Edition Release 11.2.0.2.0 - 64bit Production
With the Partitioning, Real Application Clusters, Automatic Storage Management and OLAP options
SQL> create table tt(a number);
Table created.
SQL> insert into tt values (1);
1 row created.
SQL> commit;
Commit complete.
SQL> alter table tt add c char(1 char) default 'N' not null;
Table altered.
SQL> alter table tt add constraint chk check(c in('N','Y'));
Table altered.
SQL> select * from tt;
A C
---------- -
1 N
SQL> column dump(c) format a30
SQL> select c, length(c),dump(c) from tt;
C LENGTH(C) DUMP(C)
- ---------- ------------------------------
N 1 Typ=96 Len=1: 78
So....perhaps you have a bug in your version?
Hope that helps.
The reason for the ORA-02293 error, as you've already mentioned, is because it is inserting 'N ' (with a padded white space) rather than 'N'. So your constraint is violated.
The more interesting question is, why is it adding that space? Well, by definition, a CHAR is fixed width, where a VARCHAR is not. A CHAR will always pad white space to fill the entire memory space allocated for the column. Because you've chosen a width of 1 CHAR, and AL32UTF8 is a varying width character set, that would seem to conflict with the fixed width nature of CHAR. It looks like it gets padded to fill the extra byte(s) not used by the 'N'. Or, at least, I assume that is what is happening.

Resources