I have external tables. And I'd like to extract the data from those tables and insert/merge that data in other tables.
Now when a select from => insert into query or merge query runs then it is possible (and likely possible) that the data might be in a bad quality, which will result into breaking query. Say there is 000000 as date in external table which will result breaking query if I am merging data.
How can I log these errors in a table (for example) error table which will log the error, reason of error, line number and column name? Just like you see in SQL Loader logs. For example:
Record 2324: Rejected - Error on table AA_STAG_VR_01, column KS1.
ORA-01843: not a valid month
And the query shouldn't break upon rather. Rather log the error and move on like it happens in SQL Loader.
Is it possible? I tried to look around net but I wasn't unable to find anything, or maybe I simply don't know the magic words
Thanks in advance :-)
EDIT:
Ok, I was able to solve the problem (well, partly) using the following approach.
CREATE TABLE error_table (
ora_err_number$ NUMBER,
ora_err_mesg$ VARCHAR2(2000),
ora_err_rowid$ ROWID,
ora_err_optyp$ VARCHAR2(2),
ora_err_tag$ VARCHAR2(2000)
)
INSERT INTO destination_table (column)
SELECT column FROM external_table
LOG ERRORS INTO error_table REJECT LIMIT UNLIMITED
This gives me:
SELECT * FROM error_table;
----------------------------------------------------------------------------------------------------------------------------------------------------------
ORA_ERR_NUMBER$ | ORA_ERR_MESG$ | ORA_ERR_ROWID$ | ORA_ERR_OPTYP$ | ORA_ERR_TAG$ |
----------------------------------------------------------------------------------------------------------------------------------------------------------
12899 |ORA-12899: value too large for column "SYSTEM"."destination_table"."column"
So far, so good. However, I would like to know what record number (line number in external_table) has this error. Because it is possible that fist 10 records went ok but 11th record was bad.
Check out FORALL + SAVE EXCEPTIONS clause. It might help you.
15:57:02 #> conn hr/hr#vm_xe
Connected.
15:57:15 HR#vm_xe> create table err_test(unique_column number primary key);
Table created.
Elapsed: 00:00:01.51
15:57:46 HR#vm_xe> EXECUTE DBMS_ERRLOG.CREATE_ERROR_LOG('err_test', 'errlog');
PL/SQL procedure successfully completed.
Elapsed: 00:00:00.46
15:59:22 HR#vm_xe> insert into err_test select mod(rownum, 2) from dual connect by rownum < 10
16:00:00 2 log errors into errlog ('test') reject limit unlimited;
2 rows created.
Elapsed: 00:00:00.87
16:00:27 HR#vm_xe> commit;
Commit complete.
Elapsed: 00:00:00.00
16:02:37 HR#vm_xe> col ora_err_mesg$ for a75
16:02:43 HR#vm_xe> col unique_column for a10
16:02:47 HR#vm_xe> select unique_column, ora_err_mesg$ from errlog;
UNIQUE_COL ORA_ERR_MESG$
---------- ---------------------------------------------------------------------------
1 ORA-00001: unique constraint (HR.SYS_C007056) violated
0 ORA-00001: unique constraint (HR.SYS_C007056) violated
1 ORA-00001: unique constraint (HR.SYS_C007056) violated
0 ORA-00001: unique constraint (HR.SYS_C007056) violated
1 ORA-00001: unique constraint (HR.SYS_C007056) violated
0 ORA-00001: unique constraint (HR.SYS_C007056) violated
1 ORA-00001: unique constraint (HR.SYS_C007056) violated
7 rows selected.
Elapsed: 00:00:00.03
Below some syntax, you have reject limit as in sqlloader, you have log files, bad files etc.
CREATE TABLE <table_name> (
<column_definitions>)
ORGANIZATION EXTERNAL
(TYPE oracle_loader
DEFAULT DIRECTORY <oracle_directory_object_name>
ACCESS PARAMETERS (
RECORDS DELIMITED BY newline
BADFILE <file_name>
DISCARDFILE <file_name>
LOGFILE <file_name>
[READSIZE <bytes>]
[SKIP <number_of_rows>
FIELDS TERMINATED BY '<terminator>'
REJECT ROWS WITH ALL NULL FIELDS
MISSING FIELD VALUES ARE NULL
(<column_name_list>))\
LOCATION ('<file_name>'))
[PARALLEL]
REJECT LIMIT <UNLIMITED | integer>;
See here some examples, and here the docs
Related
I'm having hard time to add a new colomn of date of birth with the check rule age between 18 and 65.
I'm using sqplus with Oracle
Alway getting the error message ORA00920
Need your help please
ALTER TABLE Vendeur ADD (dateNaissance DATE,
dateDebutProjet DATE NOT NULL,
DateFinProjet DATE NOT NULL,
CONSTRAINT chk_date_Birth CHECK ((TRUNC(CURRENT_DATE)-dateNaissance)
BETWEEN 18 AND 65),
CONSTRAINT chk_date_Projet CHECK (DateFinProjet > dateDebutProjet));
if it can help, the solution without triggers (since we didn't learn hem at that time):
ALTER TABLE Vendeur ADD (dateNaissance DATE,
debutProjet DATE DEFAULT '01/01/1000' NOT NULL,
finProjet DATE DEFAULT '02/01/1000' NOT NULL,
dateDuJour Date DEFAULT CURRENT_DATE,
CONSTRAINT chk_date_Projet CHECK (finProjet > debutProjet),
CONSTRAINT chk_date_Birth CHECK ((dateDuJour - dateNaissance)\365 BETWEEN 18 AND 65)
);
Check constraints cannot call non-deterministic functions like CURRENT_DATE. Check constraints are supposed to always be true, weird things might happen if check constraints aged out.
The below sample code shows one of the errors you might get trying to use CURRENT_DATE in a check constraint:
SQL> create table test1(a date);
Table created.
SQL> alter table test1 add constraint test1_ck1 check(a > date '2000-01-01');
Table altered.
SQL> alter table test1 add constraint test1_ck2 check(a > current_date);
alter table test1 add constraint test1_ck2 check(a > current_date)
*
ERROR at line 1:
ORA-02436: date or system variable wrongly specified in CHECK constraint
Create a trigger to workaround this problem:
create or replace trigger test1_date_gt_today
before update or insert of a on test1
for each row
begin
if :new.a is null or :new.a < current_date then
raise_application_error(-20000, 'The date cannot be earlier than today.');
end if;
end;
/
Below is an example of one INSERT working, and one failing to meet the condition in the trigger:
SQL> insert into test1 values(sysdate + 1);
1 row created.
SQL> insert into test1 values(sysdate - 1);
insert into test1 values(sysdate - 1)
*
ERROR at line 1:
ORA-20000: The date cannot be earlier than today.
ORA-06512: at "JHELLER.TEST1_DATE_GT_TODAY", line 3
ORA-04088: error during execution of trigger 'JHELLER.TEST1_DATE_GT_TODAY'
Hi I want to capture all the Oracle Errors for my DML operations in the manually created table with columns as ErrorID and Error_Descr.
How to get ORA_ERR_NUMBER$ and ORA_ERR_MESG$ values in the above columns?
This table contains user defined errors as well so I do not want to limit it to the Oracle Errors.
Is there any way of capturing Oracle as well as User Defined Errors in the User Defined Tables?
Thanks in Advance!
As per documentation Link,
Oracle allows you to use a manually created table for LOGGING only if you have included these mandatory columns.
ORA_ERR_NUMBER$
ORA_ERR_MESG$
ORA_ERR_ROWID$
ORA_ERR_OPTYP$
ORA_ERR_TAG$
If you want other columns to capture the information in those two columns, you could make them as virtual columns.
CREATE TABLE my_log_table (
ORA_ERR_NUMBER$ NUMBER
,ORA_ERR_MESG$ VARCHAR2(2000)
,ORA_ERR_ROWID$ ROWID
,ORA_ERR_OPTYP$ VARCHAR2(2)
,ORA_ERR_TAG$ VARCHAR2(2000)
,ErrorID NUMBER AS (COALESCE(ORA_ERR_NUMBER$, ORA_ERR_NUMBER$))
,Error_Descr VARCHAR2(2000) AS (COALESCE(ORA_ERR_MESG$, ORA_ERR_MESG$))
);
using COALESCE is a hack because Oracle doesn't allow you to have one column default to another directly.
Now, you could run your error logging dml normally mentioning the logging table name.
INSERT INTO t_emp
SELECT employee_id * 10000
,first_name
,last_name
,hire_date
,salary
,department_id
FROM hr.employees
WHERE salary > 10000 LOG ERRORS
INTO my_log_table('ERR_SAL_LOAD') REJECT LIMIT 25
0 row(s) inserted.
select ORA_ERR_TAG$,ErrorID,Error_Descr FROM my_log_table ;
ORA_ERR_TAG$ ERRORID ERROR_DESCR
ERR_SAL_LOAD 1438 ORA-01438: value larger than specified precision allowed for this column
ERR_SAL_LOAD 1438 ORA-01438: value larger than specified precision allowed for this column
I'm trying to insert information in a partition table, but I don't know what I'm doing wrong!
Show me this error: ORA-14400: inserted partition key does not map to any partition"
The table dba_tab_partitions shows this informations below:
1 PDIA_98_20091023 0
2 PDIA_98_20091022 0
3 PDIA_98_20091021 0
4 PDIA_98_20091020 0
5 PDIA_98_20091019 0
Please help me rs
select partition_name,column_name,high_value,partition_position
from ALL_TAB_PARTITIONS a , ALL_PART_KEY_COLUMNS b
where table_name='YOUR_TABLE' and a.table_name = b.name;
This query lists the column name used as key and the allowed values. make sure, you insert the allowed values(high_value). Else, if default partition is defined, it would go there.
EDIT:
I presume, your TABLE DDL would be like this.
CREATE TABLE HE0_DT_INF_INTERFAZ_MES
(
COD_PAIS NUMBER,
FEC_DATA NUMBER,
INTERFAZ VARCHAR2(100)
)
partition BY RANGE(COD_PAIS, FEC_DATA)
(
PARTITION PDIA_98_20091023 VALUES LESS THAN (98,20091024)
);
Which means I had created a partition with multiple columns which holds value less than the composite range (98,20091024);
That is first COD_PAIS <= 98 and Also FEC_DATA < 20091024
Combinations And Result:
98, 20091024 FAIL
98, 20091023 PASS
99, ******** FAIL
97, ******** PASS
< 98, ******** PASS
So the below INSERT fails with ORA-14400; because (98,20091024) in INSERT is EQUAL to the one in DDL but NOT less than it.
SQL> INSERT INTO HE0_DT_INF_INTERFAZ_MES(COD_PAIS, FEC_DATA, INTERFAZ)
VALUES(98, 20091024, 'CTA'); 2
INSERT INTO HE0_DT_INF_INTERFAZ_MES(COD_PAIS, FEC_DATA, INTERFAZ)
*
ERROR at line 1:
ORA-14400: inserted partition key does not map to any partition
But, we I attempt (97,20091024), it goes through
SQL> INSERT INTO HE0_DT_INF_INTERFAZ_MES(COD_PAIS, FEC_DATA, INTERFAZ)
2 VALUES(97, 20091024, 'CTA');
1 row created.
For this issue need to add the partition for date column values, If last partition 20201231245959, then inserting the 20210110245959 values, this issue will occurs.
For that need to add the 2021 partition into that table
ALTER TABLE TABLE_NAME ADD PARTITION PARTITION_NAME VALUES LESS THAN (TO_DATE('2021-12-31 24:59:59', 'SYYYY-MM-DD HH24:MI:SS', 'NLS_CALENDAR=GREGORIAN')) NOCOMPRESS
I am trying to maintain data integrity and logging errors to an error table. I've got 3 tables with unique constraints and 1 error table:
create table tbl_one (
pass_no number,
constraint tbl_one_u01 unique (pass_no) );
insert into tbl_one values(10);
insert into tbl_one values(20);
create table tbl_two (
cus_no number,
cus_name varchar2(50),
pass_no number,
constraint tbl_two_u01 unique (cus_no) );
insert into tbl_two values( 101, 'NameX',10);
insert into tbl_two values( 102, 'NameY',10);
insert into tbl_two values( 103, 'NameZ',20);
create table tbl_target (
cus_no number,
pass_no number,
constraint tbl_target_u01 unique (cus_no),
constraint tbl_target_u02 unique (pass_no));
exec dbms_errlog.create_error_log('tbl_target','tbl_target_err');
I am trying to log all ORA-00001 errors to the error table tbl_target_err like this:
begin
insert into tbl_target
select a.pass_no, b.cus_no
from tbl_one a
inner join tbl_two b on b.pass_no = a.pass_no
log errors into tbl_target_err reject limit 10;
end;
The result is:
select * from tbl_target;
-------------------
CUS_NO PASS_NO
101 10
103 20
and the error table:
CUS_NO PASS_NO
102 10
I need all the violated errors to go into the error table. If the value of pass_no 10 is violated then all 10 values should inserted into the error table; not, one into target and one to error table. I don't want to use exists statements because I won't able to log all violated values.
How could I go about doing this?
You can't use the error logging mechanism for this as it isn't designed to support it. It errors at the point it tries to create the duplicate in the table - the first value it tries to insert for pass_no 10 is valid on its own - so it would have to distinguish between data that already existed and multiple values coming from the insert, to start with. So you'd need to roll your own.
One option is to create your own table to hold the duplicates, and use an insert all to decide which values belong in each table:
create table tbl_target_dup (
cus_no number,
pass_no number
);
insert all
when cus_count = 1 and pass_count = 1 then
into tbl_target values (cus_no, pass_no)
else
into tbl_target_dup values (cus_no, pass_no)
select a.pass_no, b.cus_no,
count(*) over (partition by a.pass_no) as pass_count,
count(*) over (partition by b.cus_no) as cus_count
from tbl_one a
join tbl_two b on b.pass_no = a.pass_no;
This allows you to have more columns than those affected by the PK/UK, and insert them only into the real table if you prefer, or a subset into the 'error' table. With just those two columns in each table you get:
select * from tbl_target;
CUS_NO PASS_NO
---------- ----------
103 20
select * from tbl_target_dup;
CUS_NO PASS_NO
---------- ----------
101 10
102 10
SQL Fiddle demo.
You could do the same thing with two inserts based on the same select, one with a subquery checking that both counts are 1, other checking that at least one is not, but this might perform better.
I'm having a weird problem with index organized table. I'm running Oracle 11g standard.
i have a table src_table
SQL> desc src_table;
Name Null? Type
--------------- -------- ----------------------------
ID NOT NULL NUMBER(16)
HASH NOT NULL NUMBER(3)
........
SQL> select count(*) from src_table;
COUNT(*)
----------
21108244
now let's create another table and copy 2 columns from src_table
set timing on
SQL> create table dest_table(id number(16), hash number(20), type number(1));
Table created.
Elapsed: 00:00:00.01
SQL> insert /*+ APPEND */ into dest_table (id,hash,type) select id, hash, 1 from src_table;
21108244 rows created.
Elapsed: 00:00:15.25
SQL> ALTER TABLE dest_table ADD ( CONSTRAINT dest_table_pk PRIMARY KEY (HASH, id, TYPE));
Table altered.
Elapsed: 00:01:17.35
It took Oracle < 2 min.
now same exercise but with IOT table
SQL> CREATE TABLE dest_table_iot (
id NUMBER(16) NOT NULL,
hash NUMBER(20) NOT NULL,
type NUMBER(1) NOT NULL,
CONSTRAINT dest_table_iot_PK PRIMARY KEY (HASH, id, TYPE)
) ORGANIZATION INDEX;
Table created.
Elapsed: 00:00:00.03
SQL> INSERT /*+ APPEND */ INTO dest_table_iot (HASH,id,TYPE)
SELECT HASH, id, 1
FROM src_table;
"insert" into IOT takes 18 hours !!! I have tried it on 2 different instances of Oracle running on win and linux and got same results.
What is going on here ? Why is it taking so long ?
The APPEND hint is only useful for a heap-organized table.
When you insert into an IOT, I suspect that each row has to be inserted into the real index structure separately, causing a lot of re-balancing of the index.
When you build the index on a heap table, a temp segment is used and I'm guessing that this allows it to reduce the re-balancing overhead that would otherwise take place.
I suspect that if you created an empty, heap-organized table with the primary key, and did the same insert without the APPEND hint, it would take more like the 18 hours.
You might try putting an ORDER BY on your SELECT and see how that affects the performance of the insert into the IOT. It's not guaranteed to be an improvement by any means, but it might be.