We are using Oracle database (12c)
A table ABCD has below structure with around o(10^5) rows
CCOL* - Varchar2 column
DCOL* - Timestamp column
Name Null? Type
----- -------- -----------------
CCOL1 NOT NULL VARCHAR2(64 CHAR)
CCOL2 NOT NULL VARCHAR2(30 CHAR)
CCOL3 NOT NULL VARCHAR2(64 CHAR)
DCOL1 NOT NULL TIMESTAMP(6)
CCOL4 NOT NULL VARCHAR2(64 CHAR)
DCOL2 NOT NULL TIMESTAMP(6)
CCOL5 VARCHAR2(32 CHAR)
The primary key on this table is (CCOL1, CCOL2)
We have a below insert statement:
Insert 1:
BEGIN
INSERT INTO abcd (
ccol1, ccol2, ccol3, dcol1, ccol4, dcol2, ccol5)
SELECT
:b1, :b2, :b3, :b4, :b5, :b6, :b7
FROM
dual
WHERE
NOT EXISTS (
SELECT 1 FROM abcd WHERE ccol1 = :b1 AND ccol2 = :b2 );
EXCEPTION
WHEN dup_val_on_index THEN
NULL;
WHEN OTHERS THEN
RAISE;
END;
And alternative to this we have (removed the where not exists part)
Insert 2:
BEGIN
INSERT INTO abcd (
ccol1, ccol2, ccol3, dcol1, ccol4, dcol2, ccol5)
VALUES
(:b1, :b2, :b3, :b4, :b5, :b6, :b7);
EXCEPTION
WHEN dup_val_on_index THEN
NULL;
WHEN OTHERS THEN
RAISE;
END;
Which of these inserts is better?
I assume you're asking because you call this piece of code a large number of times. Generally, which approach is faster will depend on how probable it is that the values you're inserting already exist in the table. Throwing and catching an exception is orders of magnitude slower than checking that a primary key value does not already exist. But if you're only going to throw the exception once on every million inserts, the second approach is likely to be more efficient. When I had something like this in the past, there were also significant differences between Oracle versions in terms of how much more costly it was to throw and catch exceptions so the exact break-even point will vary depending on the Oracle version as well. Realistically, you'd need to benchmark on your system to be sure.
Personally, I'd probably write this (as #Boneist suggest) as a MERGE with just a when not matched clause
MERGE INTO abcd dest
USING( SELECT :b1 ccol1, :b2 ccol2, :b3 ccol3, ...
FROM dual ) src
ON( src.ccol1 = dest.ccol1 AND
src.ccol2 = dest.ccol2 )
WHEN NOT MATCHED THEN
INSERT( ccol1, ccol2, ccol3, ... )
VALUES( src.ccol1, src.ccol2, src.ccol3, ... )
Separately, having a WHEN OTHERS exception handler that just does a RAISE doesn't make much sense-- just don't catch the exception if you aren't going to do anything with it. In your first piece of code, it doesn't make sense to catch and ignore the dup_val_on_index check because you've already got the NOT EXISTS clause. At that point, the dup_val_on_index is no longer an expected exception so you shouldn't be catching and ignoring it.
Related
I have a table like this:
create table my_table
(
type1 varchar2(10 char),
type2 varchar2(10 char)
);
I want to uniqueness like this;
if type1 column has 'GENERIC' value then just type2 column must be unique for the table. for example;
type1 column has 'GENERIC' value and type2 column has 'value_x' then there must not any type2 column value that equals to 'value_x'.
But other uniqueness is looking for both column. I mean it should be unique by type1 and type2 columns.(of course first rule is constant)
I try to make it with trigger;
CREATE OR REPLACE trigger my_trigger
BEFORE INSERT OR UPDATE
ON my_table
FOR EACH ROW
DECLARE
lvn_count NUMBER :=0;
lvn_count2 NUMBER :=0;
errormessage clob;
MUST_ACCUR_ONE EXCEPTION;
-- PRAGMA AUTONOMOUS_TRANSACTION; --without this it gives mutating error but I cant use this because it will conflict on simultaneous connections
BEGIN
IF :NEW.type1 = 'GENERIC' THEN
SELECT count(1) INTO lvn_count FROM my_table
WHERE type2= :NEW.type2;
ELSE
SELECT count(1) INTO lvn_count2 FROM my_table
WHERE type1= :NEW.type1 and type2= :NEW.type2;
END IF;
IF (lvn_count >= 1 or lvn_count2 >= 1) THEN
RAISE MUST_ACCUR_ONE;
END IF;
END;
But it gives mutating error without pragma . I do not want to use it due to conflict on simultaneous connections. (error because I use same table on trigger)
I try to make it with unique index but I cant manage.
CREATE UNIQUE INDEX my_table_unique_ix
ON my_table (case when type1= 'GENERIC' then 'some_logic_here' else type1 end, type2); -- I know it does not make sense but maybe there is something different that I can use in here.
Examples;
**Example 1**
insert into my_table (type1,type2) values ('a','b'); -- its ok no problem
insert into my_table (type1,type2) values ('a','c'); -- its ok no problem
insert into my_table (type1,type2) values ('c','b'); -- its ok no problem
insert into my_table (type1,type2) values ('GENERIC','b'); -- it should be error because b is exist before (i look just second column because first column value is 'GENERIC')
EXAMPLE 2:
insert into my_table (type1,type2) values ('GENERIC','b'); -- its ok no problem
insert into my_table (type1,type2) values ('a','c'); -- its ok no problem
insert into my_table (type1,type2) values ('d','c'); -- its ok no problem
insert into my_table (type1,type2) values ('d','b'); -- it should be error because second column can not be same as the second column value that first column value is 'GENERIC'
What you're trying to do is not really straightforward in Oracle. One possible (although somewhat cumbersome) approach is to use a combination of
an additional materialized view with refresh (on commit)
a windowing function to compute the number of distinct values per group
a windowing function to compute the number of GENERIC rows per group
a check constraint to ensure that either we have only one DISTINCT value or we don't have GENERIC in the same group
This should work:
create materialized view mv_my_table
refresh on commit
as
select
type1,
type2,
count(distinct type1) over (partition by type2) as distinct_type1_cnt,
count(case when type1 = 'GENERIC' then 1 else null end)
over (partition by type2) as generic_cnt
from my_table;
alter table mv_my_table add constraint chk_type1
CHECK (distinct_Type1_cnt = 1 or generic_cnt = 0);
Now, INSERTing a duplicate won't fail immediately, but the subsequent COMMIT will fail because it triggers the materialized view refresh, and that will cause the check constraint to fire.
Disadvantages
duplicate INSERTs won't fail immediately (making debugging more painful)
depending on the size of your table, the MView refresh might slow down COMMITs considerably
Links
For a more detailed discussion of this approach, see AskTom on cross-row constraints
Try it like this:
CREATE TABLE my_table (
type1 VARCHAR2(10 CHAR),
type2 VARCHAR2(10 CHAR),
type1_unique VARCHAR2(10 CHAR) GENERATED ALWAYS AS ( NULLIF(type1, 'GENERIC') ) VIRTUAL
);
ALTER TABLE MY_TABLE ADD (CONSTRAINT my_table_unique_ix UNIQUE (type1_unique, type2) USING INDEX)
Or an index like this should also work:
CREATE UNIQUE INDEX my_table_unique_ix ON MY_TABLE (NULLIF(type1, 'GENERIC'), type2);
Or doing it in your style (you only missed the END):
CREATE UNIQUE INDEX my_table_unique_ix ON my_table (case when type1= 'GENERIC' then null else type1 end, type2);
Unless I'm missing something obvious, the logic in the answer from #Frank Schmitt can also be implemented using a statement level trigger. It is a lot simpler to implement and does not have the disadvantages that Frank mentions.
create or replace TRIGGER my_table_t
AFTER INSERT OR UPDATE OR DELETE
ON my_table
DECLARE
l_dummy NUMBER;
MUST_ACCUR_ONE EXCEPTION;
BEGIN
WITH constraint_violated AS
(
select
type1,
type2,
count(distinct type1) over (partition by type2) as distinct_type1_cnt,
count(case when type1 = 'GENERIC' then 1 else null end)
over (partition by type2) as generic_cnt
from my_table
)
SELECT 1 INTO l_dummy
FROM constraint_violated
WHERE NOT (distinct_type1_cnt = 1 or generic_cnt = 0) FETCH FIRST 1 ROWS ONLY;
RAISE MUST_ACCUR_ONE;
EXCEPTION WHEN NO_DATA_FOUND THEN
NULL;
END;
/
I have an error I'm failing to understand. This is happening on some production code, but I created dummy tables to simulate the issue.
create table dl_test_nullable_src(col1 varchar2(20) NOT NULL, col2 varchar2(20) NOT NULL);
create table dl_test_nullable_tgt(col1 varchar2(20) NOT NULL, col2 varchar2(20) NOT NULL);
begin
for i in 1..10 loop
insert into dl_test_nullable_src(col1,col2) values(i*100,i*300);
insert into dl_test_nullable_tgt(col1,col2) values(i*100,i*300);
end loop;
commit;
end;
The below works
merge into dl_test_nullable_tgt tgt
using
(select * from dl_test_nullable_src) src
on (tgt.col1 = src.col1)
when matched then update
set tgt.col2 = src.col2
when not matched then insert
(tgt.col2)
values
(src.col2);
10 rows merged.
Now I add some parallel and append to direct path load any NEW records.
alter session enable parallel dml;
merge /*+ append parallel(tgt,4) */ into dl_test_nullable_tgt tgt
using
(select * from dl_test_nullable_src) src
on (tgt.col1 = src.col1)
when matched then update
set tgt.col2 = src.col2
when not matched then insert
(tgt.col2)
values
(src.col2);
SQL Error: ORA-01400: cannot insert NULL into ("PBTUSER5"."DL_TEST_NULLABLE_TGT"."COL1")
01400. 00000 - "cannot insert NULL into (%s)"
*Cause: An attempt was made to insert NULL into previously listed objects.
*Action: These objects cannot accept NULL values.
Even though this doesn't make sense on its own, it should not try to be inserting any records since everything that's in the SRC table is in the TGT table. The only way to bypass this is to also add col1 in the insert and values clause.
Table Contents
as far as I can see this is described in MOS document 1547251.1 - and classified as "not a bug, but a feature". The difference is based on the evaluation of contraints in parallel and non-parallel execution.
I have a PL/SQL package in Oracle that its important function is :
function checkDuplicate(in_id in varchar2) return boolean is
cnt number;
begin
select count(*)
into cnt
from tbl_Log t
where t.id = in_id
if (cnt > 0) then
// It means the request is duplicate on in_id
return false;
end if;
insert into tbl_log (id,date) values(in_id , sysdate);
return true;
end;
When two requests call this function concurrently, both of them passed this function and two the same in_id inserted in tbl_log.
Note: tbl_log doesn't have a PK for performance issues.
Are there any solutions?
" both of them passed this function and two the same in_id inserted in tbl_log"
Oracle operates at the READ COMMITTED isolation level, so the select can only find committed records. If one thread has inserted a record for a given value but hasn't committed the transaction another thread looking for the same value will come up empty.
"Note: tbl_log doesn't have a PK for performance issues. "
The lessons of history are clear: tables without integrity constraints inevitably fall into data corruption.
"I want to recognize the duplication with this function ... Are there any solutions?"
You mean apart from adding a primary key constraint? There is no more efficient way of trapping duplication than a primary key. Maybe you should look at the performance issues. Plenty of applications mange to handle millions of inserts and still enforce integrity constraints. You should also look at the Java layer: why have you got multiple threads submitting the same ID?
Note: tbl_log doesn't have a PK for performance issues.
There is no PK nor unique index on this column in order to "avoid performance issues", but there are hundreds or thousands queries like SELECT ... WHERE t.id = .. running against this table. These queries must use a full table scan due to lack of index on this column !!!!
This can cause much bigger performance issues in my opinion.
Since the values of this columns are UUIDs, then there is a very little chance of conflicted values. In this case I would prefer not to use any locks.
Just use an unique constraint (index) on this column to prevent from inserting two duplicate values.
ALTER TABLE tbl_log ADD CONSTRAINT tbl_log_id_must_be_unique UNIQUE( id );
and then use this implementation of your function:
create or replace function checkDuplicate(in_id in varchar2) return boolean is
begin
insert into tbl_log (id,"DATE") values(in_id , sysdate);
return true;
exception when dup_val_on_index then
return false;
end;
/
In the vast majority of cases the function simply inserts a new record to the table without any delay because values are UUIDs.
In seldom cases of duplicated values, when the value is already commited in the table, the insert will immediatelly fail, without any delay.
In very very rare cases (almost impossible) when two threads are trying to simultanously insert the same UUID, the second thread will be held on INSERT command and will wait some time until the first thread will commit or rollback.
As per your condition, since you are reluctant to use Primary key data integrity enforcement( which will lead to data corruption anyhow ), i would suggest that you can use MERGE statment and keep an audit log for the latest thread updating the table. This way you will be able to eliminate the entry of duplicate record as well as keep a track of when and from which thread (latest info) the id got updated. Hope the below snippet helps.
---Create dummy table for data with duplicates
DROP TABLE dummy_hist;
CREATE TABLE dummy_hist AS
SELECT LEVEL COL1,
'AVRAJIT'
||LEVEL COL2,
SYSTIMESTAMP ACTUAL_INSERTION_DT,
SYSTIMESTAMP UPD_DT,
1 thread_val
FROM DUAL
CONNECT BY LEVEL < 100;
--Update upd_dt
UPDATE dummy_hist SET upd_dt = NULL,thread_val = NULL;
SELECT * FROM dummy_hist;
--Create function
CREATE OR REPLACE
FUNCTION checkDuplicate(
in_id IN VARCHAR2,
p_thread_val IN NUMBER)
RETURN BOOLEAN
IS
cnt NUMBER;
BEGIN
MERGE INTO dummy_hist A USING
(SELECT in_id VAL FROM dual
)B ON (A.COL1 = B.VAL)
WHEN MATCHED THEN
UPDATE
SET a.upd_dt = systimestamp,
a.thread_val = p_thread_val
WHERE a.col1 = b.val WHEN NOT MATCHED THEN
INSERT
(
a.col1,
a.col2,
a.actual_insertion_dt,
a.UPD_DT,
a.thread_val
)
VALUES
(
b.val,
'AVRAJIT',
SYSTIMESTAMP,
NULL,
p_thread_val
);
COMMIT;
RETURN true;
END;
/
--Execute the fucntion
DECLARE
rc BOOLEAN;
BEGIN
FOR I IN
(SELECT LEVEL LVL FROM DUAL CONNECT BY LEVEL BETWEEN 8 AND 50
)
LOOP
rc:=checkduplicate(I.LVL,3);
END LOOP;
END;
/
I want to have an 'updateinfo' table in order to record every update/insert/delete operations on another table.
In oracle I've written this:
CREATE TABLE updateinfo ( rnumber NUMBER(10), tablename VARCHAR2(100 BYTE), action VARCHAR2(100 BYTE), UPDATE_DATE date )
DROP TRIGGER TRI_TABLE;
CREATE OR REPLACE TRIGGER TRI_TABLE
AFTER DELETE OR INSERT OR UPDATE
ON demo
REFERENCING NEW AS NEW OLD AS OLD
FOR EACH ROW
BEGIN
if inserting then
insert into updateinfo(rnumber,tablename,action,update_date ) values(rownum,'demo', 'insert',sysdate);
elsif updating then
insert into updateinfo(rnumber,tablename,action,update_date ) values(rownum,'demo', 'update',sysdate);
elsif deleting then
insert into updateinfo(rnumber,tablename,action,update_date ) values(rownum,'demo', 'delete',sysdate);
end if;
-- EXCEPTION
-- WHEN OTHERS THEN
-- Consider logging the error and then re-raise
-- RAISE;
END TRI_TABLE;
but when checking updateinfo, all rnumber column is zero.
is there anyway to retrieve the correct row number?
The only option is to use primary key column of your "demo" table.
ROWNUM is not what you are looking for, read the explanation.
ROWID looks like a solution, but in fact it isn't, because it shouldn't be stored for a later use.
ROWNUM is not what you think it is. ROWNUM is a counter that has only a meaning within the context of one execution of a statement (i.e. the first resulting row always has rownum=1 etc.). I guess you are looking for ROWID, which identifies a row.
Say I have an Oracle PL/SQL block that inserts a record into a table and need to recover from a unique constraint error, like this:
begin
insert into some_table ('some', 'values');
exception
when ...
update some_table set value = 'values' where key = 'some';
end;
Is it possible to replace the ellipsis for something in order to catch an unique constraint error?
EXCEPTION
WHEN DUP_VAL_ON_INDEX
THEN
UPDATE
I'm sure you have your reasons, but just in case... you should also consider using a "merge" query instead:
begin
merge into some_table st
using (select 'some' name, 'values' value from dual) v
on (st.name=v.name)
when matched then update set st.value=v.value
when not matched then insert (name, value) values (v.name, v.value);
end;
(modified the above to be in the begin/end block; obviously you can run it independantly of the procedure too).
I suspect the condition you are looking for is DUP_VAL_ON_INDEX
EXCEPTION
WHEN DUP_VAL_ON_INDEX THEN
DBMS_OUTPUT.PUT_LINE('OH DEAR. I THINK IT IS TIME TO PANIC!')
As an alternative to explicitly catching and handling the exception you could tell Oracle to catch and automatically ignore the exception by including a /*+ hint */ in the insert statement. This is a little faster than explicitly catching the exception and then articulating how it should be handled. It is also easier to setup. The downside is that you do not get any feedback from Oracle that an exception was caught.
Here is an example where we would be selecting from another table, or perhaps an inner query, and inserting the results into a table called TABLE_NAME which has a unique constraint on a column called IDX_COL_NAME.
INSERT /*+ ignore_row_on_dupkey_index(TABLE_NAME(IDX_COL_NAME)) */
INTO TABLE_NAME(
INDEX_COL_NAME
, col_1
, col_2
, col_3
, ...
, col_n)
SELECT
INDEX_COL_NAME
, col_1
, col_2
, col_3
, ...
, col_n);
This is not a great solution if your goal it to catch and handle (i.e. print out or update the row that is violating the constraint). But if you just wanted to catch it and ignore the violating row then then this should do the job.