My question is how oracle treats an INSERT transaction before issuing a COMMIT.
While I am doing an INSERT transaction, will oracle wait until I have inserted all my records within that procedure and then when I issue a COMMIT statement will the records be saved in a sequence for this transaction?
In the following code, the first insert that is made is the number of rows (metadata) and then the cursor loops and starts inserting the actual data.
Is there a possibility, in one transaction when I call this procedure, first my metadata record is inserted and then some other data (not related to this transaction) be inserted and then rest of my data. So that, the first record and the rest of the records from the loop are not inserted in a Sequence.
-- This code belongs to proecdure when ever a user clicks on insert
-- button from the front end form
DECLARE
rowcnt NUMBER;
CURSOR c_get_employ IS
SELECT EMP.EMPLOYER_ID, EMP.EMPLOYER_NAME, EMP.EMPLOYER_LOCATION
FROM EMP
WHERE EMP.EMPLOYER_COUNTRY = 'USA'
ORDER BY EMP.EMPLOYER_ID;
BEGIN
Select count(*)
INTO rowcnt
FROM EMP
WHERE EMP.EMPLOYER_COUNTRY = 'USA'
ORDER BY EMP.EMPLOYER_ID;
-- I want to insert the 'number of employee records' that will be inserted (metadata)
INSERT INTO EMP_OUTPUT
(EMPID, EMPNAME, EMPLOC, ECOUNT)
VALUES
(,,,rowcnt);
-- Then loop through the cursor and start inserting the data
FOR c_post_employ IN c_get_employ LOOP
INSERT INTO EMP_OUTPUT
(EMPID, EMPNAME, EMPLOC)
VALUES
(c_post_employ.EMPLOYER_ID,c_post_employ.EMPLOYER_NAME,c_post_employ.EMPLOYER_LOCATION);
END LOOP;
COMMIT;
END;
Another transaction can perform inserts concurrently to your transaction, but your transaction won't see them:
until the other transaction commits (if your transaction is using READ COMMITTED isolation), or
ever (when using SERIALIZABLE isolation) - you'll need to start another transaction to see them.
Whether this will yield a correct behavior, is for you to decide.
Just be careful about SELECT COUNT(*) ... - it may not return what you expect. Consider the following scenario:
The EMP table is initially empty.
Transaction A starts and inserts a row in EMP, but does not commit.
Transaction B starts and inserts a row in EMP, but does not commit.
Transaction A executes SELECT COUNT(*) FROM EMP and gets 1 (because it sees its own newly inserted row, but does not see B's newly inserted row since B did not commit yet).
Transaction B executes SELECT COUNT(*) FROM EMP and also gets 1 (for the same reason but in reverse).
Transaction A inserts 1 into EMP_OUTPUT and commits.
Transaction B inserts 1 into EMP_OUTPUT and commits (assuming there is no key violation).
So, 1 is inserted despite table actually having 2 rows!
Unfortunately not even Oracle's SERIALIZABLE isolation will save you from this kind of anomaly. Pretty much the only way to guarantee the "correct" result if to lock the entire table, so no concurrent inserts (or deletes) can occur.
Use a single SQL statement if possible. It will have statement-level read consistency, and will be much faster.
insert into emp_output(empid, empname, emploc, ecount)
with employees as
(
select employer_id, employee_name, employer_location
from emp
where employer_country = 'USA'
order by employer_id
)
select null, null, null, count(*) from employees
union all
select employer_id, employee_name, employer_location, null from employees;
The term you want to google for is "read consistency":
http://docs.oracle.com/cd/B12037_01/server.101/b10743/consist.htm
Bottom line:
As you know, if you rollback, it's as though the inserts "never happened"
However, other stuff can (and probably did) "happen" in the meantime.
You need to run in the Serializable Isolation Level:
http://docs.oracle.com/cd/E11882_01/server.112/e16508/consist.htm#BABCJIDI
"Serializable transactions see only those changes that were committed at the time the transaction began, plus those changes made by the transaction itself through INSERT, UPDATE, and DELETE statements. Serializable transactions do not experience nonrepeatable reads or phantoms."
Related
I have a spring batch program that reads data from one database then processes the data then writes to an Oracle database. This batch program runs on a schedule once a day. How can I avoid adding the same record each time it runs and only new values from the source DB?
One option is to create unique index (or primary key, if possible (depending on whether you want to allow nulls or not)) which will cause Oracle to automatically reject all rows whose column(s) violate uniqueness.
Other options requires some programming.
[EDIT: "silently" skip errors]
This is what I meant:
for cur_r in (select col1, col2
from another_table#db_link)
loop
begin --> you need inner BEGIN-EXCEPTION-END block
insert into new_table (col1, col2)
values (cur_r.col1, cur_r.col2);
exception
when dup_val_on_index then
null;
end;
end loop;
Another option uses pure SQL (i.e. no PL/SQL's loop):
insert into new_table (col1, col2)
select col1, col2
from another_table#db_link
where (col1, col2) not in (select col1, col2
from new_table);
This option doesn't even require unique index (which wouldn't harm, though) because NOT IN won't insert rows whose column values already exist in the target table.
It sounds like you're concerned about not processing the same source record multiple times. If that's the case you can add a field on your source table indicating that the data has already been extracted.
Oh, and - put a unique primary key on your tables. All of them. Even the ones where you don't think you need it. The primary key you add today is the one where you WON'T say at a later date, "Damn. I wish that table had a primary key". Don't ask me how I know...
Best of luck.
I guess you are using RepositoryItemReader as the source. If that is the case, you can add a custom method in the source Repository, including a validation condition to avoid already processed records, using #Query, and then go with that method in the RepositoryItemReader.
It would be something like
#Query("SELECT r FROM Records r WHERE r.isNew = 1")
Collection<Record> findAllNewRecords();
And then configure the reader like
RepositoryItemReader<Record> recordsReader = new RepositoryItemReader<>();
recordsReader.setRepository(recordsRepository);
recordsReader.setMethodName("findAllNewRecords");
Hope it helps
create table T_UNIQUE_VALUE( a number,b number, c varchar2(100));
alter table T_UNIQUE_VALUE ADD CONSTRAINT C_UNIQUE_VALUE UNIQUE (a,b);
Define log error table. Oralce will create err$_table_name
BEGIN
DBMS_ERRLOG.create_error_log(dml_table_name => 'T_UNIQUE_VALUE');
END;
Test it.
execut 2 times
insert into T_UNIQUE_VALUE values(1,1,'a') LOG ERRORS REJECT LIMIT UNLIMITED;
--check table T_UNIQUE_VALUE and err$_T_UNIQUE_VALUE
select * from err$_T_UNIQUE_VALUE; -- 1 row
select * from T_UNIQUE_VALUE; -- 1 row
Modify spring annotation.
#Modifying
#Query(value = "insert into T_UNIQUE_VALUE values(:a,:b,:c) LOG ERRORS REJECT LIMIT UNLIMITED", nativeQuery = true)
void insert_balbla(...)
I have the following situation:
A table (MyTable) should be processed (updates/inserts/deletes etc) by a batch process (a call to a myplsql() procedure).
During myplsql execution no one should touch MyTable - so MyTable is locked in exclusive mode by myplsql.
Now MyTable has a number of on insert,on update, on delete triggers defined but those are not needed while performing batch processing - moreover they slow down the batch process extremely.
So the solution is to disable the triggers before calling myplsql().
But how to avoid someone touching the MyTable just after alter table ... disable trigger is performed and before myplsql manages to lock the table,
given that alter table performs implicit commit - so any lock acquired before that will be lost anyway?
Part of the problem is that I do not control the other code or the other user that could try to touch the Table.
In a few words I need to perform the following in a single shot:
Lock MyTable
Disable Triggers (somehow without loosing the lock)
Process MyTable
Enable Triggers
Unlock MyTable
One thought was to remove grants from the table - and render it unusable for other users.
But as it is turns out - that is not an option as the other processes/users perform their operations logged in as a table owner user.
Thanks.
A slightly different approach is to keep the triggers enabled but reduce (if not quite entirely remove) their impact, by adding a when clause something like:
create or replace trigger ...
...
for each row
when (sys_context('userenv', 'client_info') is null
or sys_context('userenv', 'client_info') != 'BATCH')
declare
...
begin
...
end;
/
Then in your procedure add a call at the start as your 'disable triggers' step:
dbms_application_info.set_client_info('BATCH');
and clear it again at the end, just in case the session is left alive and reused (so you might want to do this in an exception handler too):
dbms_application_info.set_client_info(null);
You could also use module, or action, or a combination. While that setting is in place the trigger will still be evaluated but won't fire, so any thing happening inside will skipped - the trigger body does not run, as the docs put it.
This isn't foolproof as there is nothing really stopping other users/applications making the same calls, but if you pick a more descriptive string and/or a combination of settings, it would have to be deliberate - and I think you're mostly worried about accidents not bad actors.
Quick speed test with a pointless trigger that does just slows things down a bit.
create table t42 (id number);
-- no trigger
insert into t42 (id) select level from dual connect by level <= 10000;
10,000 rows inserted.
Elapsed: 00:00:00.050
create or replace trigger tr42 before insert on t42 for each row
declare
dt date;
begin
select sysdate into dt from dual;
end;
/
-- plain trigger
insert into t42 (id) select level from dual connect by level <= 10000;
10,000 rows inserted.
Elapsed: 00:00:00.466
create or replace trigger tr42 before insert on t42 for each row
when (sys_context('userenv', 'client_info') is null
or sys_context('userenv', 'client_info') != 'BATCH')
declare
dt date;
begin
select sysdate into dt from dual;
end;
/
-- userenv trigger, not set
insert into t42 (id) select level from dual connect by level <= 10000;
10,000 rows inserted.
Elapsed: 00:00:00.460
- userenv trigger, set to BATCH
exec dbms_application_info.set_client_info('BATCH');
insert into t42 (id) select level from dual connect by level <= 10000;
10,000 rows inserted.
Elapsed: 00:00:00.040
exec dbms_application_info.set_client_info(null);
There's a bit of variation from making remote calls, but I ran a few times and it's clear that running with a plain trigger is very similar to running with the constrained trigger without BATCH set, and both are much slower than running without a trigger or with the constrained trigger with BATCH set. In my testing there's an order of magnitude difference.
I have created trigger TEST_TRIG as below:
CREATE TRIGGER TEST_TRIG
AFTER INSERT ON TEST_TABLE
FOR EACH ROW
DECLARE
PRAGMA AUTONOMOUS_TRANSACTION;
BEGIN
TEST_PROC();
END;
Procedure TEST_PROC code:
create or replace
PROCEDURE TEST_PROC
AS
BEGIN
EXECUTE IMMEDIATE 'truncate table TEST_FINAL';
INSERT INTO TEST_FINAL select * from TEST_TABLE;
commit;
END;
Initially, I disabled TRIGGER TEST_TRIG and inserted a record into TEST_TABLE and executed PROCEDURE TEST_PROC manually.
Output: I was able to fetch the same record what i inserted into TEST_TABLE from TEST_FINAL.
I flushed those records from both table and enabled the trigger TEST_TRIG.
Now when i inserts and commits the record in TEST_TABLE, I didn't found the record in TEST_FINAL table... I haven't received any error message also!!!
So I want to know whether trigger got fired or not?
I don't think you have fully grasped the implications of AUTONOMOUS_TRANSACTION. Effectively it means the code bounded by the pragma runs in a separate session . So, because of Oracle's read consistent isolation level, the autonomous transaction cannot see any of the data changes generated by the main transaction.
Thus, if TEST_TABLE is empty when you start the trigger will insert no rows into TEST_FINAL, regardless of how many rows you're inserting right now.
So: don't flush both tables. Insert some rows into TEST_TABLE and commit. TEST_FINAL will still be empty. Insert some more rows into TEST_TABLE and, lo! the first set of rows will appear in TEST_FINAL.
Obviously this is not the result you want. So you need to revisit your logic. It really doesn't make sense to truncate TEST_FINAL every time and definitely not FOR EACH ROW. That is Teh Suck! as far as performance goes. Likewise and for the same reason it doesn't make sense to populate the target table with INSERT ... SELECT .
Discarding the TRUNCATE means you don't need the pragma and everything becomes much simpler,
If you want to keep a history of the affected rows use something like this instead:
CREATE TRIGGER TEST_TRIG
AFTER INSERT ON TEST_TABLE
FOR EACH ROW
BEGIN
insert into test_final (col1, col2)
values (:new.col1, :new.col2);
END;
You'll need to change the exact code to fit your exact requirements.
Using Oracle, is it possible to indicate which rows are currently locked (and which are not) when performing a select statement (I don't want to lock any rows, just be able to display which are locked)?
For example, a pseudo column that would return the lock/transaction against the row:
SELECT lockname FROM emp;
One thing you could do is this - although it is not terribly efficient and so I wouldn't want to do use it for large data sets. Create a row-level function to try and lock the row. If it fails, then the row is already locked
CREATE OR REPLACE FUNCTION is_row_locked (v_rowid ROWID, table_name VARCHAR2)
RETURN varchar2
IS
x NUMBER;
PRAGMA AUTONOMOUS_TRANSACTION;
BEGIN
EXECUTE IMMEDIATE 'Begin
Select 1 into :x from '
|| table_name
|| ' where rowid =:v_rowid for update nowait;
Exception
When Others Then
:x:=null;
End;'
USING OUT x, v_rowid;
-- now release the lock if we got it.
ROLLBACK;
IF x = 1
THEN
RETURN 'N';
ELSIF x IS NULL
THEN
RETURN 'Y';
END IF;
END;
/
And then you could
Select field1, field2, is_row_locked(rowid, 'MYTABLE') from mytable;
It will work, but it isn't pretty nor efficient.
Indeed, it has exactly one redeeming quality - it will work even if you don't have select privs on the various v$ tables required in the linked document. If you have the privs, though, definitely go the other route.
is it possible to indicate which rows are currently locked (and which are not) when performing a select statement
A SELECT statement will never lock any rows - unless you ask it to by using FOR UPDATE.
If you want to see locks that are held due to a SELECT ... FOR UPDATE (or a real update), you can query the v$lock system view.
See the link that OMG Pony posted for an example on how to use that view.
I think #Michael Broughton's answer is the only way that will always work. This is because V$LOCK is not accurate 100% of the time.
Sessions don't wait for a row, they wait for the end of the transaction that modified that row. Most of the time those two concepts are the same thing, but not when you start using savepoints.
For example:
Session 1 creates a savepoint and modifies a row.
Session 2 tries to modify that same
row, but sees session 1 already has that row,
and waits for session 1 to finish.
Session 1 rolls back to the
savepoint. This removes its entry
from the ITL but does not end the
transaction. Session 2 is still
waiting on session 1. According to
V$LOCK session 2 is still waiting on
that row, but that's not really true
because now session 3 can modify that
row. (And if session 1 executes a
commit or rollback, session 2 will
wait on session 3.)
Sorry if that's confusing. You may want to step through the link provided by OMG Ponies, and then try it again with savepoints.
We have an Oracle database, and the customer account table has about a million rows. Over the years, we've built four different UIs (two in Oracle Forms, two in .Net), all of which remain in use. We have a number of background tasks (both persistent and scheduled) as well.
Something is occasionally holding a long lock (say, more than 30 seconds) on a row in the account table, which causes one of the persistent background tasks to fail. The background task in question restarts itself once the update times out. We find out about it a few minutes after it happens, but by then the lock has been released.
We have reason to believe that it might be a misbehaving UI, but haven't been able to find a "smoking gun".
I've found some queries that list blocks, but that's for when you've got two jobs contending for a row. I want to know which rows have locks when there's not necessarily a second job trying to get a lock.
We're on 11g, but have been experiencing the problem since 8i.
Oracle's locking concept is quite different from that of the other systems.
When a row in Oracle gets locked, the record itself is updated with the new value (if any) and, in addition, a lock (which is essentially a pointer to transaction lock that resides in the rollback segment) is placed right into the record.
This means that locking a record in Oracle means updating the record's metadata and issuing a logical page write. For instance, you cannot do SELECT FOR UPDATE on a read only tablespace.
More than that, the records themselves are not updated after commit: instead, the rollback segment is updated.
This means that each record holds some information about the transaction that last updated it, even if the transaction itself has long since died. To find out if the transaction is alive or not (and, hence, if the record is alive or not), it is required to visit the rollback segment.
Oracle does not have a traditional lock manager, and this means that obtaining a list of all locks requires scanning all records in all objects. This would take too long.
You can obtain some special locks, like locked metadata objects (using v$locked_object), lock waits (using v$session) etc, but not the list of all locks on all objects in the database.
you can find the locked tables in Oracle by querying with following query
select
c.owner,
c.object_name,
c.object_type,
b.sid,
b.serial#,
b.status,
b.osuser,
b.machine
from
v$locked_object a ,
v$session b,
dba_objects c
where
b.sid = a.session_id
and
a.object_id = c.object_id;
Rather than locks, I suggest you look at long-running transactions, using v$transaction. From there you can join to v$session, which should give you an idea about the UI (try the program and machine columns) as well as the user.
Look at the dba_blockers, dba_waiters and dba_locks for locking. The names should be self explanatory.
You could create a job that runs, say, once a minute and logged the values in the dba_blockers and the current active sql_id for that session. (via v$session and v$sqlstats).
You may also want to look in v$sql_monitor. This will be default log all SQL that takes longer than 5 seconds. It is also visible on the "SQL Monitoring" page in Enterprise Manager.
The below PL/SQL block finds all locked rows in a table. The other answers only find the blocking session, finding the actual locked rows requires reading and testing each row.
(However, you probably do not need to run this code. If you're having a locking problem, it's usually easier to find the culprit using GV$SESSION.BLOCKING_SESSION and other related data dictionary views. Please try another approach before you run this abysmally slow code.)
First, let's create a sample table and some data. Run this in session #1.
--Sample schema.
create table test_locking(a number);
insert into test_locking values(1);
insert into test_locking values(2);
commit;
update test_locking set a = a+1 where a = 1;
In session #2, create a table to hold the locked ROWIDs.
--Create table to hold locked ROWIDs.
create table locked_rowids(the_rowid rowid);
--Remove old rows if table is already created:
--delete from locked_rowids;
--commit;
In session #2, run this PL/SQL block to read the entire table, probe each row, and store the locked ROWIDs. Be warned, this may be ridiculously slow. In your real version of this query, change both references to TEST_LOCKING to your own table.
--Save all locked ROWIDs from a table.
--WARNING: This PL/SQL block will be slow and will temporarily lock rows.
--You probably don't need this information - it's usually good enough to know
--what other sessions are locking a statement, which you can find in
--GV$SESSION.BLOCKING_SESSION.
declare
v_resource_busy exception;
pragma exception_init(v_resource_busy, -00054);
v_throwaway number;
type rowid_nt is table of rowid;
v_rowids rowid_nt := rowid_nt();
begin
--Loop through all the rows in the table.
for all_rows in
(
select rowid
from test_locking
) loop
--Try to look each row.
begin
select 1
into v_throwaway
from test_locking
where rowid = all_rows.rowid
for update nowait;
--If it doesn't lock, then record the ROWID.
exception when v_resource_busy then
v_rowids.extend;
v_rowids(v_rowids.count) := all_rows.rowid;
end;
rollback;
end loop;
--Display count:
dbms_output.put_line('Rows locked: '||v_rowids.count);
--Save all the ROWIDs.
--(Row-by-row because ROWID type is weird and doesn't work in types.)
for i in 1 .. v_rowids.count loop
insert into locked_rowids values(v_rowids(i));
end loop;
commit;
end;
/
Finally, we can view the locked rows by joining to the LOCKED_ROWIDS table.
--Display locked rows.
select *
from test_locking
where rowid in (select the_rowid from locked_rowids);
A
-
1
Given some table, you can find which rows are not locked with SELECT FOR UPDATESKIP LOCKED.
For example, this query will lock (and return) every unlocked row:
SELECT * FROM mytable FOR UPDATE SKIP LOCKED
References
Ask TOM "How to get ROWID for locked rows in oracle".