I have multiple threads that are processing rows from the same table that is - in fact - a queue.
I want that each row will be handled by one thread only. So, I've added a column - "IsInProccess", and in the threads SELECT statement I've added "WHERE IsInProccess = 0".
In addition I use "SELECT FOR UPDATE" so after a thread get a row from the table no other thread will get it before it puts 1 in "IsInProccess".
The problem is that I have many threads and in many times the following scenario happens:
Thread A selecting by "SELECT FOR UPDATE" form the table and getting row no. 1. Before it changes IsInProccess to 1, thread B selects in the same way from the table and get row no 1 too. Oracle save row no. 1 to thread A session and thread B can't change this row and return an error - "Fail to be fetched".
I want that when some thread select from the table Oracle will return rows that are no saved to other open session.
Can I do that?
This is a sketch of a solution that I've seen used very successfully before:
Use the SELECT FOR UPDATE NOWAIT syntax so that if the session cannot get a lock on a row immediately it raises an exception instead of waiting for the lock. The exception handler could wait a few seconds (e.g. with dbms_lock.sleep); the whole block could be wrapped in a loop which tries again a certain number of times before it gives up.
Add WHERE ROWNUM<=n to the query so that it only tries to get a certain number of rows (e.g. 1) at a time; if it succeeds, update the rows as "in process".
A good way to mark rows as "in process" (that I've seen used successfully) is to have two columns - SID and SERIAL# - and update these columns with the SID and SERIAL# for the current session.
In case a session fails while it has the rows marked as "in process", another process could "clean up" the rows that were marked as "in process" by searching for any rows that have SID/SERIAL# that are not found as active sessions in v$session.
Oracle have already solved this for you: use the Advanced Queueing API
If you have 11g, look at SKIP LOCKED
It is there, but undocumented (and therefore unsupported and maybe buggy) in 10g.
That way, when Session A locks the row, Session B can skip it and process the next.
One solution:
Put the select and update queries into a transaction. This sort of problem is exactly why transactions were invented.
You also need to worry about "orphan" rows - e.g. a thread picks up the row and then dies without finishing the work. To solve that, one solution is to have 2 columns: "IsInProcess" and "StartprocessingTime".
isInProcess will have 3 values: 0 (not processed), 1 (picked up), 2 (done).
The original transaction will set the row's "isInProcess" to 1 and "StartprocessingTime" to something else, but the select will also add this to where clause (assuming you can specify a valid timeout period)
"WHERE isInProcess = 0 OR (isInProcess = 1 AND StartprocessingTime < getdate()-timeout)".
Please note that the syntax above is not ORACLE, just pseudo-code.
Related
I have 5 rows in table. And some of the rows are locked in some sessions .
I don't want to generate any error, Just want to wait until any row will become free for further processing
I tired with nowait and skip locked:-
nowait , But there is a problem with nowait. query has written in cursor , when I used "nowait" under cursor , query will return null and control will go out with an error by saying- resource busy
I tried with skip locked with for update-But if table contain 5 rows and all
5 rows are locked then it is giving error.
CURSOR cur_name_test IS
SELECT def.id , def.name
FROM def_map def
WHERE def.id = In_id
FOR UPDATE skip locked;
Why don't use select for update only ? The below is being test locally in plsql developer.
in first sesssion I do the below
SELECT id , name
FROM ex_employee
FOR UPDATE;
in second session i run the below however it hang.
SET serveroutput ON size 2000
/
begin
declare curSOR cur_name_test IS
SELECT id , name
FROM ex_employee
WHERE id = 1
FOR UPDATE ;
begin
for i in cur_name_test loop
dbms_output.put_line('inside cursor');
end loop;
end;
end;
/
commit
/
when I commit in the first session , the lock will be released and the second session will do its work. i guess that you want , infinite wait.
However such locking mechanism (pessimistic locking) can lead to deadlocks if its not managed correctly and carefully ( first session waiting second session , and second session waiting first session).
As for the nowait its normal to have error resource busy, because you are saying for the query don't wait if there are locking. you can instead wait 30, which will wait 30 second then output error but thats not what you want(I guess).
As for skip locked, the select will skip the locked data for example you have 5 rows , and one of them is locked then the select will not read this row . thats why when all the data is locked its throwing error because nothing can be skipped. and I guess is not what you want in your scenario.
This sounds like you need to think about transaction control.
If you are doing work in a transaction then the implication is that that unit of work needs to complete in order to be valid.
What you are saying is that some of the work in my update transaction doesn't need to complete in order for the transaction to be committed.
Not only that but you have two transactions running at the same time performing operations against the same object. In itself that may be valid but
if it is then you really need to go back to the first sentence and think hard about transaction control and process flow and see if there's a way you can have the second transaction only attempt to update rows that aren't being updated in the first transaction.
A VB 6 program is processing records and inserting in a temporary table, then these records are moved from this temporary table to actual table as
connection.Execute "INSERT INTO MAIN_TABLE SELECT * FROM TEMP_TABLE"
The temporary table is then truncated when records are moved
connection.Execute "TRUNCATE TABLE TEMP_TABLE"
This is working fine untill I use PARALLEL hint for INSERT query. I receive this error on TRUNCATE
ORA-00054: resource busy and acquire with NOWAIT specified or timeout
expired
It looks to me that parallel query returns before completing the job and TRUNCATE command is issued causing the lock.
I checked the number of records inserted as below and found that it is far less than the number of records in temporary table
connection.Execute "INSERT /*+ PARALLEL */ INTO MAIN_TABLE SELECT * FROM TEMP_TABLE", recordsAffected
Is there any way to wait for INSERT to complete?
Delete may be slower but Truncate is DDL which you can't run at the same time as DML. In fact, Truncate requires exclusive access to the table. DML on tables will request a share mode lock on the table which means you can't do DDL against the table at the same time.
A possible alternate solution would be to use synonyms. You have your table A
and a synonym S pointing to A
create table B as select * from A where 1=0;
create or replace synonym S for B
Your app now uses B instead of A so you can do what you want with A.
Do this every time you want to "truncate"
This assumes you're using ADO - though I now notice you don't have
that tag in your question.
Can you monitor the connection state with a loop waiting for executing to finish?
Something like
EDIT - Fix Boolean Add to use + instead of "AND"
While Conn.State = (adStateOpen + adStateExecuting)
DoEvents
Sleep 500 ' uses Sleep API command to delay 1/2 second
Wend
Sleep API declare
Edit - Add Asynch Hint/Option
Also - it might help the ADO connection to give it a hint that its running asynchronously, by adding the adAsyncExecute to end of your execute command
ie. Change the execute sql command to look like
conn.execute sqlString, recordsaffected, adAsyncExecute
I'm trying to see whether I can use database lock to deal with race conditions. For example
CREATE TABLE ORDER
(
T1_ID NUMBER PRIMARY KEY,
AMT NUMBER,
STATUS1 CHAR(1),
STATUS2 CHAR(1),
UPDATED_BY VARCHAR(25)
);
insert into order values (order_seq.nextval, 1, 'N', 'N', 'U0');
Later two users can update the order record at the same time. Requirement is that only one can proceed while the other should NOT. We can certainly use a distributed lock manager (DLM) to do this but I figure database lock may be more efficient.
User 1:
update T1 set status1='Y', updated_by='U1' where status1='N';
User 2:
update T1 set status2='Y', updated_by='U2' where status1='N';
Two users are doing these at the same time. Ideally only one should be allowed to proceed. I played using Sql Plus and also wrote a little java test program letting two threads do these simultaneously. I got the same result. Let's say User 1 got the DB row lock first. It returns 1 row updated. The second session will be blocked waiting for the row lock before the 1st session commits or rollbacks. The question is REALLY this:
Update with a where clause seems like two operations: first it will do an implicit select based on the where clause to pick the row that will be updated. Since Oracle only supports READ COMMITTED isolation level, I expect both UPDATE statements will pick the single record in the DB. As a result, I expected both UPDATE statement will eventually return "1 row updated" although one will wait till the other transaction commits. HOWEVER that's not what I saw. The second UPDATE returns "0 row updated" after the first commits. I feel that Oracle actually runs the where clause AGAIN after the first session commits, which results in "0 row updated" result.
This is strange to me. I thought I would run into the classical "lost update" phenomenon.
can somebody please explain what's going on here? Thanks very much!
Our application reads a record from an Oracle 'Event' table. When the event record exists we update the 'count' field of that record. If the record doesn't exist we insert it. So we want only 1 record for a particular event in the table.
The problem with this is probably quite predictable: one application thread will read the table, see the event is not there, insert the new event and commit. But before it commits a second thread will also read the table and see the event is not there. And then both threads will insert the event and we end up with 2 records for the same event.
I guess synchronizing access to this particular method in our application will prevent this problem, but what is the best option in Oracle to prevent this? Will MERGE for example always prevent this problem?
Serialising access to the procedure that implements this functionality would be trivial to implement, using DBMS_LOCK to define and take an exclusive lock.
Serialising through SQL based methods is practically impossible, due to the read consistency model.
CREATE TABLE EVENTS (ID NUMBER PRIMARY KEY, COUNTER NUMBER NOT NULL);
MERGE INTO EVENTS
USING (SELECT ID, COUNTER FROM DUAL LEFT JOIN EVENTS ON EVENTS.ID = :EVENT_ID) SRC
ON (EVENTS.ID = SRC.ID)
WHEN MATCHED THEN UPDATE SET COUNTER = SRC.COUNTER + 1
WHEN NOT MATCHED THEN INSERT (ID, COUNTER) VALUES (:EVENT_ID, 1);
Simple SQL securing single record for each ID and consistently increasing the counter no matter what application fires it or number of concurrent thread. You don't need to code anything at all and it's very lightweight as well.
It also doesn't produce any exception related to data consistency so you don't need any special handling.
UPDATE: It actually produces unique violation exception if both threads are inserting. I thought the second merge would switch to update, but it doesn't.
UPDATE: Just tested the same case on SQL Server and when executing in parallel and the record doesn't exist one MERGE inserts and the second updates.
I tried to put it in a sentence but it is better to give an example:
SELECT * FROM someTable WHERE id = someID;
returns no rows
...
some time passes (no inserts are done to the table and no ID updates)
...
SELECT * FROM someTable WHERE id = someID;
returns one row!
Is it possible that some DB mechanism prevents first SELECT to return row?
Oracle log has no errors.
No transactions are rolled back when two selects are executed.
You can't see uncommitted data in another session. When did the commit happen?
EDIT1: Are you the only one using this database? Or did/do you have multiple sessions?
I think in another session you or someone else has inserted this row, you do your select and you don't see this row. After that a commit happens in the other session (maybe implicit because a session is closed) and then you see this row when you select again.
I can think of other explanations, but I first want to know are you only one using this database.
With read consistency as provided by Oracle, you should not see a row appear like that. If you are running in some mode with automatic commits, so that each statement is a self-contained transaction, then read consistency is not being violated. Which program are you using to access the database? I agree with the other observations; the row should not appear if your session is not inserting it and no other session is active at the same time. I don't know of a DBMS that indulges in spontaneous data generation.
Don't you have scheduled jobs in that Oracle?