I have the following setup:
- a spring boot application makes a post request to insert an object using hibernate into a postgresql table TableA;
- that certain postgresql table has a trigger that on certain conditions triggers a function that does an insert into a TableB;
- TableB has 2 triggers: the first one uses listen/notify function to send notifications on websocket after insert on TableB and the second one uses a function to insert into TableC
The problem: If the first insert the client does in TableA is successfully, he receives 200 Code, but down the road there are possibly another 2 inserts.
If a problems arises during either of the following inserts, the client sees 200 Code, but missed important data.
My logic says this has to be wrapped into a transaction-style entity of all-or-nothing but how do i do that ?
Kind Regards,
EDIT1: I just made a test, i annotated the first post request method with #Transactional and during the following inserts, if it encounters any error and any of the insert is not executed, the client is returned 500 error code. Sorry for wasting your time.
Annonate the first request method with #Transactional and control all possible errors, even by using try and catch blocks.
You should also define which errors your code is going to return, not all of them will be caused by the server, when you should be returning a 5XX error, but also by the client (e.g. sending wrong or unexpected parameters) and you should probably return 4XX code.
Related
I have an application that is connected to an Oracle database where it can execute select, insert and update statements. I want to call a function in that application that would tell the Oracle database to send a http request.
My objectives:
Fire and forget - the application and database don't care about the response or whether the request was successful
The application and database should never wait for the response
I was thinking about these approaches:
Application inserts a row to a table, trigger after insert calls a db function that uses UTL_HTTP to perform the request
Application calls a db function that uses UTL_HTTP to perform the request via a select statement
What are the advantages/pitfalls of said approaches with regard to my objectives?
Here are some thoughts of the top of my head, I'm sure there are other considerations:
If you do it in a trigger, the http request will go out even if you then decide to rollback the insert. So are you okay with having the http request sent but no row ends up in the table? If so, a trigger would be fine.
If you do it in a trigger, then any other interfaces or humans manually needing to load rows to the table will cause the http request to fire. Is that what you want? Then a trigger is great. If not, a trigger isn't so great.
Triggers are dropped if the table is dropped. If you do maintenance of the kind we do all the time in data warehousing (CTAS a new table, drop the old, rename the new to the old, etc..) you can easily lose your code because the code is contained in the trigger and the trigger disappears with the table drop. Triggers are not safe places for complex code. If you do end up using a trigger, consider having the trigger simply call a procedure, and store all your real code in that procedure.
If you decide to not use a trigger at all, you are better off writing a procedure than a function you call through a select statement. The point of a function is to return something, and in this case, you don't have anything from the http process you are wishing to return. And the select statement would be arbitrary. Just use a normal procedure call.
Our application reads a record from an Oracle 'Event' table. When the event record exists we update the 'count' field of that record. If the record doesn't exist we insert it. So we want only 1 record for a particular event in the table.
The problem with this is probably quite predictable: one application thread will read the table, see the event is not there, insert the new event and commit. But before it commits a second thread will also read the table and see the event is not there. And then both threads will insert the event and we end up with 2 records for the same event.
I guess synchronizing access to this particular method in our application will prevent this problem, but what is the best option in Oracle to prevent this? Will MERGE for example always prevent this problem?
Serialising access to the procedure that implements this functionality would be trivial to implement, using DBMS_LOCK to define and take an exclusive lock.
Serialising through SQL based methods is practically impossible, due to the read consistency model.
CREATE TABLE EVENTS (ID NUMBER PRIMARY KEY, COUNTER NUMBER NOT NULL);
MERGE INTO EVENTS
USING (SELECT ID, COUNTER FROM DUAL LEFT JOIN EVENTS ON EVENTS.ID = :EVENT_ID) SRC
ON (EVENTS.ID = SRC.ID)
WHEN MATCHED THEN UPDATE SET COUNTER = SRC.COUNTER + 1
WHEN NOT MATCHED THEN INSERT (ID, COUNTER) VALUES (:EVENT_ID, 1);
Simple SQL securing single record for each ID and consistently increasing the counter no matter what application fires it or number of concurrent thread. You don't need to code anything at all and it's very lightweight as well.
It also doesn't produce any exception related to data consistency so you don't need any special handling.
UPDATE: It actually produces unique violation exception if both threads are inserting. I thought the second merge would switch to update, but it doesn't.
UPDATE: Just tested the same case on SQL Server and when executing in parallel and the record doesn't exist one MERGE inserts and the second updates.
in our work we create two .net listener,
first one:
calling oracle stored procedure that insert bulk of data into table(table1) using insert into select syntax:
insert into table1 select c1,c2... from tbl2 inner join tbl3....
then we use explicity commit;
second listener:
calling oracle procedure that reading data inserted into table1 via listener1
but we notice that even the record inserted into table1 listener2 couldn't see that recordat same time even that commit is use.
my question is how does cmmit work when we use insert ...select?
is this issue related to session?when listener 1 session end listener 2 can read data?
please help,
thank in advance.
You're using the wrong terms...
A listener is a server application that listens to the incoming client requests and hands it to the DB engine. A listener is not being used on the client end.
A session is not related to the data you can see, a transaction is the object that controls that.
Oracle works in a very clear way - After a transaction has committed - all the new transactions can see it, and already existing transactions can see the new content based on it transaction configurations..
I recommend you reading about isolation levels in that context http://msdn.microsoft.com/en-us/library/system.transactions.isolationlevel(v=vs.110).aspx
By default - the moment (and in DB it is defined by SCN) a transaction have been committed - the data is visible to the client.
Bottom line - your issue is related either to transaction isolation levels (in the case the reading transaction started before the commit), or to the writer, which does not commit the data when you think it is (a transaction issue).
After the call to transaction.Commit() in .net returned - the data is already visible, and other transactions are seeing it.
You're second question was how commit works.
This is a very complicated process in Oracle, so I'll give a really short description:
1. When you commit, Oracle first runs some verifications before the commit itself (for example, runs the deferred constraints).
2. After oracle knows it can safely commit the changes it gets the system time (SCN) , write the commit itself to the redo log, and flushes the data to disk (for consistency).
3. Sends an ACK to the user, that the data is already visible to the world.
4. marks the buffers been used as free.
Something I want to add, just to make sure (and I'm writing it half a sleep - so excuse me if it does not compile...)
In you're .net code - your code should be logically equivalent to it:
OracleConnection con = new OracleConnection(connStr);
con.Open();
OracleTransaction trans = con.BeginTransaction();
OracleCommand cmd = con.CreateCommand();
cmd.Connection = cmd;
cmd.CommandText = "insert into ...";
cmd.ExecuteNonQuery();
cmd.Dispose();
trans.Commit();
trans.Dispose();
con.Close();
con.Dispose();
and if you're using LINQ - make sure you create the transaction scope on the right area.
We know that it is possible to dynamically figure out the name of the procedure or package that is currently executing as explained here and here. This generally applies to statements being executed from other stored procedures (compiled) in the database.
The problem:
We have been trying to log all UPDATE activity on a specific column (called STATE) by placing a trigger on the table and invoking who_called_me from within the trigger. The purpose of doing this is apparently as per the application design the column STATE could get updated by multiple pieces of code (residing in the database) based on certain business conditions. In addition to that, the column could also get updated by the application which is a hibernate based application and at times when the update happens by a hibernate query the who_called_me function returns nothing. There are multiple parts in the application that could also UPDATE the column STATE based on certain conditions.
The who_called_me strategy is working well for us in cases where a stored procedure (which resides in the database) issues the UPDATE statement and who_called_me is successfully capturing the corresponding owner, name, line no. etc. of the stored procedure. But in case the UPDATE happens from hibernate, the function captures no details.
Is there a way to capture which hibernate query UPDATEd the row through the trigger? Or is there any other way?
Note: The trigger code is similar to the answer posted on this question.
you can track the query with ora_sql_text function, e.g. this is the function I use for that:
-- getting sql code, which is calling the current event, as clob
function getEventSQLtext
return clob
is
sqllob clob;
sql_text ora_name_list_t;
dummy integer;
begin
dummy := ora_sql_txt(sql_text);
dbms_lob.createtemporary(sqllob,false);
for i in 1..sql_text.count loop
dbms_lob.writeappend(sqllob,length(sql_text(i)),sql_text(i));
end loop;
return sqllob;
if dummy is null then null; end if; -- removing warning of non-used variable :)
end;
This will be a query which is generated by hibernate and this is the only information you can get because this should be the only thing hibernate can do with DB.
It turns out, the who_called_me approach works better for stored procedure calls where the stack trace can point exactly which line invoked a DML. In, case of hibernate it is possible that the code may not call a stored procedure but in-turn may have individual DMLs which get invoked based on certain conditions. As opposed to other answer given by #simon, the ora_sql_txt function may only work in system event triggers or I may be wrong, but either way it is not capable of capturing the SQL Statement issued by Hibernate (tested that it does not works and retunrs a NULL value).
So at the end of the day, to find what SQL Hibernate is using, DB Trace files and Hibernate debug level logs is the only way for now.
I have and application which uses Adodb to insert data in Oracle table(customers database).
Data is successfully inserted if there are no errors.
If there is any error like invalid datatype etc. Error is raised and captured by my application and dumped in log gile.
My customer has written their own triggers on this particular table. When a record is inserted few other checking are done be fore the data insertion
Now all fine until now.
But recently we found that many a times data is not inserted in the oracle table.
When checked in log file no error was found.
Then I logged the query which was executed.
Copied the query to oracle Sql prompt and executed it gave error of trigger.
My Issue is
Customer is not ready to share the details of trigger.
Error is not raised while inserting to oracle table so we are not able to log it or take any action.
The same qry when executed directly in oracle the trigger errors are show.
Help needed for
Why the error is not raised in ADODB
Do I have to inform customer to implement any error raising
Anything that you can suggest for resolving the issue
I have 0% to 10% knowledge of Oracle
"Copied the query to oracle Sql prompt and executed it gave error of trigger." Since the ADO session doesn't report an error, it may be that the error from the trigger is misleading. It may simply be a check on the lines of "Hey, you are not allowed to insert into this table except though the application".
"Error is not raised while inserting to oracle table so we are not able to log it or take any action."
If the error isn't raised at the time of insert, it MAY be raised at the time of committing. Deferred constraints and materialized views could give this.
Hypothetically, I could reproduce your experience as follows:
1. Create a table tab_a with a deferrable constraint initially deferred (eg val_a > 10)
2. The ADO session inserts a row violating the constraint but it dooesn't error because the constraint is deferred
3. The commit happens and the constraint violation exception fires and the transaction is rolled back instead of being committed.
So see if you are catering for the possibility of an error in the commit.
It may also be something else later in the transaction which results in a rollback of the whole transaction (eg a deadlock). Session tracing would be good. Failing that, look into a SERVERERROR trigger on the user to log the error (eg in a file, so it won't be rolled back)
http://download-west.oracle.com/docs/cd/B19306_01/server.102/b14200/statements_7004.htm#i2153530
You can log your business logic in log table.
But you have to use stored procedure to log the message.
Stored procedure should have pragma Transaction such that your log data must be saved in log table.
You are trigger should have error handling - and in error handling , you have to call Logged stored procedure (which have pragma transaction)
I've never used adodb ( and I assume that is what you are using, not ADO.NET?).. But, a quick look at its references leads to this question.. Are you actually checking the return state of your query?
$ok = $DB->Execute("update atable set aval = 0");
if (!$ok) mylogerr($DB->ErrorMsg());