I have Web app(as ORM I use Hibernate) that populates data from Oracle 11 DB.
For short period of time some Oracle packages becomes invalid and then becomes valid back (it's legacy data load and during this process user can use other UI).
When data load finishes and user perform any query to those packages I have an error:
ORA-04068: existing state of packages has been discarded ORA-04061:
existing state of package "sche.pck" has been invalidated ORA-04065:
not executed, altered or dropped package "sche.pck" ORA-06508: PL/SQL:
could not find program unit being called: "sche.pck"
If user press F5 (on error message screen) then the query executes successfully. If there any way to repeat user query when such errors happen?
Yes - try/catch the exception, inspect the exception message, looking for ORA-04068, and if it is found, rerun the query.
Ideally, you should have a number of retries. Something like:
for (int i = 0; i < 3; i++) {
try {
executeQuery();
break; //if successful;
} catch (..) {
if (!ex.getMessage().contains("ORA-06508")){
throw ex;
}
}
}
Looks a bit hacky, and I'd suggest to try to fix the original problem instead.
Update:
It seems you have to do that in many places, so the above will be tedious. If you really cannot fix the underlying oracle problem, you can try wrapping your DataSource, Connection and Statement objects into your own implementations that simply delegate to the underlying object, but in the case of executeQuery(), performs the retry.
Related
I am currently making a turn based strategy game with laravel (mysql DB with InnoDB) engine and want to make sure that I don't have bugs due to race conditions, duplicate requests, bad actors etc...
Because these kind of bugs are hard to test, I wanted to get some clarification.
Many actions in the game can only occur once per turn, like buying a new unit. Here is a simplified bit of code for purchasing a unit.
$player = Player::find($player_id);
if($player->gold >= $unit_price && $player->has_purchased == false){
$player->has_purchased = true;
$player->gold -= $unit_price;
$player->save();
$unit = new Unit();
$unit->player_id = $player->id;
$unit->save();
}
So my concern would be if two threads both made it pass the if statement and then executed the block of code at the same time.
Is this a valid concern?
And would the solution be to wrap everything in a database transaction like https://betterprogramming.pub/using-database-transactions-in-laravel-8b62cd2f06a5 ?
This means that a good portion of my code will be wrapped around database transactions because I have a lot of instances that are variations of the above code for different actions.
Also there is a situation where multiple users will be able to update a value in the database so I want to avoid a situation where 2 users increment the value at the same time and it only gets incremented once.
Since you are using Laravel to presumably develop a web-based game, you can expect multiple concurrent connections to occur. A transaction is just one part of the equation. Transactions ensure operations are performed atomically, in your case it ensures that both the player and unit save are successful or both fail together, so you won't have the situation where the money is deducted but the unit is not granted.
However there is another facet to this, if there is a real possibility you have two separate requests for the same player coming in concurrently then you may also encounter a race condition. This is because a transaction is not a lock so two transactions can happen at the same time. The implication of this is (in your case) two checks happen on the same player instance to ensure enough gold is available, both succeed, and both deduct the same gold, however two distinct units are granted at the end (i.e. item duplication). To avoid this you'd use a lock to prevent other threads from obtaining the same player row/model, so your full code would be:
DB::transaction(function () use ($unit_price) {
$player = Player::where('id',$player_id)->lockForUpdate()->first();
if($player->gold >= $unit_price && $player->has_purchased == false){
$player->has_purchased = true;
$player->gold -= $unit_price;
$player->save();
$unit = new Unit();
$unit->player_id = $player->id;
$unit->save();
}
});
This will ensure any other threads trying to retrieve the same player will need to wait until the lock is released (which will happen at the end of the first request).
There's more nuances to deal with here as well like a player sending a duplicate request from double-clicking for example, and that can get a bit more complex.
For you purchase system, it's advisable to implement DB:transaction since it protects you from false records. Checkout the laravel docs for more information on this https://laravel.com/docs/9.x/database#database-transactions As for reactive data you need to keep track of, simply bind a variable to that data in your frontEnd, then use the variable to update your DB records.
In the case you need to exit if any exception or error occurs. If an exception is thrown the data will not save and rollback all the transactions. I recommand to use transactions as possible as you can. The basic format is:
DB::beginTransaction();
try {
// database actions like create, update etc.
DB::commit(); // finally commit to database
} catch (\Exception $e) {
DB::rollback(); // roll back if any error occurs
// something went wrong
}
See the laravel docs here
I recently ran into an instance where I wanted to hit the database from a Task I have running periodically within a web application. I refactored the code to use the ThreadStaticSessionContext so that I could get a session without an HttpContext. This works fine for reads, but when I try to flush an update from the Task, I get the "Index was out of range. Must be non-negative and less than the size of the collection." error. Normally what I see for this error has to do with using a column name twice in the mapping, but that doesn't seem to be the issue here, as I'm able to update that table if the session is associated with a request (and I looked and I'm not seeing any duplicates). It's only when the Task tries to flush that I get the exception.
Does anyone know why it would work fine from a request, but not from a call from a Task?
Could it be because the Task is asynchronous?
Call Stack:
at System.ThrowHelper.ThrowArgumentOutOfRangeException()
at System.Collections.Generic.List`1.System.Collections.IList.get_Item(Int32 index)
at NHibernate.Engine.ActionQueue.ExecuteActions(IList list)
at NHibernate.Engine.ActionQueue.ExecuteActions()
at NHibernate.Event.Default.AbstractFlushingEventListener.PerformExecutions(IEventSource session)
at NHibernate.Event.Default.DefaultFlushEventListener.OnFlush(FlushEvent event)
at NHibernate.Impl.SessionImpl.Flush()
Session Generation:
internal static ISession CurrentSession {
get {
if(HasSession) return Initializer.SessionFactory.GetCurrentSession();
ISession session = Initializer.SessionFactory.OpenSession();
session.BeginTransaction();
CurrentSessionContext.Bind(session);
return session;
}
}
private static bool HasSession {
get { return CurrentSessionContext.HasBind(Initializer.SessionFactory); }
}
Task that I want to access the database from:
_maid = Task.Factory.StartNew(async () => {
while(true) {
if(CleaningSession != null) CleaningSession(Instance, new CleaningSessionEventArgs { Session = UnitOfWorkProvider.CurrentSession });
UnitOfWorkProvider.TransactionManager.Commit();
await Task.Delay(AppSettings.TempPollingInterval, _paycheck.Token);
}
//I know this function never returns, I'm using the cancellation token for that
// ReSharper disable once FunctionNeverReturns
}, _paycheck.Token);
_maid.GetAwaiter().OnCompleted(() => _maid.Dispose());
Edit: Quick clarification about some of the types above. CleaningSession is an event that is fired to run the various things that need to be done, and _paycheck is the CancellationTokenSource for the Task.
Edit 2: Oh yeah, and this is using NHibernate version 4.0.0.4000
Edit 3: I have since attempted this using a Timer, with the same results.
Edit 4: From what I can see of the source, it's doing a foreach loop on an IList. Questions pertaining to an IndexOutOfRangeException in a foreach loop tend to suggest a concurrency issue. I still don't see how that would be an issue, unless I misunderstand the purpose of ThreadStaticSessionContext.
Edit 5: I thought it might be because of requests bouncing around between threads, so I tried creating a new SessionContext that combines the logic of the WebSessionContext and ThreadStaticSessionContext. Still getting the issue, though...
Edit 6: It seems this has something to do with a listener I have set up to update some audit fields on entities just before they're saved. If I don't run it, the commit occurs properly. Would it be better to do this through an event than OnPreInsert, or use an interceptor instead?
After muddling through, I found out exactly where the problem was. Basically, there was a query that was run to load the current user record called from inside of the PreUpdate event in my listener.
I came across two solutions to this. I could cache the user in memory, avoiding the query, but having possibly stale data (not that anything other than the id matters here). Alternatively, I could open a temporary stateless session and use that to look up the user in question.
I am trying to test a workflow where the change i made reordered the deletes and how it cleans up the other indices from hbase.
There are 3 different indices being deleted. The logic somehow roughly resembles this operation.
try{
try{
hTable.delete(firstIndexDeletes);
} catch(IOException ie) {
// clean up and exception handling for first index
}
//more processing logic for second index
try{
hTable.delete(secondIndexDeletes)
} catch(IOException ie) {
// Clean up and exception handling for second index
}
//more processing logic
hTable.delete(thirdIndex);
} catch(IOException ie) {
//Clean up and exception handling for third index
}
I am trying to test the exception handling part via integration tests (i was able to get it tested throughly via unit tests) and i am trying to make the delete thrown an exception and i decided to use a lock on a specific index so that if an delete happens on that row it will throw an exception.
hTable.lockRow(Bytes.toBytes(firstIndexKey));
ideally i expected it to throw an exception for that row when it was deleted as part of firstIndexDeletes but somehow it just doesn't make any difference in my tests, it's not going to the exception handling part like i wanted. Is there something elementary i am missing?
To my knowledge (from routine, close examination of the source) explicit row locks are being retired from HBase. That said I've never tried to use them.
In my opinion, I would expect thorough unit test coverage (where you can exploit mocking) to be sufficient.
I'm using the TransactionScope class within a project based on Silverlight and RIA services. Each time I need to save some data, I create a TransactionScope object, save my data using Oracle ODP, then call the Complete method on my TransactionScope object and dispose the object itself:
public override bool Submit(ChangeSet changeSet)
{
TransactionOptions txopt = new TransactionOptions();
txopt.IsolationLevel = IsolationLevel.ReadCommitted;
using (TransactionScope tx = new TransactionScope(TransactionScopeOption.Required, txopt))
{
// Here I open an Oracle connection and fetch some data
GetSomeData();
// This is where I persist my data
result = base.Submit(changeSet);
tx.Complete();
}
return result;
}
My problem is, the first time I get the Submit method to be called, everything is fine, but if I call it a second time, the execution gets stuck for a couple of minutes after the call to Complete (so, when disposing tx), then I get the Oracle error "ORA-12154". Of course, I already checked that my persistence code completes without errors. Any ideas?
Edit: today I repeated the test and for some reason I'm getting a different error instead of the Oracle exception:
System.InvalidOperationException: Operation is not valid due to the current state of the object.
at System.Transactions.TransactionState.ChangeStatePromotedAborted(InternalTransaction tx)
at System.Transactions.InternalTransaction.DistributedTransactionOutcome(InternalTransaction tx, TransactionStatus status)
at System.Transactions.Oletx.RealOletxTransaction.FireOutcome(TransactionStatus statusArg)
at System.Transactions.Oletx.OutcomeEnlistment.InvokeOutcomeFunction(TransactionStatus status)
at System.Transactions.Oletx.OletxTransactionManager.ShimNotificationCallback(Object state, Boolean timeout)
at System.Threading._ThreadPoolWaitOrTimerCallback.PerformWaitOrTimerCallback(Object state, Boolean timedOut)
I somehow managed to solve this problem, although I still can't figure out the reason it showed up in the first place: I just moved the call to GetSomeData outside the scope of the distributed transaction. Since the call to Submit may open many connections and perform any kind of operations on the DB, I just can't tell why GetSomeData was causing this problem (it just opens a connection, calls a very simple stored function and returns a boolean). I can only guess that has something to do with the implementation of the Submit method and/or with the instantiation of multiple oracle connections within the same transaction scope.
Is it possible to execute COMMIT WRITE BATCH NOWAIT in Hibernate?
I didn't search extensively but I couldn't find any evidence that you can access this functionality at the JDBC driver level.
And this leaves you with the option to specify the COMMIT_WRITE parameter at the instance or session level, if this makes sense for you.
Just in case, let me quote this blog post (I'm pasting the content for reference because the original site is either unavailable or dead and I had to use Google Cache):
Using "Commit Write Batch Nowait" from within JDBC
Anyone who has used the new
asynchronous commit feature of Oracle
10.2 will be aware that it's very useful for transaction processing
systems that would traditionally be
bound by log_file_sync wait events.
COMMIT WRITE BATCH NOWAIT is faster
because it doesn't wait for a message
assuring it that the transaction is
safely in the redo log - instead it
assumes it will make it. This nearly
eliminates log_file_sync events. It
also arguably undermines the whole
purpose of commit, but there are many
situations where the loss of a
particular transaction (say to delete
a completed session) is perfectly
survivable and far more preferable
than being unable to serve incoming
requests because all your connections
are busy with log_file_sync wait
events.
The problem anyone using Oracle's JDBC
driver is that neither the 10.2 or
11.1 drivers have any extensions which allow you to access this functionality
easily - while Oracle have lots of
vendor specific extensions for all
sorts of things support for async
commit is missing.
This means you can:
Turn on async commit at the instance level by messing with the
COMMIT_WRITE init.ora parameter.
There's a really good chance this will
get you fired, as throughout the
entire system COMMIT will be
asynchronous. While we think this is
insane for production systems there
are times where setting it on a
development box makes sense, as if you
are 80% log file sync bound setting
COMMIT_WRITE to COMMIT WRITE BATCH
NOWAIT will allow you to see what
problems you face if you can somehow
fix your current ones.
Change COMMIT_WRITE at the session level. This isn't as dangerous as
doing it system wide but it's hard to
see it being viable for a real world
system with transactions people care
about.
Prepare and use a PL/SQL block that goes "BEGIN COMMIT WRITE BATCH NOWAIT;
END". This is safer than the first
two ideas but still involves a network
round trip.
Wrap your statement in an anonymous block with an asynchronous commit.
This is the best approach we've seen.
Your code will look something like
this:
BEGIN
--
insert into generic_table
(a_col, another_col, yet_another_col)
values
(?,?,?);
--
COMMIT WRITE BATCH NOWAIT;
--
END;
I was looking for a way to do this but couldn't get it working in a test. The reason for my hold up was that I was expecting the wrong results from my test. I was testing by manually acquiring a shared table lock to simulate adding an index - but in this case, the insert query acquires the lock, not the commit. So it doesn't actually solve the problem I was looking to solve. I got round my problem by moving these insertions into a background queue, so that they don't hold up the main web request.
Anyway I think you can still do asynchronous commits in Hibernate. Basically you can use the Session.doWork() method to get access to the native Connection object (or in older versions of Hibernate, the Session.connection() method). I also moved the commit SQL into a strategy interface, so that we can run our HSQLDB-based tests which wouldn't understand the Oracle specific SQL.
In fact, it may be fine to use Session.createSQLQuery and give that the SQL, avoiding having to directly use Connection. Try it and see how it works.
private NativeStrategy nativeStrategy = new OracleStrategy();
interface NativeStrategy {
String commit();
}
public static final class OracleStrategy implements NativeStrategy {
public String commit() {
return "COMMIT WRITE BATCH NOWAIT";
}
}
public void saveAsynchronously(MyItem item) {
session.save(item);
session.flush();
// Try to issue an asynchronous commit where supported.
session.doWork(new Work() {
public void execute(Connection connection) throws SQLException {
Statement commit = connection.createStatement();
try {
commit.execute( nativeStrategy.commit() );
} finally {
commit.close();
}
}
});
}