I have the following situation:
One process is reading from a SQLite database.
Another processes is updating the database. Updates do not happen very often and all transactions are short. (less than 0.1ms on average)
The process that is reading should have low latencies for the queries. (around 0.1ms)
If the locking of SQLite would work like a mutex or readers-writer lock, everything would be ok.
From reading http://www.sqlite.org/lockingv3.html this should be possible. SQLite is using
LockFileEx(), sometimes without the LOCKFILE_FAIL_IMMEDIATELY, which would block the calling
process as desired.
However I could not figure out how to use/configure SQLite to achieve this behavior. Using a busy-handler
would involve polling, what is not acceptable because the minimal sleep time is usually 15ms on Windows.
I would like that the query is executed as soon as the update transaction ends.
Is this possible without changing the source code of SQLite. If not, is there such a patch available somewhere?
SQLite does not use a synchronization mechanism that would wait just until a lock is released.
SQLite never uses a blocking locking call; when it finds that the database is locked, is waits for some time and tries again.
(You could install your own busy handler to wait for a shorter time.)
The easiest way to prevent readers and a writer from blocking each other is to use WAL mode.
If you cannot use WAL mode, you can synchronize your transactions by implementing your own synchronization mechanism: use a common named mutex in all processes, and lock it around all your transactions.
(This would reduce concurrency if you had multiple readers.)
Related
I have a multi process Windows application that uses SQLite locally on the disk.
When one process creates contention on the DB, other processes are starved since they poll for the DB and are not "notified" by the OS scheduler when no locks are being held (like mutexes do).
I found a built-in VFS called unix-namedsem for WXWorks, perhaps that's what I'm looking for - only for Windows.
Is there an existing solution for my problem? Or must I implement my own VFS?
The Windows VFS does not implement Mutex locking, and there is no other such VFS for Windows.
You have to implement that VFS yourself, or simply modify your own DB functions to lock a mutex around all transactions.
I created a chronicle map with createOrRecoverPersistedTo method. But after restarting the jvm process some entries disappeared. How long will it take to persist the latest key-value after I put the entry? Is there anyway forcing the latest data to be persisted?
Theoretically, unless some "failure" events happen like JVM process crash or kill, or the server shutdown, you shouldn't wait until entries are persisted, OS memory subsystem takes care of that.
As was explained here in details, using createOrRecoverPersistedTo() after "normal" server restart (moreover, just the JVM process restart) is a harmful antipattern. You should simply use createPersistedTo().
Moreover, in the recent issue entry disappearence was also reported when ChronicleMap recovered without catastrophic events, that could be a bug in the recovery proceduces in Chronicle Map.
If you restart your server, mapped memory may not be fully flushed to files, e. g. in Linux the "safe" timeout when memory should be flushed is defined in "vm.dirty_expire_centisecs" configuration (google it), which should be 30 seconds by default.
According to the MSDN documentation, transactional NTFS doesn't seem to allow one to block on opening a file for write - instead the open operation fails with ERROR_SHARING_VIOLATION. I'd like to block on writes instead - how can I do this?
Ideally I'd like the following properties for the solution:
Works over a network share (so no local named mutex handles)
Auto-releases if the owning process dies
Doesn't require a separate file (named streams are OK)
Allows the locking wait to have a timeout (or be cancellable from another thread or APC)
Does anyone have some experience with a locking method that works with transactional NTFS with these properties?
I'm not sure I understand exactly what you're asking. TXF doesn't work across SMB shares.
My knee-jerk suggestion would be that if you are using files for this before using TXF, you could continue to use a file for this in non-transacted mode...
FYI, the reason TXF fails these transactional lock conflicts is to help applications avoid deadlocks.
Here: http://download.oracle.com/docs/html/A95907_01/diff_uni.htm#1077398
I found that on Windows Oracle is thread based, while on Unix this is process based. Why it is like that?
What's more, there are many Oracle processes http://www.adp-gmbh.ch/ora/concepts/processes/index.html regardless the system.
Why log writer and db writer are implemented as processes... and the query execution is done using threads (windows) or processes (unix).
Oracle makes use of a SGA shared memory area to store information that is (and has to be) accessible to all sessions/transactions. For example, when a row is locked, that lock is in memory (as an attribute of the row) and all the other transactions need to see it is locked.
In windows a thread cannot access another process's memory
threads cannot access memory that
belongs to another process, which
protects a process from being
corrupted by another process.
As such, in Windows Oracle must be a single process with multiple threads.
On OS's supporting the sharing of memory between processes then it is less work for Oracle to work as a multi-process architecture and leave the process management to the OS.
Oracle runs a number of background threads/processes to do work that is (or can be) asynchronous to the other processes. That way those can continue even when other processes/threads are blocked or busy.
See this answer I posted earlier on in similar vein to this question 'What is process and thread?'. Windows makes extensive use of threads in this fashion. Unlike *nix/Linux based systems which are based on threads. And see here also, this link is a direct link(which is embedded in the first link I have given) to the explanation I gave on how Linux time divisions threads and processes.
Hope this helps,
Best regards,
Tom.
Is there any workaround to remove deadlock without killing the session?
From the Concepts Guide:
Oracle automatically detects deadlock situations and resolves them by rolling back one of the statements involved in the deadlock, thereby releasing one set of the conflicting row locks.
You don't have to do anything to remove a deadlock, Oracle takes care of it automatically. The session is not killed, it is rolled back to a point just before the trigger statement. The other session is unaffected (i-e it still waits for the lock until the rolled back session either commits or rolls back its transaction).
In most situations deadlocks should be exceptionally rare. You can prevent all deadlocks by using FOR UPDATE NOWAIT statements instead of FOR UPDATE.
See also
Discussion about removing deadlock on AskTom
Deadlocks are automatically cleared in Oracle by cancelling one of the locked statements. You need not do it manually. One of the sessions will get "ORA-00060" and it should decide whether to retry or roll back.
But from you description it looks like you have a block, not deadlock.
Anyway, blocking session should somehow release its lock -- by commiting or rolling back its transaction. You can just wait for it (possibly for a long time). If you can change code of your application -- you probably can rewrite it to release lock or avoid it. Otherwise, you have to kill session to immediately unlock resources.
No, Oracle 10g does not seem to resolve deadlocks automatically in practice. We did have dealocks and we had to clear the sessions manually.
This page can help in identifying if you have deadlocks
Identifying and Resolving Oracle ITL Deadlock