I mean when I write to java.util.concurrent.LinkedBlockingQueue I have a visibility guarantee happens-before. Single object put to the queue happens-before pull of the same object from the queue.
So I can assume in the reading thread, that any other state of variable is not older then the time at which the object was put to the LBQueue.
Do we have the same visibility guarantee in Chronicle-Queue ?
Do we have the same visibility guarantee in any of the usage style ( methodReader/writer vs wire().read().object() ) ?
Indeed you have the same (in fact, slightly stronger) guarantee. Each write to Chronicle Queue (in any style) results in a sequence of volatile write/reads to the memory-mapped file, hence provides memory barriers.
Additionally, each write is guarded by a write lock, which is implemented as a CAS-driven update to a value stored in a memory-mapped file.
Related
As per the reference, it says
Otherwise, each read of a single-word-sized or sub-word-sized memory location must observe a value actually written to that location (perhaps by a concurrent executing goroutine) and not yet overwritten.
Since a pointer is word-sized, does that mean synchronization is not necessary for mere purpose of visiblity?
No, visibility means that when one goroutine writes to a variable, other goroutines see the change. So, visibility requires synchronization. Otherwise a reading goroutine will possibly see a stale (not updated) value of the variable.
I write a code which rarely creates/removes objects (up to several thousands) but very frequently modifies them in soft IRQ context. These objects are also rarely read (and probably will also be rarely modified) from task context (via procfs: file per object). Currently my code contains global per-CPU data blocks, each one guarded by a spinlock. Such a block contains a fixed-sized hashtable for object storage.
Obviously the current design is not optimal, especially when having very high object update loads: reading objects from procfs will cause data losses in updating soft IRQs. I need to rewrite the synchronisation scheme to get rid of global locks. The most obvious choice - to have a spinlock for each hashtable bucket - it should scale well. The problem is that I'll probably need to use my own hashtable implementation or at least to reimplement several top-level macros (didn't find those in linux/hashtable.h for spinlock-protected buckets). Should I also look towards RCU-enabled hashtable (yet I have no solid understanding of this synchronisation approach)?
Buckets with lock protection are declared in the header linux/list_bl.h. They use lowest bit of the head pointer as a lock bit.
RCU-protected access to the bucket is defined with other hash table functions in the header linux/hashtable.h (they have _rcu suffix).
Choosing between locks and RCU is up to you. Note, that RCU itself cannot resolve modify-modify conflicts. And it helps mostly for frequently-read data, which seems is not your case.
As only one locking function - hlist_bl_lock - is declared for struct hlist_bl_head, and this function is unaware for irq's, additional actions should be performed when hash table can be used in irq or bottom halves:
spin_lock_irqsave:
local_irq_save(flags);
hlist_bl_lock(...);
spin_unlock_irqrestore:
hlist_bl_unlock(...);
local_irq_restore(flags);
spin_lock_bh:
local_bh_disable();
hlist_bl_lock(...);
spin_unlock_bh:
hlist_bl_unlock(...);
local_bh_enable();
There is a parm which is read more and written less, And I don't want to use a mutex .I got it done like this by unsafe and atomic:
type tmp struct {
}
var value unsafe.Pointer
func GetValue() *tmp {
return (*tmp)(atomic.LoadPointer(&value))
}
func SetValue(p *tmp) {
atomic.StorePointer(&value, unsafe.Pointer(p))
}
Is this thread-safe ? and atomic.StorePointer happen before atomic.LoadPointer ?
It will be thread safe in the sense that you don't know what happens first, but the updating is atomic. Have you considered using a RWMutex instead? This won't lock readers unless there's a write going on.
As far as I know, storing and retrieving a pointer using atomic.LoadPointer and atomic.StorePointer is thread-safe (in the sense that the pointer itself will not be corrupted).
Of course, the pointed object must be immutable, and this is not enforced by this mechanism. This is your job to be sure that updating the configuration will result in a new object being created before calling UpdateConfig.
However, the main issue is the code relies on unsafe operations. Making it type safe is difficult, since two pointers have actually to be stored (one for the object, one for the type). Atomic pointer operations are not enough anymore to guarantee the consistency.
This is why a specific type has been recently added in the sync/atomic package: the sync.Value type. It is designed exactly to do what you want (i.e. optimizing access to mostly constant data).
You can find an example here: http://golang.org/pkg/sync/atomic/#example_Value_config
If you look in the implementation, you will realize why atomic pointer operations alone are not enough to implement a type safe mechanism.
According to https://code.google.com/p/go/issues/detail?id=5045
Regarding Add/CAS, it should be formulated in in more general terms, along the
lines of: at atomic operation that stores a value (incl ADD/CAS) happens before
atomic operation that reads that value from the memory location (incl ADD/CAS).
I think We can use atomic to ensure it thread-safe.
I am new to linux kernel. My question is about the task_struct.
I know that Each task_struct has a reference to its parent process via a pointer to the task_struct of the parent.
After looking at the sched.h in the task_struct definition I noticed the following :
struct task_struct __rcu *real_parent; /* real parent process */
I found that it is referenced to compiler.h. I guess that the "__rcu" stands for "read copy update"
Can someone clarify the syntax ?
Read-copy-update is an algorithm that enables concurrent access to readers of a data structure without having to lock the structure. It can be read about here.
If the kernel is built with the CONFIG_SPARSE_RCU_POINTER config option, __rcu is defined in include/linux/compiler.h as
# define __rcu __attribute__((noderef, address_space(4)))
This is an annotation for a the Sparse code analysis tool that can warn about certain things the programmer may have overlooked. How this is relevant to RCU is explained in Documentation/RCU/checklist.txt:
__rcu sparse checks: tag the pointer to the RCU-protected data
structure with __rcu, and sparse will warn you if you
access that pointer without the services of one of the
variants of rcu_dereference().
rcu_dereference() returns a pointer that can be safely dereferenced by the code and documents the programmer's intention to protect the pointer with the RCU mechanism, enabling tools like Sparse to check for programming errors and omissions.
RCU stands for "read, copy, update". It is an algorithm that allows multiple readers to access data which can be updated or even deleted at the same time by writers.
Under RCU, writers still have to ensure mutual exclusion with regard to one another, but readers do not acquire a lock. Care has to be taken that the shared data structure is updated in ways that do not violate read integrity. If something has to be removed or deleted, the unlinking of that item from the data structure can be done in parallel with the readers but the actual deletion of the memory has to wait until the last reader has finished.
Rather than making the readers acquire a lock, the whereabouts of the readers are inferred in other ways. Threads can announce their intent to browse the data structure by joining a "read side critical section" which is not really a lock but a kind of global phase.
For instance, suppose that some threads entered the RCU read side critical section in phase 0. An updater has performed a deletion and want to free a piece of memory. It has to simply wait for all threads in the system to vacate phase 0. In the meanwhile, other readers are looking at the data structure already, but when they declare their intent to RCU, they do so by entering the RCU read-side critical section under phase 1. Only the phase 0 threads can possibly still have a pointer to the object that was deleted, and so when the last thread leaves phase 0, the object can safely be deleted. Newly arriving threads in phase 1 do not see the object, because the object has been removed from the data structure, so they have no way to find it.
RCU takes advantage of the idea that we do not need lock objects that are "owned" in order to know information like "no thread can be accessing this object any more".
Is NSObject's retain method atomic?
For example, when retaining the same object from two different threads, is it promised that the retain count has gone up twice, or is it possible for the retain count to be incremented just once?
Thanks.
NSObject as well as object allocation and retain count functions are thread-safe — see Appendix A: Thread Safety Summary in the Thread Programming Guide.
Edit: I’ve decided to take a look at the open source part of Core Foundation. In CFRuntime.c, __CFDoExternRefOperation() is the function responsible for updating the the retain counters. It tests whether the process has more than one thread and, if there’s more than one thread, it acquires a spin lock before updating the retain count, hence making this operation thread safe.
Interestingly enough, the retain count is not an attribute (or instance variable) of an object in the struct (class) sense. The runtime keeps a separate structure with retain counters. In fact, if I understand it correctly, this structure is an array of hash tables and there’s a spin lock for each hash table. This means that a lock refers to multiple objects that have been placed in the same hash table, i.e., the lock is neither global (for all instances) nor per instance.