Concurrent, non-blocking, fixed-size list? - ruby

Question

I'm looking to implement a data structure with the following characteristics:
Operations
Push: Add an element to the front of the list.
Read: Read all elements in the list
Behavior
Fixed-size: The list should not grow beyond a specified threshold, and it should automatically truncate from the end (oldest item) if that threshold is exceeded. This does not need to be strictly enforced, but the list should eventually be truncated once it passes the threshold.
Concurrency-safe: The structure should safely accommodate multiple parallel pushers and readers
Non-blocking: This is the real problem. I'd like to use an implementation without locks. Many threads should be able to push/read simultaneously if possible. A less-desirable, but acceptable option would be an implementation that has locks, but minimizes contention between multiple pushers/readers. I'm familiar with reader-writer locks, but those assume infrequent writes, which is not my use-case.
Optional but nice-to-have
Write-read consistency: If a single thread pushes to the structure, a read immediately following should contain the written item. This would be nice, but I'm wondering whether excluding this requirement could make the above requirements easier to implement.
I'm mostly a novice in concurrent data structures. Does an example of such a data structure exist? Ring buffers are interesting, but I don't think they can be non-blocking. Linked-lists are promising, but the concurrency-safe, non-blocking requirements complicate the implementation considerably.
I have found some good papers on implementing non-blocking linked lists using atomic CAS (compare-and-swap) operations, but the fixed-size requirement throws a bit of a wrench into those. Maybe that idea can be adapted to a fixed-size list?
For what it's worth, I'm interested in using this in Ruby. I understand that MRI has the global-interpreter-lock, which makes this a bit useless for MRI, but other Ruby runtimes could take advantage of this, and I'm thinking of it as a learning exercise to grow my concurrent programming skills.

Answer 1

Analysis
This question might be a better fit on Software Engineering, rather than here on Stack Overflow, as it seems to be more of a design question. That said, I suggest using thread-safe arrays, or delegating resource contention to an MVCC database if you can't redesign your application to avoid a singular shared object altogether.
Recommendations
You can implement a thread-safe list or simulate a circular buffer using Concurrent::Array with the #unshift and #pop methods. You can also choose to externalize locking to something like a database, where Ruby's GIL is largely irrelevant to the underlying queue or locking mechanisms. However, to the best of my knowledge, there's no way to create a truly lockless concurrent access object in Ruby, although implementing your own multiversion concurrency control might come close.
The low-hanging fruit is probably to externalize your reads and writes to an MVCC-capable database such as PostgreSQL. If you can't or won't do that, you may need to accept the trade-offs inherent in the ACID properties and performance characteristics of your application and data structures. In particular, the use of a single shared data structure is a design decision you should perhaps re-evaluate if you can.
Before you start down that path, just make sure that you have a real performance problem to solve. While there are certainly cases where locks add noticeable overhead, many real-world applications are sufficiently performant even with Ruby's GIL in the mix. Your mileage may certainly vary.

Answer 2

You might consider creating a class such as the following. I don't consider this to be complete. Moreover, I have not considered non-blocking issues, which is a broad topic that is not specific to this class.
class TruncatedList
attr_reader :max_size
alias to_s inspect
def initialize(max_size=Float::INFINITY)
#max_size = max_size
#list = []
end
def pop(n=1)
return nil if #list.empty?
case n
when 0
nil
when 1
#list.pop
else
#list.pop([n, #list.size].min)
end
end
def >>(obj)
#list.pop if #list.size == #max_size
#list.unshift(obj)
end
def unshift(*arr)
arr.each do |obj|
#list.pop if #list.size == #max_size
#list >> obj
end
end
def <<(obj)
if #list.size < #max_size
#list << obj
else
#list
end
end
def push(*arr)
arr.each do |obj|
break(#list) if #list.size == #max_size
#list << obj
end
end
def shift(n=1)
return #list if #list.empty?
case n
when 0
nil
when 1
#list.shift
else
#list.shift([n, #list.size].min)
end
end
def pop(n=1)
return nil if #list.empty?
case n
when 0
nil
when 1
#list.pop
else
#list.pop([n, #list.size].min)
end
end
def inspect
#list.to_str
end
def to_a
#list
end
def size
#list.size
end
end
Here is an example of how this set might be used.
t = TruncatedList.new(6)
#=> #<TruncatedList:0x00007fe2db0512a0 #max_size=6, #list=[]>
t.inspect
#=> "[]"
t.to_a
#=> []
t >> 1
#=> 1
t.inspect
#=> "[1]"
t.unshift(2,3)
#=> [2, 3]
t.inspect
#=> "[3, 2, 1]"
t.unshift(4,5,6,7,8)
#=> [4, 5, 6, 7, 8]
t.inspect
#=> "[8, 7, 6, 5, 4, 3]"
t.to_a
#=> [8, 7, 6, 5, 4, 3]

Answer 3

I think I came up with an interesting solution that meets the requirements.
Theory
We use a linked-list as the foundation, but add thread-safety and truncation on top.
Pushes
During pushes, we accomplish thread-safety by using a compare-and-set operation. The push will succeed only if another thread has not already pushed to the last-known list head. If the push fails, we simply retry until it succeeds.
Truncation
When the first node is pushed, we designate it as the "prune node". As items get pushed to the list, that node is pushed further down, but we maintain a reference to it. When the list reaches capacity, we set break the link on the "prune node" to allow the following nodes to be garbage collected. Then we set the newest node as the "prune node". This way, the list size never exceeds "capacity * 2".
Reads
Because it's a linked-list without arbitrary insertions, we get mostly consistent reads because the list nodes will never be rearranged. We dereference the head when we start reading the list. We never read more elements than the configured capacity. If the list is truncated during a read, it's possible that we might not read enough nodes (this could be mitigated by saving the prune node when starting enumeration so that pruned nodes could still be read while the enumerator is active).
Thoughts
I'm pretty happy about the truncation mechanism, but it seems likely that a Mutex-based solution would perform just-as-well or even better than the CAS solution. It likely depends on how heavily contested the push operation is, and would need to be benchmarked.
require 'concurrent-ruby'
class SizedList
attr_reader :capacity
class Node
attr_reader :value
attr_reader :nxt
def initialize(value, nxt = nil)
#value = value
#nxt = Concurrent::AtomicReference.new(nxt)
#count = Concurrent::AtomicFixnum.new(0)
end
def increment
#count.increment
end
end
def initialize(capacity)
#capacity = capacity
#head = Node.new(nil)
#prune_node = Concurrent::AtomicReference.new
end
def push(element)
succeeded = false
node = nil
# Maybe should just use a mutex for this write instead of CAS
# It needs to be benchmarked
until succeeded
first = #head.nxt.get
node = Node.new(element, first)
succeeded = #head.nxt.compare_and_set(first, node)
end
# Every N nodes where N=#capacity is designated as the "prune node"
# Once we push N times, we drop all the nodes after the prune node by setting
# it's nxt value to nil.
# Then we set the first node as the new prune node
#prune_node.compare_and_set(nil, node) if #prune_node.get.nil?
prune_node = #prune_node.get
count = prune_node.increment
if count >= #capacity
if #prune_node.compare_and_set(prune_node, node)
prune_node.nxt.set(nil)
end
end
nil
end
def each(&block)
enum = Enumerator.new do |yielder|
current = #head
# Here we just iterate through the list, but limit the results to #capacity
#capacity.times do
current = current.nxt.get
break if current == nil
yielder.yield(current.value)
end
end
block ? enum.each(&block) : enum
end
end