Queue data structure requiring K accesses before removal - algorithm

Question

I need a specialized queue-like data structure. It can be used by multiple consumers, but each item in queue must be removed from queue after k consumers read it.
Is there any production ready implementation? Or Should I implement a queue with read-counter in each item, and handle item removal myself?
Thanks in advance.

Answer 1

I think this is what you are looking for. Derived from the source code for BlockingQueue. Caveat emptor, not tested.
I tried to find a way to wrap Queue, but Queue doesn't expose its concurrency members, so you can't get the right semantics.
public class CountingQueue<E> {
private class Entry {
Entry(int count, E element) {
this.count = count;
this.element = element;
}
int count;
E element;
}
public CountingQueue(int capacity) {
if (capacity <= 0) {
throw new IllegalArgumentException();
}
this.items = new Object[capacity];
this.lock = new ReentrantLock(false);
this.condition = this.lock.newCondition();
}
private final ReentrantLock lock;
private final Condition condition;
private final Object[] items;
private int takeIndex;
private int putIndex;
private int count;
final int inc(int i) {
return (++i == items.length) ? 0 : i;
}
final int dec(int i) {
return ((i == 0) ? items.length : i) - 1;
}
private static void checkNotNull(Object v) {
if (v == null)
throw new NullPointerException();
}
/**
* Inserts element at current put position, advances, and signals.
* Call only when holding lock.
*/
private void insert(int count, E x) {
items[putIndex] = new Entry(count, x);
putIndex = inc(putIndex);
if (count++ == 0) {
// empty to non-empty
condition.signal();
}
}
private E extract() {
Entry entry = (Entry)items[takeIndex];
if (--entry.count <= 0) {
items[takeIndex] = null;
takeIndex = inc(takeIndex);
if (count-- == items.length) {
// full to not-full
condition.signal();
}
}
return entry.element;
}
private boolean waitNotEmpty(long timeout, TimeUnit unit) throws InterruptedException {
long nanos = unit.toNanos(timeout);
while (count == 0) {
if (nanos <= 0) {
return false;
}
nanos = this.condition.awaitNanos(nanos);
}
return true;
}
private boolean waitNotFull(long timeout, TimeUnit unit) throws InterruptedException {
long nanos = unit.toNanos(timeout);
while (count == items.length) {
if (nanos <= 0)
return false;
nanos = condition.awaitNanos(nanos);
}
return true;
}
public boolean put(int count, E e) {
checkNotNull(e);
final ReentrantLock localLock = this.lock;
localLock.lock();
try {
if (count == items.length)
return false;
else {
insert(count, e);
return true;
}
} finally {
localLock.unlock();
}
}
public boolean put(int count, E e, long timeout, TimeUnit unit)
throws InterruptedException {
checkNotNull(e);
final ReentrantLock localLock = this.lock;
localLock.lockInterruptibly();
try {
if (!waitNotFull(timeout, unit)) {
return false;
}
insert(count, e);
return true;
} finally {
localLock.unlock();
}
}
public E get() {
final ReentrantLock localLock = this.lock;
localLock.lock();
try {
return (count == 0) ? null : extract();
} finally {
localLock.unlock();
}
}
public E get(long timeout, TimeUnit unit) throws InterruptedException {
final ReentrantLock localLock = this.lock;
localLock.lockInterruptibly();
try {
if (waitNotEmpty(timeout, unit)) {
return extract();
} else {
return null;
}
} finally {
localLock.unlock();
}
}
public int size() {
final ReentrantLock localLock = this.lock;
localLock.lock();
try {
return count;
} finally {
localLock.unlock();
}
}
public boolean isEmpty() {
final ReentrantLock localLock = this.lock;
localLock.lock();
try {
return count == 0;
} finally {
localLock.unlock();
}
}
public int remainingCapacity() {
final ReentrantLock lock= this.lock;
lock.lock();
try {
return items.length - count;
} finally {
lock.unlock();
}
}
public boolean isFull() {
final ReentrantLock localLock = this.lock;
localLock.lock();
try {
return items.length - count == 0;
} finally {
localLock.unlock();
}
}
public void clear() {
final ReentrantLock localLock = this.lock;
localLock.lock();
try {
for (int i = takeIndex, k = count; k > 0; i = inc(i), k--)
items[i] = null;
count = 0;
putIndex = 0;
takeIndex = 0;
condition.signalAll();
} finally {
localLock.unlock();
}
}
}

Answer 2

A memory efficient way that retains the info you need:
Each queue entry becomes a
Set<ConsumerID>
so that you ensure the k times are for k distinct consumers: your app logic checks if the
set.size()==k
and removes it from queue in that case.
In terms of storage: you will have tradeoffs of which Set implementation based on
size and type of the ConsumerID
speed of retrieval requirement
E.g if k is very small and your queue retrieval logic has access to a
Map<ID,ConsumerId>
then you could have simply an Int or even a Short or Byte depending on # distinct ConsumerID's and possibly store in an Array . This is slower than accessing a set since it would be traversed linearly - but for small K that may be reasonable.