Storing and retrieving object in ray.io - actor

Question

I have a ray cluster running on a machine as below:
ray start --head --redis-port=6379
I have two files that need to run on the cluster.
Producer p_ray.py:
import ray
ray.init(address='auto', redis_password='5241590000000000')
#ray.remote
class Counter(object):
def __init__(self):
self.n = 0
def increment(self):
self.n += 1
def read(self):
return self.n
counters = [Counter.remote() for i in range(4)]
[c.increment.remote() for c in counters]
futures = [c.read.remote() for c in counters]
print(futures, type(futures[0]))
obj_id = ray.put(ray.get(futures))
print(obj_id)
print(ray.get(obj_id))
while True:
pass
Consumer c_ray.py:
import ray
ray.init(address='auto', redis_password='5241590000000000')
[objs] = ray.objects()
print('OBJ-ID:', objs, 'TYPE:', type(objs))
print(ray.get([objs]))
My intention is to store the futures objects from producer and retrieve it in the consumer. I can retrieve the Object ID in the consumer. However the get in the consumer never returns.
What am I doing wrong?
How do I resolve my requirement?

Answer 1

This particular case can be a bug (I am not 100% sure). I created an issue at Ray Github.
But, this is not a good way to get object created by the p_ray.py. If you have many objects, it will be extremely complicated to manage. You can implement a similar thing using a detached actor. https://ray.readthedocs.io/en/latest/advanced.html#detached-actors.
The idea is to create a detached actor that can be retrieved by any driver/worker running in the same cluster.
p_ray.py
import ray
ray.init(address='auto', redis_password='5241590000000000')
#ray.remote
class DetachedQueue:
def __init__(self):
self.dict = {}
def put(key, value):
self.dict[key] = value
def get(self):
return self.dict
#ray.remote
class Counter(object):
def __init__(self):
self.n = 0
def increment(self):
self.n += 1
def read(self):
return self.n
queue = DetachedQueue.remote(name="queue_1", detached=True)
counters = [Counter.remote() for i in range(4)]
[c.increment.remote() for c in counters]
futures = [c.read.remote() for c in counters]
print(futures, type(futures[0]))
queue.put.remote("key", ray.get(futures)))
while True:
pass
c_ray.py:
import ray
ray.init(address='auto', redis_password='5241590000000000')
queue = ray.util.get_actor("queue_1")
print(ray.get(queue.get.remote()))