Most of the time I am using asyncio API. But how would I create an async generator "from scratch"?
Say I have a classic generator, and I want to make it async. How can I do that?
I naively thought that I could do something like below, but it does not run asynchronously (the 3 "for loops" are running one after the other, instead of concurrently). My hope was to make some_loop() asynchronous by calling it from some_async_loop() and releasing the event loop after each iteration with asyncio.sleep(0):
#!/usr/bin/env python3
import asyncio
async def run():
task = asyncio.ensure_future(run_async_loop())
asyncio.ensure_future(run_async_loop())
asyncio.ensure_future(run_async_loop())
await task
async def run_async_loop():
async for i in some_async_loop():
print(i)
async def some_async_loop():
for i in some_loop():
yield i
asyncio.sleep(0)
def some_loop():
for i in range(10):
yield i
if __name__ == "__main__":
loop = asyncio.get_event_loop()
loop.run_until_complete(run())
I'm glad the fixing the call to asyncio.sleep() got things running.
I'm concerned partly about your run() function, because you're only await-ing on the first task, which means your code could exit before the other tasks are complete. I would suggest this:
async def run():
tasks = [run_async_loop(), run_async_loop(), run_async_loop()]
await asyncio.gather(*tasks)
I think you can also simplify your __main__ block:
if __name__ == '__main__':
asyncio.run(run())
Related
I'm learning about asycio and trying to run this script I get this error:
ValueError: a coroutine was expected, got <async_generator object
mygen at 0x7fa2af959a60>
What am I missing here?
import asyncio
async def mygen(u=10):
"""Yield powers of 2."""
i = 0
while i < int(u):
yield 2 ** i
i += 1
await asyncio.sleep(0.1)
asyncio.run(mygen(5))
The asyncio.run function expects a coroutine but mygen is an asynchronous generator.
You can try something like this:
test.py:
import asyncio
async def mygen(u=10):
"""Yield powers of 2."""
i = 0
while i < int(u):
yield 2**i
i += 1
await asyncio.sleep(0.1)
async def main():
async for i in mygen(5):
print(i)
if __name__ == "__main__":
asyncio.run(main())
Test:
$ python test.py
1
2
4
8
16
References:
What does the "yield" keyword do?
PEP 525 -- Asynchronous Generators
How can I invoke f and reach the next line?
from SomeLib import f
f()
print('never reaches')
I would prefer not to mess with the internals of 'SomeLib', but f it's a quick fix I'll do it:
def f():
asyncio.get_event_loop.run_until_complete(ag())
async def ag():
async with websockets.client.connect(...) as websocket:
:
await wait_for_recv(...)
async def wait_for_recv(...):
while True:
message = await asyncio.wait_for(websocket.recv(), timeout=time_out)
process(message)
Calling ag directly is an option, but how to do it?
I've tried using a thread.
I've tried executors.
I've tried new_event_loop.
I'm out of ideas.
I figured out a solution. I suspect it is terrible, so I would appreciate feedback.
At the callsite:
worker_thread = threading.Thread(target=worker)
worker_thread.start()
def worker:
f()
And fiddling the library:
def f():
self.loop = asyncio.new_event_loop()
asyncio.set_event_loop(self.loop)
self.loop.call_soon_threadsafe(ag)
# added this
def kill(self):
self.loop.stop()
I am trying to make a command that loops to check when a server wipes but I found out that you can't use ctx.guild.id in a tasks is there anyway to make a command loop but still be able to use guild:
#tasks.loop(seconds=120)
async def pop_status():
for x in collection.find():
if x["_id"] == client.get_guild(id):
wipe = x["pop"]
response = requests.get('https://api.battlemetrics.com/servers/' + wipe)
pass_times = response.json()
Server_wipe = pass_times['data']['attributes']['details']['rust_last_wipe']
print(Server_wipe)
There's a few options, you can pass the guild argument in the task when starting it, or you can get it inside the task:
#tasks.loop(seconds=120)
async def pop_status(guild):
# you can use the guild argument
# You have to pass it when starting it, example in a command
#bot.command()
async def start(ctx):
pop_status.start(ctx.guild)
Getting the guild in the loop itself
#tasks.loop(seconds=120)
async def pop_status():
guild = bot.get_guild(ID_HERE)
PS: You shouldn't really use the requests library, it's blocking, you should use aiohttp instead
I'm trying to build a message counter for discord.py that counts specific messages and then responds with the number of times the message was said in that day.
I have the base but I don't know how to build the actual counter... Here is my code:
import discord
from discord.ext import commands
import discord.utils
class Message_Counter(commands.Cog):
def __init__(self, client):
self.client = client
#commands.Cog.listener()
async def on_message(self, ctx, message):
if "oof" in message.content:
await ctx.send(str(counter))
elif "Thot" in message.content:
await ctx.send(str(counter))
def setup(client):
client.add_cog(Message_Counter(client))
Any help would be much appreciated. I'm using the rewrite branch of discord.py if that helps.
Basically for Thot it would respond with **Thot counter**: <number>
For oof it would respond with **oof counter**: <number>
so on so forth.
I would also like it to reset the counter on a daily basis so that around every 24 hours the counter starts over.
Using json (quick introduction to JSON here)
We want to create a json file with name counters.json in the same folder as the file(s) for your bot. Its contents should look like this:
{
"Thot": 0,
"oof": 0
}
Loading a json file into a dictionary works with the json library:
(If you have no idea what the "with open" stuff is about, here is a primer on file reading and writing operations)
import json
def load_counters():
with open('counters.json', 'r') as f:
counters = json.load(f)
return counters
Saving the dictionary back to json works in a very similar vein:
def save_counters(counters):
with open('counters.json', 'w') as f:
json.dump(counters, f)
Now that we have a way of loading and unloading our counters from json, we can change the bot code to use them:
import discord
from discord.ext import commands
import discord.utils
class Message_Counter(commands.Cog):
def __init__(self, client):
self.client = client
#commands.Cog.listener()
async def on_message(self, ctx, message):
if "oof" in message.content:
counters = load_counters()
counters["oof"] += 1
await ctx.send(str(counters["oof"]))
save_counters(counters)
elif "Thot" in message.content:
counters = load_counters()
counters["Thot"] += 1
await ctx.send(str(counters["Thot"]))
save_counters(counters)
def setup(client):
client.add_cog(Message_Counter(client))
First example:
import asyncio
async def req():
print('request')
await asyncio.sleep(1)
async def run():
print(len(asyncio.Task.all_tasks()))
asyncio.ensure_future(req())
print(len(asyncio.Task.all_tasks()))
await asyncio.sleep(2)
print(len(asyncio.Task.all_tasks()))
loop = asyncio.get_event_loop()
loop.run_until_complete(run())
The result is:
1
2
request
1
Second example:
import asyncio
async def req():
print('request')
await asyncio.sleep(1)
async def run():
print(len(asyncio.Task.all_tasks()))
t = asyncio.ensure_future(req())
print(len(asyncio.Task.all_tasks()))
await t
print(len(asyncio.Task.all_tasks()))
loop = asyncio.get_event_loop()
loop.run_until_complete(run())
The result is:
1
2
request
2
So, why in first example the last call asyncio.Task.all_tasks() return 1 and in second example it's return 2?
In other words, why in first example the task, that wrap req() was deleted from a set of all tasks for an event loop, and why it is not true for the second example.
The task is removed from all_tasks() when it is destroyed.
Add a del statement:
[...]
await t
del t
print(len(asyncio.Task.all_tasks()))
and it will yield:
1
2
request
1