TL;DR
We can write a function wrap a sync function an async function:
import asyncio from functools import wraps, partial def async_wrap(func): @wraps(func) async def run(*args, loop=None, executor=None, **kwargs): if loop is None: loop = asyncio.get_event_loop() pfunc = partial(func, *args, **kwargs) return await loop.run_in_executor(executor, pfunc) return run To use it:
import time import os async_sleep = async_wrap(time.sleep) async_remove = async_wrap(os.remove) # or use decorator style @async_wrap def my_async_sleep(duration): time.sleep(duration) Longer Description
If we use sync function time.sleep in a function and run it 3 times:
import time def count(): print("func start") time.sleep(1) print("func end") def main(): funcs = [count, count, count] for func in funcs: func() if __name__ == "__main__": start = time.time() main() end = time.time() print(f"Time elapse: {end-start}") Run it:
func start func end func start func end func start func end Time elapse: 3.00303053855896 The total time cost is around 3 seconds.
Now let's use the async version of time.sleep (We can use asyncio.sleep from Python's asyncio standard library, but here to demonstrate our async_wrap function works, we are going to use our own async_sleep function).
import asyncio from functools import wraps, partial import time def async_wrap(func): @wraps(func) async def run(*args, loop=None, executor=None, **kwargs): if loop is None: loop = asyncio.get_event_loop() pfunc = partial(func, *args, **kwargs) return await loop.run_in_executor(executor, pfunc) return run async_sleep = async_wrap(time.sleep) async def count(): print("func start") await async_sleep(1) print("func end") async def main(): await asyncio.gather(count(), count(), count()) if __name__ == "__main__": start = time.time() asyncio.run(main()) end = time.time() print(f"Time elapse: {end-start}") Here we use asyncio.gather to run the count function three times. Now run it:
func start func start func start func end func end func end Time elapse: 1.007828950881958 We can see our async version only cost around 1 second! And our async_sleep function works!
Top comments (1)
Pay attention that this code,as is, will only enable multi-threading of the sync functions.
To be able to run the function in a sub-process by using an explicit ProcessPoolExecutor, the original function, prior to been decorated, have to be preserved - and the decorated async-function needs to exist with a different name.
I just made it work for someone who has hit this: stackoverflow.com/questions/743598...