當(dāng)前位置：首頁(yè) > 编程语言 > python >内容正文

python

Python 内置模块之 asyncio（异步iO）

發(fā)布時(shí)間：2023/12/20 python 34 豆豆

生活随笔收集整理的這篇文章主要介紹了 Python 内置模块之 asyncio（异步iO）小編覺(jué)得挺不錯(cuò)的,現(xiàn)在分享給大家,幫大家做個(gè)參考.

python3.0，標(biāo)準(zhǔn)庫(kù)里的異步網(wǎng)絡(luò)模塊：select(非常底層) ，第三方異步網(wǎng)絡(luò)庫(kù)：Tornado，gevent

python3.4，asyncio：支持 TCP ，子進(jìn)程

現(xiàn)在的asyncio，有了很多的模塊已經(jīng)在支持：aiohttp,aiodns,aioredis等等 https://github.com/aio-libs 這里列出了已經(jīng)支持的內(nèi)容，并在持續(xù)更新。

asyncio的使用上，感覺(jué)和gevent有異曲同工之妙

1、基礎(chǔ)概念：

event_loop 事件循環(huán)：理解為一個(gè)循環(huán)的池，里面存放一些async關(guān)鍵詞定義的協(xié)程函數(shù)，只有放到循環(huán)池里才能執(zhí)行
coroutine 協(xié)程：協(xié)程對(duì)象，指一個(gè)使用async關(guān)鍵字定義的函數(shù)，它的調(diào)用不會(huì)立即執(zhí)行函數(shù)，而是會(huì)返回一個(gè)協(xié)程對(duì)象。協(xié)程對(duì)象需要注冊(cè)到事件循環(huán)，由事件循環(huán)調(diào)用。
task 任務(wù)：一個(gè)協(xié)程對(duì)象就是一個(gè)原生可以掛起的函數(shù)，任務(wù)則是對(duì)協(xié)程進(jìn)一步封裝，其中包含任務(wù)的各種狀態(tài)。
future：代表將來(lái)執(zhí)行或沒(méi)有執(zhí)行的任務(wù)的結(jié)果。它和task上沒(méi)有本質(zhì)的區(qū)別
async/await 關(guān)鍵字：python3.5 用于定義協(xié)程的關(guān)鍵字，async定義一個(gè)協(xié)程，await用于掛起阻塞的異步調(diào)用接口。

來(lái)看一個(gè)簡(jiǎn)單是示例

import time import asyncionow = lambda : time.time() async def do_some_work(x): ? ?# 使用async關(guān)鍵字定義協(xié)程，當(dāng)然協(xié)程不能直接運(yùn)行，需要將協(xié)程加入到事件循環(huán)loop中print('Waiting: ', x) start = now() coroutine = do_some_work(2) ? ? # 這里是一個(gè)協(xié)程對(duì)象，這個(gè)時(shí)候do_some_work函數(shù)并沒(méi)有執(zhí)行 loop = asyncio.get_event_loop() ? ? # 第一步：創(chuàng)建一個(gè)事件循環(huán)（池） loop.run_until_complete(coroutine) ? ?# 第二步：將協(xié)程加入到事件循環(huán)loop，并啟動(dòng)事件循環(huán)2 Waiting: ?2 TIME: ?0.0004658699035644531

創(chuàng)建task

協(xié)程對(duì)象不能直接運(yùn)行，需要包裝成任務(wù)才能運(yùn)行，上面是通過(guò)run_until_complete()方法包裝成task（隱式包裝），還有下面兩種方式進(jìn)行顯式包裝：

import asyncio import timenow = lambda : time.time() async def do_some_work(x):print('Waiting: ', x) start = now() coroutine = do_some_work(2)loop = asyncio.get_event_loop() # task = asyncio.ensure_future(coroutine) # 方式一 task = loop.create_task(coroutine) # 方式二 print(task)loop.run_until_complete(task) print(task) print('TIME: ', now() - start)# 以下為輸出 2 3 4 <Task pending coro=<do_some_work() running at /Users/ghost/Rsj217/python3.6/async/async-main.py:17>> Waiting: 2 <Task finished coro=<do_some_work() done, defined at /Users/ghost/Rsj217/python3.6/async/async-main.py:17> result=None> TIME: 0.0003490447998046875

創(chuàng)建task后，task在加入事件循環(huán)之前是pending狀態(tài)，加入loop后運(yùn)行中是running狀態(tài)，loop調(diào)用完是Done，運(yùn)行完是finished狀態(tài)，雖說(shuō)本質(zhì)上協(xié)程函數(shù)和task指的東西都一樣，但是task有了協(xié)程函數(shù)的狀態(tài)。

其中l(wèi)oop.run_until_complete()接受一個(gè)future參數(shù)，futurn具體指代一個(gè)協(xié)程函數(shù)，而task是future的子類，所以我們不聲明一個(gè)task直接傳入?yún)f(xié)程函數(shù)也能執(zhí)行。

關(guān)于上面通過(guò)loop.create_task(coroutine)創(chuàng)建task,同樣的可以通過(guò) asyncio.ensure_future(coroutine)創(chuàng)建task

關(guān)于這兩個(gè)命令的官網(wǎng)解釋： https://docs.python.org/3/library/asyncio-task.html#asyncio.ensure_future

綁定回調(diào)函數(shù)

通過(guò)task的task.add_done_callback(callback)方法綁定回調(diào)函數(shù)，回調(diào)函數(shù)接收一個(gè)future對(duì)象參數(shù)如task，在內(nèi)部通過(guò)future.result()獲得協(xié)程函數(shù)的返回值。

import asyncioasync def test(x):return x+3 def callback(y):print(y.result()) coroutine = test(5) loop = asyncio.get_event_loop() task = loop.create_task(coroutine) task.add_done_callback(callback) loop.run_until_complete(task)

通過(guò)add_done_callback方法給task任務(wù)添加回調(diào)函數(shù)，當(dāng)task（也可以說(shuō)是coroutine）執(zhí)行完成的時(shí)候,就會(huì)調(diào)用回調(diào)函數(shù)。并通過(guò)參數(shù)future獲取協(xié)程執(zhí)行的結(jié)果。這里我們創(chuàng)建的task和回調(diào)里的future對(duì)象實(shí)際上是同一個(gè)對(duì)象

await（掛起耗時(shí)操作）

多任務(wù)聲明了協(xié)程函數(shù)，也同時(shí)在loop中注冊(cè)了，他的執(zhí)行也是順序執(zhí)行的，因?yàn)樵诋惒胶瘮?shù)中沒(méi)有聲明那些操作是耗時(shí)操作，所以會(huì)順序執(zhí)行。await的作用就是告訴控制器這個(gè)步驟是耗時(shí)的，async可以定義協(xié)程對(duì)象，使用await可以針對(duì)耗時(shí)的操作進(jìn)行掛起

import asyncio import timeasync def test(1):time.sleep(1)print(time.time()) tasks = [asyncio.ensure_future(test()) for _ in range(3)] loop = asyncio.get_event_loop() loop.run_until_complete(asyncio.wait(tasks))1547187398.7611663 1547187399.7611988 1547187400.7632194Out[8]:? ({<Task finished coro=<test() done, defined at <ipython-input-5-1534f9ca2d8e>:4> result=None>,<Task finished coro=<test() done, defined at <ipython-input-5-1534f9ca2d8e>:4> result=None>,<Task finished coro=<test() done, defined at <ipython-input-5-1534f9ca2d8e>:4> result=None>},set())

上面執(zhí)行并不是異步執(zhí)行，而是順序執(zhí)行，但是改成下面形式那就是異步執(zhí)行：

import asyncio import time async def test(t):await asyncio.sleep(1) # asyncio 的sleepprint(time.time()) tasks = [asyncio.ensure_future(test()) for _ in range(3)] loop = asyncio.get_event_loop() loop.run_until_complete(asyncio.wait(tasks))1547187398.7611663 1547187399.7611988 1547187400.7632194 Out[11]:? ({<Task finished coro=<test() done, defined at <ipython-input-9-3a874803716b>:4> result=None>,<Task finished coro=<test() done, defined at <ipython-input-9-3a874803716b>:4> result=None>,<Task finished coro=<test() done, defined at <ipython-input-9-3a874803716b>:4> result=None>},set())

可見(jiàn)三個(gè)任務(wù)的間隔時(shí)間幾乎忽略不計(jì)，這里要注意可以使用await成功掛起的對(duì)應(yīng)應(yīng)該是下面三種：

原生異步函數(shù)（coroutine ）
由 types.coroutine() 修飾的生成器，這個(gè)生成器可以返回 coroutine 對(duì)象。
包含 __await 方法的對(duì)象返回的一個(gè)迭代器

所以即使使用saync修飾requests的方法也不支持異步，而是需要專門的異步網(wǎng)絡(luò)請(qǐng)求庫(kù)aiohttp，aiodns,aioredis。

aiohttp

aiohttp需要單獨(dú)安裝，然后和asyncio庫(kù)一起使用，看一下案例

async def get(url):async with aiohttp.ClientSession() as session:async with session.get(url) as response:print(response)print(time.time())import time async def request():url = "http://www.baidu.com"resulit = await get(url)tasks = [asyncio.ensure_future(request()) for _ in range(10)] loop = asyncio.get_event_loop() loop.run_until_complete(asyncio.wait(tasks))<ClientResponse(http://www.baidu.com) [200 OK]> <CIMultiDictProxy('Bdpagetype': '1', 'Bdqid': '0x94343a8f0000d2ac', 'Cache-Control': 'private', 'Connection': 'Keep-Alive', 'Content-Encoding': 'gzip', 'Content-Type': 'text/html', 'Cxy_all': 'baidu+fcb1f5fc4ea50a8475457d9dba4ffb75', 'Date': 'Fri, 11 Jan 2019 07:20:37 GMT', 'Expires': 'Fri, 11 Jan 2019 07:19:54 GMT', 'P3p': 'CP=" OTI DSP COR IVA OUR IND COM "', 'Server': 'BWS/1.1', 'Set-Cookie': 'BAIDUID=76DA9E559DEFD4059F7858332E63E4CA:FG=1; expires=Thu, 31-Dec-37 23:55:55 GMT; max-age=2147483647; path=/; domain=.baidu.com', 'Set-Cookie': 'BIDUPSID=76DA9E559DEFD4059F7858332E63E4CA; expires=Thu, 31-Dec-37 23:55:55 GMT; max-age=2147483647; path=/; domain=.baidu.com', 'Set-Cookie': 'PSTM=1547191237; expires=Thu, 31-Dec-37 23:55:55 GMT; max-age=2147483647; path=/; domain=.baidu.com', 'Set-Cookie': 'delPer=0; path=/; domain=.baidu.com', 'Set-Cookie': 'BDSVRTM=0; path=/', 'Set-Cookie': 'BD_HOME=0; path=/', 'Set-Cookie': 'H_PS_PSSID=26525_1426_21079_28132_28266; path=/; domain=.baidu.com', 'Vary': 'Accept-Encoding', 'X-Ua-Compatible': 'IE=Edge,chrome=1', 'Transfer-Encoding': 'chunked')> 1547191237.4161415 <ClientResponse(http://www.baidu.com) [200 OK]> <CIMultiDictProxy('Bdpagetype': '1', 'Bdqid': '0xb19b30e80000e08d', 'Cache-Control': 'private', 'Connection': 'Keep-Alive', 'Content-Encoding': 'gzip', 'Content-Type': 'text/html', 'Cxy_all': 'baidu+6035b8e98737e4cc11dcc73ec79566cc', 'Date': 'Fri, 11 Jan 2019 07:20:37 GMT', 'Expires': 'Fri, 11 Jan 2019 07:19:48 GMT', 'P3p': 'CP=" OTI DSP COR IVA OUR IND COM "', 'Server': 'BWS/1.1', 'Set-Cookie': 'BAIDUID=76DA9E559DEFD405C594443631339D6D:FG=1; expires=Thu, 31-Dec-37 23:55:55 GMT; max-age=2147483647; path=/; domain=.baidu.com', 'Set-Cookie': 'BIDUPSID=76DA9E559DEFD405C594443631339D6D; expires=Thu, 31-Dec-37 23:55:55 GMT; max-age=2147483647; path=/; domain=.baidu.com', 'Set-Cookie': 'PSTM=1547191237; expires=Thu, 31-Dec-37 23:55:55 GMT; max-age=2147483647; path=/; domain=.baidu.com', 'Set-Cookie': 'delPer=0; path=/; domain=.baidu.com', 'Set-Cookie': 'BDSVRTM=0; path=/', 'Set-Cookie': 'BD_HOME=0; path=/', 'Set-Cookie': 'H_PS_PSSID=26522_1423_21104_28132_28267_22158; path=/; domain=.baidu.com', 'Vary': 'Accept-Encoding', 'X-Ua-Compatible': 'IE=Edge,chrome=1', 'Transfer-Encoding': 'chunked')> 1547191237.417142 <ClientResponse(http://www.baidu.com) [200 OK]> <CIMultiDictProxy('Bdpagetype': '1', 'Bdqid': '0xfdf776e30000dfb4', 'Cache-Control': 'private', 'Connection': 'Keep-Alive', 'Content-Encoding': 'gzip', 'Content-Type': 'text/html', 'Cxy_all': 'baidu+0810232ebbebf660004801978cbc7056', 'Date': 'Fri, 11 Jan 2019 07:20:37 GMT', 'Expires': 'Fri, 11 Jan 2019 07:20:15 GMT', 'P3p': 'CP=" OTI DSP COR IVA OUR IND COM "', 'Server': 'BWS/1.1', 'Set-Cookie': 'BAIDUID=76DA9E559DEFD40584DF85554050AB79:FG=1; expires=Thu, 31-Dec-37 23:55:55 GMT; max-age=2147483647; path=/; domain=.baidu.com', 'Set-Cookie': 'BIDUPSID=76DA9E559DEFD40584DF85554050AB79; expires=Thu, 31-Dec-37 23:55:55 GMT; max-age=2147483647; path=/; domain=.baidu.com', 'Set-Cookie': 'PSTM=1547191237; expires=Thu, 31-Dec-37 23:55:55 GMT; max-age=2147483647; path=/; domain=.baidu.com', 'Set-Cookie': 'delPer=0; path=/; domain=.baidu.com', 'Set-Cookie': 'BDSVRTM=0; path=/', 'Set-Cookie': 'BD_HOME=0; path=/', 'Set-Cookie': 'H_PS_PSSID=1465_21118_28131_28267_20718; path=/; domain=.baidu.com', 'Vary': 'Accept-Encoding', 'X-Ua-Compatible': 'IE=Edge,chrome=1', 'Transfer-Encoding': 'chunked')> 1547191237.4221385 <ClientResponse(http://www.baidu.com) [200 OK]> <CIMultiDictProxy('Bdpagetype': '1', 'Bdqid': '0x879158430000a46a', 'Cache-Control': 'private', 'Connection': 'Keep-Alive', 'Content-Encoding': 'gzip', 'Content-Type': 'text/html', 'Cxy_all': 'baidu+acdef638e6acee7494d7fce1008c87ca', 'Date': 'Fri, 11 Jan 2019 07:20:37 GMT', 'Expires': 'Fri, 11 Jan 2019 07:20:03 GMT', 'P3p': 'CP=" OTI DSP COR IVA OUR IND COM "', 'Server': 'BWS/1.1', 'Set-Cookie': 'BAIDUID=76DA9E559DEFD40593C8E085477DD125:FG=1; expires=Thu, 31-Dec-37 23:55:55 GMT; max-age=2147483647; path=/; domain=.baidu.com', 'Set-Cookie': 'BIDUPSID=76DA9E559DEFD40593C8E085477DD125; expires=Thu, 31-Dec-37 23:55:55 GMT; max-age=2147483647; path=/; domain=.baidu.com', 'Set-Cookie': 'PSTM=1547191237; expires=Thu, 31-Dec-37 23:55:55 GMT; max-age=2147483647; path=/; domain=.baidu.com', 'Set-Cookie': 'delPer=0; path=/; domain=.baidu.com', 'Set-Cookie': 'BDSVRTM=0; path=/', 'Set-Cookie': 'BD_HOME=0; path=/', 'Set-Cookie': 'H_PS_PSSID=1448_21109_28131_28267; path=/; domain=.baidu.com', 'Vary': 'Accept-Encoding', 'X-Ua-Compatible': 'IE=Edge,chrome=1', 'Transfer-Encoding': 'chunked')> 1547191237.424138 <ClientResponse(http://www.baidu.com) [200 OK]> <CIMultiDictProxy('Bdpagetype': '1', 'Bdqid': '0xe5c481900000cd70', 'Cache-Control': 'private', 'Connection': 'Keep-Alive', 'Content-Encoding': 'gzip', 'Content-Type': 'text/html', 'Cxy_all': 'baidu+fb1596a42119b92bcb6a321cfd1bde58', 'Date': 'Fri, 11 Jan 2019 07:20:37 GMT', 'Expires': 'Fri, 11 Jan 2019 07:19:51 GMT', 'P3p': 'CP=" OTI DSP COR IVA OUR IND COM "', 'Server': 'BWS/1.1', 'Set-Cookie': 'BAIDUID=76DA9E559DEFD405BD554041F5821AB7:FG=1; expires=Thu, 31-Dec-37 23:55:55 GMT; max-age=2147483647; path=/; domain=.baidu.com', 'Set-Cookie': 'BIDUPSID=76DA9E559DEFD405BD554041F5821AB7; expires=Thu, 31-Dec-37 23:55:55 GMT; max-age=2147483647; path=/; domain=.baidu.com', 'Set-Cookie': 'PSTM=1547191237; expires=Thu, 31-Dec-37 23:55:55 GMT; max-age=2147483647; path=/; domain=.baidu.com', 'Set-Cookie': 'delPer=0; path=/; domain=.baidu.com', 'Set-Cookie': 'BDSVRTM=0; path=/', 'Set-Cookie': 'BD_HOME=0; path=/', 'Set-Cookie': 'H_PS_PSSID=1448_21105_18560_28132_28266_20719; path=/; domain=.baidu.com', 'Vary': 'Accept-Encoding', 'X-Ua-Compatible': 'IE=Edge,chrome=1', 'Transfer-Encoding': 'chunked')> 1547191237.4261389 <ClientResponse(http://www.baidu.com) [200 OK]> <CIMultiDictProxy('Bdpagetype': '1', 'Bdqid': '0x85ab35690000c4fd', 'Cache-Control': 'private', 'Connection': 'Keep-Alive', 'Content-Encoding': 'gzip', 'Content-Type': 'text/html', 'Cxy_all': 'baidu+15e5fc3bd83c4ffcdf9698e3264f7621', 'Date': 'Fri, 11 Jan 2019 07:20:37 GMT', 'Expires': 'Fri, 11 Jan 2019 07:20:00 GMT', 'P3p': 'CP=" OTI DSP COR IVA OUR IND COM "', 'Server': 'BWS/1.1', 'Set-Cookie': 'BAIDUID=76DA9E559DEFD405C594443631339D6D:FG=1; expires=Thu, 31-Dec-37 23:55:55 GMT; max-age=2147483647; path=/; domain=.baidu.com', 'Set-Cookie': 'BIDUPSID=76DA9E559DEFD405C594443631339D6D; expires=Thu, 31-Dec-37 23:55:55 GMT; max-age=2147483647; path=/; domain=.baidu.com', 'Set-Cookie': 'PSTM=1547191237; expires=Thu, 31-Dec-37 23:55:55 GMT; max-age=2147483647; path=/; domain=.baidu.com', 'Set-Cookie': 'delPer=0; path=/; domain=.baidu.com', 'Set-Cookie': 'BDSVRTM=0; path=/', 'Set-Cookie': 'BD_HOME=0; path=/', 'Set-Cookie': 'H_PS_PSSID=26522_1423_21104_28132_28267_22158; path=/; domain=.baidu.com', 'Vary': 'Accept-Encoding', 'X-Ua-Compatible': 'IE=Edge,chrome=1', 'Transfer-Encoding': 'chunked')> 1547191237.428144 <ClientResponse(http://www.baidu.com) [200 OK]> <CIMultiDictProxy('Bdpagetype': '1', 'Bdqid': '0x9620ed6b0000f26c', 'Cache-Control': 'private', 'Connection': 'Keep-Alive', 'Content-Encoding': 'gzip', 'Content-Type': 'text/html', 'Cxy_all': 'baidu+a2bfd2645e7c3d7514192a060f9644f5', 'Date': 'Fri, 11 Jan 2019 07:20:37 GMT', 'Expires': 'Fri, 11 Jan 2019 07:20:12 GMT', 'P3p': 'CP=" OTI DSP COR IVA OUR IND COM "', 'Server': 'BWS/1.1', 'Set-Cookie': 'BAIDUID=76DA9E559DEFD4055EFEDF62083FAFD3:FG=1; expires=Thu, 31-Dec-37 23:55:55 GMT; max-age=2147483647; path=/; domain=.baidu.com', 'Set-Cookie': 'BIDUPSID=76DA9E559DEFD4055EFEDF62083FAFD3; expires=Thu, 31-Dec-37 23:55:55 GMT; max-age=2147483647; path=/; domain=.baidu.com', 'Set-Cookie': 'PSTM=1547191237; expires=Thu, 31-Dec-37 23:55:55 GMT; max-age=2147483647; path=/; domain=.baidu.com', 'Set-Cookie': 'delPer=0; path=/; domain=.baidu.com', 'Set-Cookie': 'BDSVRTM=0; path=/', 'Set-Cookie': 'BD_HOME=0; path=/', 'Set-Cookie': 'H_PS_PSSID=1427_21127_28132_28267; path=/; domain=.baidu.com', 'Vary': 'Accept-Encoding', 'X-Ua-Compatible': 'IE=Edge,chrome=1', 'Transfer-Encoding': 'chunked')> 1547191237.4291408 <ClientResponse(http://www.baidu.com) [200 OK]> <CIMultiDictProxy('Bdpagetype': '1', 'Bdqid': '0x912a1be40000e841', 'Cache-Control': 'private', 'Connection': 'Keep-Alive', 'Content-Encoding': 'gzip', 'Content-Type': 'text/html', 'Cxy_all': 'baidu+599a770e18be144be77bd13c371daf0a', 'Date': 'Fri, 11 Jan 2019 07:20:37 GMT', 'Expires': 'Fri, 11 Jan 2019 07:20:35 GMT', 'P3p': 'CP=" OTI DSP COR IVA OUR IND COM "', 'Server': 'BWS/1.1', 'Set-Cookie': 'BAIDUID=76DA9E559DEFD405106191D066098188:FG=1; expires=Thu, 31-Dec-37 23:55:55 GMT; max-age=2147483647; path=/; domain=.baidu.com', 'Set-Cookie': 'BIDUPSID=76DA9E559DEFD405106191D066098188; expires=Thu, 31-Dec-37 23:55:55 GMT; max-age=2147483647; path=/; domain=.baidu.com', 'Set-Cookie': 'PSTM=1547191237; expires=Thu, 31-Dec-37 23:55:55 GMT; max-age=2147483647; path=/; domain=.baidu.com', 'Set-Cookie': 'delPer=0; path=/; domain=.baidu.com', 'Set-Cookie': 'BDSVRTM=0; path=/', 'Set-Cookie': 'BD_HOME=0; path=/', 'Set-Cookie': 'H_PS_PSSID=1424_21111_28132_28266; path=/; domain=.baidu.com', 'Vary': 'Accept-Encoding', 'X-Ua-Compatible': 'IE=Edge,chrome=1', 'Transfer-Encoding': 'chunked')> 1547191237.4311435 <ClientResponse(http://www.baidu.com) [200 OK]> <CIMultiDictProxy('Bdpagetype': '1', 'Bdqid': '0x943943940000b92b', 'Cache-Control': 'private', 'Connection': 'Keep-Alive', 'Content-Encoding': 'gzip', 'Content-Type': 'text/html', 'Cxy_all': 'baidu+17014bf10c56f72b235b529f8f9c177b', 'Date': 'Fri, 11 Jan 2019 07:20:37 GMT', 'Expires': 'Fri, 11 Jan 2019 07:20:31 GMT', 'P3p': 'CP=" OTI DSP COR IVA OUR IND COM "', 'Server': 'BWS/1.1', 'Set-Cookie': 'BAIDUID=76DA9E559DEFD40504EF38ED596AEC59:FG=1; expires=Thu, 31-Dec-37 23:55:55 GMT; max-age=2147483647; path=/; domain=.baidu.com', 'Set-Cookie': 'BIDUPSID=76DA9E559DEFD40504EF38ED596AEC59; expires=Thu, 31-Dec-37 23:55:55 GMT; max-age=2147483647; path=/; domain=.baidu.com', 'Set-Cookie': 'PSTM=1547191237; expires=Thu, 31-Dec-37 23:55:55 GMT; max-age=2147483647; path=/; domain=.baidu.com', 'Set-Cookie': 'delPer=0; path=/; domain=.baidu.com', 'Set-Cookie': 'BDSVRTM=0; path=/', 'Set-Cookie': 'BD_HOME=0; path=/', 'Set-Cookie': 'H_PS_PSSID=1446_21118_28131_26350_28267_22158; path=/; domain=.baidu.com', 'Vary': 'Accept-Encoding', 'X-Ua-Compatible': 'IE=Edge,chrome=1', 'Transfer-Encoding': 'chunked')> 1547191237.4331403 <ClientResponse(http://www.baidu.com) [200 OK]> <CIMultiDictProxy('Bdpagetype': '1', 'Bdqid': '0xfd3e1b1f0000d880', 'Cache-Control': 'private', 'Connection': 'Keep-Alive', 'Content-Encoding': 'gzip', 'Content-Type': 'text/html', 'Cxy_all': 'baidu+39d965c50587bb578c5714a0d732b2e4', 'Date': 'Fri, 11 Jan 2019 07:20:37 GMT', 'Expires': 'Fri, 11 Jan 2019 07:20:25 GMT', 'P3p': 'CP=" OTI DSP COR IVA OUR IND COM "', 'Server': 'BWS/1.1', 'Set-Cookie': 'BAIDUID=76DA9E559DEFD4059A93CF4E300A8EEB:FG=1; expires=Thu, 31-Dec-37 23:55:55 GMT; max-age=2147483647; path=/; domain=.baidu.com', 'Set-Cookie': 'BIDUPSID=76DA9E559DEFD4059A93CF4E300A8EEB; expires=Thu, 31-Dec-37 23:55:55 GMT; max-age=2147483647; path=/; domain=.baidu.com', 'Set-Cookie': 'PSTM=1547191237; expires=Thu, 31-Dec-37 23:55:55 GMT; max-age=2147483647; path=/; domain=.baidu.com', 'Set-Cookie': 'delPer=0; path=/; domain=.baidu.com', 'Set-Cookie': 'BDSVRTM=0; path=/', 'Set-Cookie': 'BD_HOME=0; path=/', 'Set-Cookie': 'H_PS_PSSID=1445_21113_28131_28267_22158; path=/; domain=.baidu.com', 'Vary': 'Accept-Encoding', 'X-Ua-Compatible': 'IE=Edge,chrome=1', 'Transfer-Encoding': 'chunked')> 1547191237.4341416

幾個(gè)任務(wù)的時(shí)間之差基本忽略不計(jì)，那親測(cè)發(fā)送一千個(gè)請(qǐng)求也就11秒完成，確實(shí)很給力。

多進(jìn)程配合使用

asyncio、aiohttp需要配合aiomultiprocess庫(kù)使用，版本要求至少3.6，貼上該庫(kù)的github上的使用示例，目前還在驗(yàn)證：

Usage
Most of aiomultiprocess mimics the standard multiprocessing module whenever possible, while accounting for places that benefit from async functionality.

import asyncio from aiohttp import request from aiomultiprocess import Processasync def put(url, params):async with request("PUT", url, params=params) as response:passasync def main():p = Process(target=put, args=("https://jreese.sh", ))await pasyncio.run(main())# If you want to get results back from that coroutine, Worker makes that available: import asyncio from aiohttp import request from aiomultiprocess import Workerasync def get(url):async with request("GET", url) as response:return await response.text("utf-8")async def main():p = Worker(target=get, args=("https://jreese.sh", ))response = await pasyncio.run(main())#If you want a managed pool of worker processes, then use Pool: import asyncio from aiohttp import request from aiomultiprocess import Poolasync def get(url):async with request("GET", url) as response:return await response.text("utf-8")async def main():urls = ["https://jreese.sh", ...]async with Pool() as pool:result = await pool.map(get, urls)asyncio.run(main())

多協(xié)程并發(fā)

使用loop.run_until_complete(syncio.wait(tasks)) 也可以使用 loop.run_until_complete(asyncio.gather(*tasks)） ,前者傳入task列表，會(huì)對(duì)task進(jìn)行解包操作。

協(xié)程嵌套

顧名思義是一個(gè)協(xié)程中調(diào)用另一個(gè)協(xié)程，但是涉及到兩個(gè)協(xié)程函數(shù)的結(jié)果處理和返回。

被調(diào)用協(xié)程返回結(jié)果有下列三種方式;

async def get(url):async with aiohttp.ClientSession() as session:async with session.get(url) as response:print(response)print(time.time()) async def request():url = "http://www.baidu.com"tasks = [asyncio.ensure_future(url) for _ in range（1000）] 方式一：dones, pendings = await asyncio.wait(tasks) # 返回future對(duì)象，不返回直接結(jié)果for task in dones:print('Task ret: ', task.result()) 方式二：results = await asyncio.gather(*tasks) # 直接返回結(jié)果方式三：for task in asyncio.as_completed(tasks):result = await taskprint('Task ret: {}'.format(result)) # 迭代方式返回結(jié)果tasks = asyncio.ensure_future(request()) loop = asyncio.get_event_loop() loop.run_until_complete(asyncio.wait(tasks))

停止協(xié)程任務(wù)

實(shí)現(xiàn)結(jié)束task有兩種方式：關(guān)閉單個(gè)task、關(guān)閉loop，涉及主要函數(shù)：

asyncio.Task.all_tasks()獲取事件循環(huán)任務(wù)列表
KeyboardInterrupt捕獲停止異常（Ctrl+C）
loop.stop()停止任務(wù)循環(huán)
task.cancel()取消單個(gè)任務(wù)
loop.run_forever()
loop.close()關(guān)閉事件循環(huán)，不然會(huì)重啟

重啟

方式一：適用于內(nèi)嵌協(xié)程函數(shù)，先取內(nèi)嵌協(xié)程任務(wù)

async def get(url): async with aiohttp.ClientSession() as session: async with session.get(url) as response: print(response) print(time.time()) async def request(): url = "http://www.baidu.com" tasks = [asyncio.ensure_future(url) for _ in range（1000）] dones, pendings = await asyncio.wait(tasks)task = asyncio.ensure_future(request()) loop = asyncio.get_event_loop() try: loop.run_until_complete(task) except KeyboardInterrupt as e: asyncio.gather(*asyncio.Task.all_tasks()).cancel() loop.stop() loop.run_forever() finally: loop.close()

方式二：適用于無(wú)內(nèi)嵌函數(shù)，直接遍歷協(xié)程任務(wù)

loop = asyncio.get_event_loop() try: loop.run_until_complete(asyncio.wait(tasks)) except KeyboardInterrupt as e: for task in asyncio.Task.all_tasks(): print(task.cancel()) loop.stop() loop.run_forever() finally: loop.close()

總結(jié)

以上是生活随笔為你收集整理的Python 内置模块之 asyncio（异步iO）的全部?jī)?nèi)容，希望文章能夠幫你解決所遇到的問(wèn)題。

如果覺(jué)得生活随笔網(wǎng)站內(nèi)容還不錯(cuò)，歡迎將生活随笔推薦給好友。

上一篇：石头机器人红灯快闪_机器人集体“快闪”活
下一篇：百度云html提取码,百度网盘提取码怎么