請求網頁

方法一：

import asyncio import requestsasync def main():loop = asyncio.get_event_loop()future1 = loop.run_in_executor(None, requests.get, 'http://www.baidu.com')future2 = loop.run_in_executor(None, requests.get, 'http://www.python.org')response1 = await future1response2 = await future2print(response1.text)print(response2.text)loop = asyncio.get_event_loop() loop.run_until_complete(main())

方法二：

copy from liaoxuefeng
https://www.liaoxuefeng.com/wiki/0014316089557264a6b348958f449949df42a6d3a2e542c000/001432090954004980bd351f2cd4cc18c9e6c06d855c498000

import asyncio@asyncio.coroutine def wget(host):print('wget %s...' % host)connect = asyncio.open_connection(host, 80)reader, writer = yield from connectheader = 'GET / HTTP/1.0\r\nHost: %s\r\n\r\n' % hostwriter.write(header.encode('utf-8'))yield from writer.drain()while True:line = yield from reader.readline()if line == b'\r\n':breakprint('%s header > %s' % (host, line.decode('utf-8').rstrip()))# Ignore the body, close the socketwriter.close()loop = asyncio.get_event_loop() tasks = [wget(host) for host in ['www.sina.com.cn', 'www.sohu.com', 'www.163.com']] loop.run_until_complete(asyncio.wait(tasks)) loop.close()

方法三：

在一個web上，一個工程師做了一個探索。發現，當使用aiohtt+asyncio時候是比requests要快很多倍的。所以，推薦使用原裝的庫。

import aiohttp import asyncioasync def fetch(session, url):async with session.get(url) as response:return await response.text()async def main():async with aiohttp.ClientSession() as session:html = await fetch(session, 'http://python.org')print(html)loop = asyncio.get_event_loop() loop.run_until_complete(main())

aiohttp簡單測試

import aiohttp import asyncio import time import requestsasync def fetch(session, url):async with session.get(url) as response:return await response.text()async def main():async with aiohttp.ClientSession() as session:await fetch(session, 'http://python.org')st = time.time() loop = asyncio.get_event_loop() loop.run_until_complete(main()) et = time.time() print(et - st)st = time.time() res_text = requests.get('http://python.org').text et = time.time() print(et - st)

輸出結果是：

3.109215259552002 2.7337353229522705

目前來看，只爬一個頁面的話，其實反而request會快點。

但是當爬取的數目提高的時候就發送了變化了。

import aiohttp import asyncio import time import requestsasync def fetch(session, url):async with session.get(url) as response:return await response.text()async def main():async with aiohttp.ClientSession() as session:await fetch(session, 'http://python.org')st = time.time() loop = asyncio.get_event_loop() loop.run_until_complete(asyncio.wait([main(), main()])) et = time.time() print(et - st)st = time.time() res_text = requests.get('http://python.org').text res_text_2 = requests.get('http://python.org').text et = time.time() print(et - st)

輸出：

3.1400091648101807 5.478497505187988

當然啦，這是因為第一個其實是采用是協程技術，所以，差別有點大也是可以理解的。

所以，我們要接著探究。

對比 gevent和aiohttp

這個非常有意思，因為兩者都是關于協程的，關于隨者更優對比一下就可以知道結果了。

文件結構：
三個文件放在同一個目錄下：

• main_Test.py
• asyncio_Test.py
• gevent_Test.py

main_Test.py

import asyncio_Test import gevent_Test import matplotlib.pyplot as pltasyncio_list = [] gevent_list = []N = 50 Begin = 1 url = 'http://www.python.org' for i in range(N):asyncio_list.append(asyncio_Test.f(Begin + i, url))gevent_list.append(gevent_Test.f(Begin + i, url))plt.plot(asyncio_list, label='asyncio') plt.plot(gevent_list, label='gevent')plt.legend() plt.savefig('1.png') plt.show()

asyncio_Test.py

import aiohttp import asyncio import timeasync def fetch(session, url):async with session.get(url) as response:return await response.text()async def main(Target_url):async with aiohttp.ClientSession() as session:await fetch(session, Target_url)def f(Times, Target_url):st = time.time()loop = asyncio.get_event_loop()task = [main(Target_url)] * Timesloop.run_until_complete(asyncio.wait(task))et = time.time()return et - st

gevent_Test.py

import gevent from gevent import monkey import time import requests monkey.patch_socket()def request_f(Target_url):res_text = requests.get(Target_url).textdef f(Times, Target_url):st = time.time()WaitList = [gevent.spawn(request_f, Target_url)] * Timesgevent.joinall(WaitList)et = time.time()return et - st

對比效果如下：

可以發現，當只爬取一個網頁的時候，其實，反而用gevent+request會更快。
隨著每次爬取的網頁是數目的累積，只有當爬取的數目接近20個的時候，asyncio才普遍低于gevent（雖然也是基本接近）。

相差只有0.01秒左右（我直接把數值輸出來過）

會接著更新這個。。

《新程序員》：云原生和全面數字化實踐50位技術專家共同創作，文字、視頻、音頻交互閱讀

總結

以上是生活随笔為你收集整理的asyncio简单入门（二）的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。