IO多路復用

作用:檢測多個socket是否已經發生變化(是否已經連接成功/是否已經獲取數據)

setblocking(Flase)

讓原本阻塞的變成非阻塞(不等待)狀態

import socket import selectsk = socket.socket() sk.setblocking(False) try:sk.connect(("www.baidu.com",80)) except BlockingIOError as e:passsk1 = socket.socket() sk1.setblocking(False) try:sk1.connect(("www.douban.com",80)) except BlockingIOError as e:passsk2 = socket.socket() sk2.setblocking(False) try:sk2.connect(("www.zhihu.com",80)) except BlockingIOError as e:pass

基于IO多路復用+socket實現并發請求

單線程的并發

import socket import selectclient1 = socket.socket() client1.setblocking(False) # 百度創建連接: 非阻塞try:client1.connect(('www.baidu.com',80)) except BlockingIOError as e:passclient2 = socket.socket() client2.setblocking(False) # 百度創建連接: 非阻塞 try:client2.connect(('www.sogou.com',80)) except BlockingIOError as e:passclient3 = socket.socket() client3.setblocking(False) # 百度創建連接: 非阻塞 try:client3.connect(('www.oldboyedu.com',80)) except BlockingIOError as e:passsocket_list = [client1,client2,client3] conn_list = [client1,client2,client3]while True:rlist,wlist,elist = select.select(socket_list,conn_list,[],0.005)# wlist中表示已經連接成功的socket對象for sk in wlist:if sk == client1:sk.sendall(b'GET /s?wd=alex HTTP/1.0\r\nhost:www.baidu.com\r\n\r\n')elif sk==client2:sk.sendall(b'GET /web?query=fdf HTTP/1.0\r\nhost:www.sogou.com\r\n\r\n')else:sk.sendall(b'GET /s?wd=alex HTTP/1.0\r\nhost:www.oldboyedu.com\r\n\r\n')conn_list.remove(sk)for sk in rlist:chunk_list = []while True:try:chunk = sk.recv(8096)if not chunk:breakchunk_list.append(chunk)except BlockingIOError as e:breakbody = b''.join(chunk_list)# print(body.decode('utf-8'))print('------------>',body)sk.close()socket_list.remove(sk)if not socket_list:break

單線程的并發高級

基于事件循環實現的異步非阻塞框架:lzl

Python中有開源 基于事件循環實現的異步非阻塞框架 Twisted?

非阻塞:讓過程不再等待,但是會報BLockIOError的錯誤,只要捕獲就可以

異步：執行完某個任務后自動調用我給他的函數,也就是回調函數

import socket import selectclass Req(object):def __init__(self,sk,func):self.sock = skself.func = funcdef fileno(self):return self.sock.fileno()class Nb(object):def __init__(self):self.conn_list = []self.socket_list = []def add(self,url,func):client = socket.socket()client.setblocking(False) # 非阻塞try:client.connect((url, 80))except BlockingIOError as e:passobj = Req(client,func)self.conn_list.append(obj)self.socket_list.append(obj)def run(self):while True:rlist,wlist,elist = select.select(self.socket_list,self.conn_list,[],0.005)# wlist中表示已經連接成功的req對象for sk in wlist:# 發生變換的req對象sk.sock.sendall(b'GET /s?wd=alex HTTP/1.0\r\nhost:www.baidu.com\r\n\r\n')self.conn_list.remove(sk)for sk in rlist:chunk_list = []while True:try:chunk = sk.sock.recv(8096)if not chunk:breakchunk_list.append(chunk)except BlockingIOError as e:breakbody = b''.join(chunk_list)# print(body.decode('utf-8'))sk.func(body)sk.sock.close()self.socket_list.remove(sk)if not self.socket_list:breakdef baidu_repsonse(body):print('百度下載結果：',body)def sogou_repsonse(body):print('搜狗下載結果：', body)def oldboyedu_repsonse(body):print('老男孩下載結果：', body)t1 = Nb() t1.add('www.baidu.com',baidu_repsonse) t1.add('www.sogou.com',sogou_repsonse) t1.add('www.oldboyedu.com',oldboyedu_repsonse) t1.run()

具體使用方法

from lzl import Nbdef baidu_repsonse(body):print('百度下載結果：',body)def sogou_repsonse(body):print('搜狗下載結果：', body)def oldboyedu_repsonse(body):print('老男孩下載結果：', body)t1 = Nb() t1.add('www.baidu.com',baidu_repsonse) t1.add('www.sogou.com',sogou_repsonse) t1.add('www.oldboyedu.com',oldboyedu_repsonse) t1.run()　

協程

本身是不存在的,是由開發人員創造出來的一個不是真實存在的東西

微線程,對一個線程/進程進行分片,使得線程在代碼塊之間進行來回切換執行,而不是原來那樣逐行執行

單純的協程是沒有什么實際用處的

import greenletdef f1():print(11)gr2.switch()print(22)gr2.switch()def f2():print(33)gr1.switch()print(44)# 協程 gr1 gr1 = greenlet.greenlet(f1) # 協程 gr2 gr2 = greenlet.greenlet(f2)# 輸出結果: 11 33 22 44# 單純的協程編寫起來還不如我直接寫函數調用來的方便呢,所以單純的協程是無用的def f1():print(11)print(33)def f2():print(22)print(44)f1() f2()# 輸出結果是一樣的: 11 33 22 44　　

協程 + IO切換,那它才真正的厲害起來

from gevent import monkey monkey.patch_all()import requests import geventdef get_path(url):ret = requests.get(url)print(url, ret.content)def get_path2(url):ret2 = requests.get(url)print("-----------",url,ret2.content)def get_path3(url):ret3 = requests.get(url)print(url,ret3.content)ret = gevent.joinall([gevent.spawn(get_path,'https://www.zhihu.com'),gevent.spawn(get_path2,"https://www.douban.com"),gevent.spawn(get_path3,"https://www.baidu.com/s?ie=utf-8&f=8&rsv_bp=0&rsv_idx=1&tn=baidu&wd=zkx"), ])

　　

轉載于:https://www.cnblogs.com/--kai/p/9643122.html

總結

以上是生活随笔為你收集整理的单线程的并发,协程的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。