0.前言

(1) 拆部分reques中感興趣t的輪子

(2)對一些感興趣的pythonic寫法做一些歸納

1.用object.__setattr__來初始化構(gòu)造函數(shù)

反正我之前就是直接實(shí)例對象時(shí)把所有參數(shù)傳入構(gòu)造函數(shù)的,一般人都這樣..但事實(shí)證明這種方式并不好(可能),所以后來作者又把這種方式改掉了...但原諒我也不知道這兩者有什么好壞之分..

class Request(object):"""The :class:`Request` object. It carries out all functionality ofRequests. Recommended interface is with the Requests functions."""_METHODS = ('GET', 'HEAD', 'PUT', 'POST', 'DELETE')def __init__(self):self.url = Noneself.headers = dict() self.method = None self.params = {} self.data = {} self.response = Response() self.auth = None self.sent = False def __repr__(self): try: repr = '<Request [%s]>' % (self.method) except: repr = '<Request object>' return repr def __setattr__(self, name, value): if (name == 'method') and (value): if not value in self._METHODS: raise InvalidMethod() object.__setattr__(self, name, value)

初始化操作:

def get(url, params={}, headers={}, auth=None):"""Sends a GET request. Returns :class:`Response` object.:param url: URL for the new :class:`Request` object.:param params: (optional) Dictionary of GET Parameters to send with the :class:`Request`.:param headers: (optional) Dictionary of HTTP Headers to sent with the :class:`Request`.:param auth: (optional) AuthObject to enable Basic HTTP Auth."""r = Request()r.method = 'GET'r.url = urlr.params = params r.headers = headers r.auth = _detect_auth(url, auth) r.send() return r.response

2.大量復(fù)雜的參數(shù)傳遞時(shí)采用**kwargs

用**kwargs可在方法間的傳遞大量參數(shù)，不需要自己每次都初始化一個(gè)dict用來傳參(嗯,之前我就是這樣的傻逼)

def get(url, params={}, headers={}, cookies=None, auth=None):return request('GET', url, params=params, headers=headers, cookiejar=cookies, auth=auth)def request(method, url, **kwargs):data = kwargs.pop('data', dict()) or kwargs.pop('params', dict())r = Request(method=method, url=url, data=data, headers=kwargs.pop('headers', {}),cookiejar=kwargs.pop('cookies', None), files=kwargs.pop('files', None), auth=kwargs.pop('auth', auth_manager.get_auth(url))) r.send() return r.response

3.monkey patch

熱修復(fù)技術(shù)方案，可以參考協(xié)程,協(xié)程為了實(shí)現(xiàn)異步效果,替換了python原生的很多庫。就是模塊在加載前，把自己的模塊在系統(tǒng)加載前替換掉原系統(tǒng)模塊，然后達(dá)到自己的（不可告人的）目的。

這里其實(shí)不是requests使用了monkey patch,而是pyopenssl這個(gè)庫,這個(gè)是為了修復(fù)python2.7中SNI的bug,將原來的ssl_wrap_socket方法做了替換(不過我沒看到requests有任何注入操作,坑爹...)

# 替換 def inject_into_urllib3():'Monkey-patch urllib3 with PyOpenSSL-backed SSL-support.'connection.ssl_wrap_socket = ssl_wrap_socketutil.HAS_SNI = HAS_SNIutil.IS_PYOPENSSL = True# 還原 def extract_from_urllib3(): 'Undo monkey-patching by :func:`inject_into_urllib3`.' connection.ssl_wrap_socket = orig_connection_ssl_wrap_socket util.HAS_SNI = orig_util_HAS_SNI util.IS_PYOPENSSL = False

如果在請求https過程中出現(xiàn)SNIMissing的問題,可以考慮這么解決:

pip install pyopenssl ndg-httpsclient pyasn1try:import urllib3.contrib.pyopensslurllib3.contrib.pyopenssl.inject_into_urllib3() except ImportError:pass

相當(dāng)于就是執(zhí)行主動(dòng)注入的操作(但這個(gè)不應(yīng)該是requests框架自己該集成的么...)

4.hook函數(shù)

requests中有一個(gè)鉤子函數(shù)，看歷史版本，原來提供的回調(diào)入口有好幾個(gè)，目前只有response一個(gè)回調(diào)入口了，測試代碼如下

import requestsdef print_url(r, *args, **kwargs):print r.contentprint r.urlrequests.get('http://httpbin.org', hooks=dict(response=print_url))

這會(huì)發(fā)生什么呢？requests會(huì)在requests.Response返回前回調(diào)這個(gè)print_url這個(gè)方法。可以看到，回調(diào)操作是在requests拿到請求結(jié)果后才去操作的

def send(self, request, **kwargs):"""Send a given PreparedRequest.:rtype: requests.Response"""...# Get the appropriate adapter to useadapter = self.get_adapter(url=request.url)# Start time (approximately) of the requeststart = datetime.utcnow()# Send the requestr = adapter.send(request, **kwargs)# Total elapsed time of the request (approximately)r.elapsed = datetime.utcnow() - start # Response manipulation hooks r = dispatch_hook('response', hooks, r, **kwargs)

?

那dispatch_hook又干了什么呢？

def dispatch_hook(key, hooks, hook_data, **kwargs):"""Dispatches a hook dictionary on a given piece of data."""hooks = hooks or dict()hooks = hooks.get(key)if hooks:if hasattr(hooks, '__call__'): hooks = [hooks] for hook in hooks: _hook_data = hook(hook_data, **kwargs) if _hook_data is not None: hook_data = _hook_data return hook_data

可以看到dispatch_hook本身是可以拓展的，但可惜的是目前requests只有response入口了，也許是為了安全吧。

其實(shí)說真的,requests的hook使用起來真的不夠好,真正好用的hook,可以看看flask.

5.上下文管理器（歷史版本）

with requests.settings(timeout=0.5):requests.get('http://example.org')requests.get('http://example.org', timeout=10)

在with之中，所有的配置加載都是在局部生效的，就算requests.get('http://example.org', timeout=10)，但requests對象中的timeout屬性依然是0.5而不是10，怎么實(shí)現(xiàn)的呢？

class settings:"""Context manager for settings."""cache = {}def __init__(self, timeout):self.module = inspect.getmodule(self)# Cache settingsself.cache['timeout'] = self.module.timeout self.module.timeout = timeout def __enter__(self): pass def __exit__(self, type, value, traceback): # Restore settings for key in self.cache: setattr(self.module, key, self.cache[key])

其實(shí)很簡單，只要在進(jìn)入這個(gè)context時(shí)，將原有的屬性儲存起來，退出context時(shí)，重新set回去就行了。

6.重定向redirect

requests對每一個(gè)send請求都會(huì)做重定向的判斷,具體就是如果是重定向,那就執(zhí)行以下這個(gè)方法

def resolve_redirects(self, resp, req, stream=False, timeout=None,verify=True, cert=None, proxies=None, **adapter_kwargs):"""Receives a Response. Returns a generator of Responses."""i = 0hist = [] # keep track of historywhile resp.is_redirect:prepared_request = req.copy() if i > 0: # Update history and keep track of redirects. hist.append(resp) new_hist = list(hist) resp.history = new_hist 　　　　　　　... url = resp.headers['location'] # Handle redirection without scheme (see: RFC 1808 Section 4) if url.startswith('//'): parsed_rurl = urlparse(resp.url) url = '%s:%s' % (parsed_rurl.scheme, url) 　　　　　　　... extract_cookies_to_jar(prepared_request._cookies, req, resp.raw) prepared_request._cookies.update(self.cookies) prepared_request.prepare_cookies(prepared_request._cookies) # Rebuild auth and proxy information. proxies = self.rebuild_proxies(prepared_request, proxies) self.rebuild_auth(prepared_request, resp) # Override the original request. req = prepared_request resp = self.send( req, stream=stream, timeout=timeout, verify=verify, cert=cert, proxies=proxies, allow_redirects=False, **adapter_kwargs ) extract_cookies_to_jar(self.cookies, prepared_request, resp.raw) i += 1 yield resp

可以看到,requests會(huì)從url = resp.headers['location']取出重定向后的url,將resp追加到history中,然后重設(shè)head,cookie,proxy,auth執(zhí)行self.send操作,然后yield resp后進(jìn)入下一次循環(huán),判斷是否是redirect,最多redirect次數(shù)為30次.

?

轉(zhuǎn)載于:https://www.cnblogs.com/alexkn/p/6266573.html

總結(jié)

以上是生活随笔為你收集整理的requests源码分析的全部內(nèi)容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網(wǎng)站內(nèi)容還不錯(cuò)，歡迎將生活随笔推薦給好友。