第二篇：白话tornado源码之待请求阶段
生活随笔
收集整理的這篇文章主要介紹了
第二篇：白话tornado源码之待请求阶段
小編覺得挺不錯的,現(xiàn)在分享給大家,幫大家做個參考.
上篇《白話tornado源碼之一個腳本引發(fā)的血案》用上帝視角多整個框架做了一個概述，同時也看清了web框架的的本質(zhì)，下面我們從tornado程序的起始來分析其源碼。
概述
上圖是tornado程序啟動以及接收到客戶端請求后的整個過程，對于整個過程可以分為兩大部分：
- 啟動程序階段，又稱為待請求階段（上圖1、2所有系列和3.0）
- 接收并處理客戶端請求階段（上圖3系列）
簡而言之：
1、在啟動程序階段，第一步，獲取配置文件然后生成url映射（即：一個url對應(yīng)一個XXRequestHandler，從而讓XXRequestHandler來處理指定url發(fā)送的請求）；第二步，創(chuàng)建服務(wù)器socket對象并添加到epoll中；第三步，創(chuàng)建無線循環(huán)去監(jiān)聽epoll。
2、在接收并處理請求階段，第一步，接收客戶端socket發(fā)送的請求（socket.accept）；第二步，從請求中獲取請求頭信息，再然后根據(jù)請求頭中的請求url去匹配某個XXRequestHandler；第三步，匹配成功的XXRequestHandler處理請求；第四步，將處理后的請求發(fā)送給客戶端；第五步，關(guān)閉客戶端socket。
本篇的內(nèi)容主要剖析【啟動程序階段】，下面我們就來一步一步的剖析整個過程，在此階段主要是有下面重點(diǎn)標(biāo)注的三個方法來實(shí)現(xiàn)。
import tornado.ioloop import tornado.webclass MainHandler(tornado.web.RequestHandler):def get(self):self.write("Hello, world")application = tornado.web.Application([(r"/index", MainHandler), ])if __name__ == "__main__": application.listen(8888)tornado.ioloop.IOLoop.instance().start()一、application = tornado.web.Application([(xxx,xxx)])
執(zhí)行Application類的構(gòu)造函數(shù)，并傳入一個列表類型的參數(shù)，這個列表里保存的是url規(guī)則和對應(yīng)的處理類，即：當(dāng)客戶端的請求url可以配置這個規(guī)則時，那么該請求就交由對應(yīng)的Handler去執(zhí)行。
注意：Handler泛指繼承自RequestHandler的所有類
? ? ? ? Handlers泛指繼承自RequestHandler的所有類的集合
上述代碼主要完成了以下功能：加載配置信息和生成url映射，并且把所有的信息封裝在一個application對象中。
加載的配置信息包括：
- 編碼和返回方式信息
- 靜態(tài)文件路徑
- ui_modules（模版語言中使用，暫時忽略）
- ui_methods（模版語言中使用，暫時忽略）
- 是否debug模式運(yùn)行
以上的所有配置信息，都可以在settings中配置，然后在創(chuàng)建Application對象時候，傳入?yún)?shù)即可。如：application = tornado.web.Application([(r"/index", MainHandler),],**settings)
生成url映射：
- 將url和對應(yīng)的Handler添加到對應(yīng)的主機(jī)前綴中，如：safe.index.com、www.auto.com
?封裝數(shù)據(jù)：
將配置信息和url映射關(guān)系封裝到Application對象中，信息分別保存在Application對象的以下字段中：
- self.transforms，保存著編碼和返回方式信息
- self.settings，保存著配置信息
- self.ui_modules，保存著ui_modules信息
- self.ui_methods，保存這ui_methods信息
- self.handlers，保存著所有的主機(jī)名對應(yīng)的Handlers，每個handlers則是url正則對應(yīng)的Handler
二、application.listen(xxx)
第一步操作將配置和url映射等信息封裝到了application對象中，而這第二步執(zhí)行application對象的listen方法，該方法內(nèi)部又把之前包含各種信息的application對象封裝到了一個HttpServer對象中，然后繼續(xù)調(diào)用HttpServer對象的liseten方法。
class Application(object):#創(chuàng)建服務(wù)端socket，并綁定IP和端口并添加相應(yīng)設(shè)置，注：未開始通過while監(jiān)聽accept，等待客戶端連接 def listen(self, port, address="", **kwargs):from tornado.httpserver import HTTPServerserver = HTTPServer(self, **kwargs)server.listen(port, address)詳細(xì)代碼：
class HTTPServer(object):def __init__(self, request_callback, no_keep_alive=False, io_loop=None,xheaders=False, ssl_options=None):#Application對象self.request_callback = request_callback#是否長連接self.no_keep_alive = no_keep_alive#IO循環(huán)self.io_loop = io_loopself.xheaders = xheaders#Http和Httpself.ssl_options = ssl_optionsself._socket = Noneself._started = Falsedef listen(self, port, address=""):self.bind(port, address)self.start(1)def bind(self, port, address=None, family=socket.AF_UNSPEC):assert not self._socket#創(chuàng)建服務(wù)端socket對象，IPV4和TCP連接self._socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM, 0)flags = fcntl.fcntl(self._socket.fileno(), fcntl.F_GETFD)flags |= fcntl.FD_CLOEXECfcntl.fcntl(self._socket.fileno(), fcntl.F_SETFD, flags)#配置socket對象self._socket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)self._socket.setblocking(0)#綁定IP和端口 self._socket.bind((address, port))#最大阻塞數(shù)量self._socket.listen(128)def start(self, num_processes=1):assert not self._startedself._started = Trueif num_processes is None or num_processes <= 0:num_processes = _cpu_count()if num_processes > 1 and ioloop.IOLoop.initialized():logging.error("Cannot run in multiple processes: IOLoop instance ""has already been initialized. You cannot call ""IOLoop.instance() before calling start()")num_processes = 1#如果進(jìn)程數(shù)大于1if num_processes > 1:logging.info("Pre-forking %d server processes", num_processes)for i in range(num_processes):if os.fork() == 0:import randomfrom binascii import hexlifytry:# If available, use the same method as# random.pyseed = long(hexlify(os.urandom(16)), 16)except NotImplementedError:# Include the pid to avoid initializing two# processes to the same valueseed(int(time.time() * 1000) ^ os.getpid())random.seed(seed)self.io_loop = ioloop.IOLoop.instance()self.io_loop.add_handler(self._socket.fileno(), self._handle_events,ioloop.IOLoop.READ)returnos.waitpid(-1, 0)#進(jìn)程數(shù)等于1，默認(rèn)else:if not self.io_loop:#設(shè)置成員變量self.io_loop為IOLoop的實(shí)例，注：IOLoop使用methodclass完成了一個單例模式self.io_loop = ioloop.IOLoop.instance()#執(zhí)行IOLoop的add_handler方法，將socket句柄、self._handle_events方法和IOLoop.READ當(dāng)參數(shù)傳入 self.io_loop.add_handler(self._socket.fileno(),self._handle_events,ioloop.IOLoop.READ)def _handle_events(self, fd, events):while True:try:#====important=====#connection, address = self._socket.accept()except socket.error, e:if e.args[0] in (errno.EWOULDBLOCK, errno.EAGAIN):returnraiseif self.ssl_options is not None:assert ssl, "Python 2.6+ and OpenSSL required for SSL"try:#====important=====#connection = ssl.wrap_socket(connection,server_side=True,do_handshake_on_connect=False,**self.ssl_options)except ssl.SSLError, err:if err.args[0] == ssl.SSL_ERROR_EOF:return connection.close()else:raiseexcept socket.error, err:if err.args[0] == errno.ECONNABORTED:return connection.close()else:raisetry:if self.ssl_options is not None:stream = iostream.SSLIOStream(connection, io_loop=self.io_loop)else:stream = iostream.IOStream(connection, io_loop=self.io_loop)#====important=====# HTTPConnection(stream, address, self.request_callback,self.no_keep_alive, self.xheaders) except:logging.error("Error in connection callback", exc_info=True) HTTPServer class IOLoop(object):# Constants from the epoll module_EPOLLIN = 0x001_EPOLLPRI = 0x002_EPOLLOUT = 0x004_EPOLLERR = 0x008_EPOLLHUP = 0x010_EPOLLRDHUP = 0x2000_EPOLLONESHOT = (1 << 30)_EPOLLET = (1 << 31)# Our events map exactly to the epoll eventsNONE = 0READ = _EPOLLINWRITE = _EPOLLOUTERROR = _EPOLLERR | _EPOLLHUP | _EPOLLRDHUPdef __init__(self, impl=None):self._impl = impl or _poll()if hasattr(self._impl, 'fileno'):self._set_close_exec(self._impl.fileno())self._handlers = {}self._events = {}self._callbacks = []self._timeouts = []self._running = Falseself._stopped = Falseself._blocking_signal_threshold = None# Create a pipe that we send bogus data to when we want to wake# the I/O loop when it is idleif os.name != 'nt':r, w = os.pipe()self._set_nonblocking(r)self._set_nonblocking(w)self._set_close_exec(r)self._set_close_exec(w)self._waker_reader = os.fdopen(r, "rb", 0)self._waker_writer = os.fdopen(w, "wb", 0)else:self._waker_reader = self._waker_writer = win32_support.Pipe()r = self._waker_writer.reader_fdself.add_handler(r, self._read_waker, self.READ)@classmethoddef instance(cls):if not hasattr(cls, "_instance"):cls._instance = cls()return cls._instancedef add_handler(self, fd, handler, events):"""Registers the given handler to receive the given events for fd."""self._handlers[fd] = stack_context.wrap(handler)self._impl.register(fd, events | self.ERROR) IOLoop def wrap(fn):'''Returns a callable object that will resore the current StackContextwhen executed.Use this whenever saving a callback to be executed later in adifferent execution context (either in a different thread orasynchronously in the same thread).'''if fn is None:return None# functools.wraps doesn't appear to work on functools.partial objects#@functools.wraps(fn)def wrapped(callback, contexts, *args, **kwargs):# If we're moving down the stack, _state.contexts is a prefix# of contexts. For each element of contexts not in that prefix,# create a new StackContext object.# If we're moving up the stack (or to an entirely different stack),# _state.contexts will have elements not in contexts. Use# NullContext to clear the state and then recreate from contexts.if (len(_state.contexts) > len(contexts) orany(a[1] is not b[1]for a, b in itertools.izip(_state.contexts, contexts))):# contexts have been removed or changed, so start overnew_contexts = ([NullContext()] +[cls(arg) for (cls,arg) in contexts])else:new_contexts = [cls(arg)for (cls, arg) in contexts[len(_state.contexts):]]if len(new_contexts) > 1:with contextlib.nested(*new_contexts):callback(*args, **kwargs)elif new_contexts:with new_contexts[0]:callback(*args, **kwargs)else:callback(*args, **kwargs)if getattr(fn, 'stack_context_wrapped', False):return fncontexts = _state.contextsresult = functools.partial(wrapped, fn, contexts)result.stack_context_wrapped = Truereturn result stack_context.wrap備注：stack_context.wrap其實(shí)就是對函數(shù)進(jìn)行一下封裝，即：函數(shù)在不同情況下上下文信息可能不同。
上述代碼本質(zhì)上就干了以下這么四件事：
目前，我們只是看到上述代碼大致干了這四件事，而其目的有什么？他們之間的聯(lián)系又是什么呢？
答：現(xiàn)在不妨先來做一個猜想，待之后再在源碼中確認(rèn)驗(yàn)證是否正確！猜想：通過epoll監(jiān)聽服務(wù)端socket事件，一旦請求到達(dá)時，則執(zhí)行3中被封裝了的_handle_events函數(shù)，該函數(shù)又利用application中封裝了的各種配置信息對客戶端url來指定判定，然后指定對應(yīng)的Handler處理該請求。
注意：使用epoll創(chuàng)建服務(wù)端socket
?上述，其實(shí)就是利用epoll對象的poll(timeout)方法去輪詢已經(jīng)注冊在epoll中的socket句柄，當(dāng)有讀可用的信息時候，則返回包含當(dāng)前句柄和Event Code的序列，然后在通過句柄對客戶端的請求進(jìn)行處理
三、tornado.ioloop.IOLoop.instance().start()
上一步中創(chuàng)建了socket對象并使得socket對象和epoll建立了關(guān)系，該步驟則就來執(zhí)行epoll的epoll方法去輪詢已經(jīng)注冊在epoll對象中的socket句柄，當(dāng)有讀可用信息時，則觸發(fā)一些操作什么的....
class IOLoop(object):def add_handler(self, fd, handler, events):#HttpServer的Start方法中會調(diào)用該方法self._handlers[fd] = stack_context.wrap(handler)self._impl.register(fd, events | self.ERROR)def start(self):while True:poll_timeout = 0.2try:#epoll中輪詢event_pairs = self._impl.poll(poll_timeout)except Exception, e:#省略其他#如果有讀可用信息，則把該socket對象句柄和Event Code序列添加到self._events中self._events.update(event_pairs)#遍歷self._events，處理每個請求while self._events:fd, events = self._events.popitem()try:#以socket為句柄為key，取出self._handlers中的stack_context.wrap(handler)，并執(zhí)行#stack_context.wrap(handler)包裝了HTTPServer類的_handle_events函數(shù)的一個函數(shù)#是在上一步中執(zhí)行add_handler方法時候，添加到self._handlers中的數(shù)據(jù)。self._handlers[fd](fd, events)except:#省略其他 class IOLoop(object):def start(self):"""Starts the I/O loop.The loop will run until one of the I/O handlers calls stop(), whichwill make the loop stop after the current event iteration completes."""if self._stopped:self._stopped = Falsereturnself._running = Truewhile True:# Never use an infinite timeout here - it can stall epollpoll_timeout = 0.2# Prevent IO event starvation by delaying new callbacks# to the next iteration of the event loop.callbacks = self._callbacksself._callbacks = []for callback in callbacks:self._run_callback(callback)if self._callbacks:poll_timeout = 0.0if self._timeouts:now = time.time()while self._timeouts and self._timeouts[0].deadline <= now:timeout = self._timeouts.pop(0)self._run_callback(timeout.callback)if self._timeouts:milliseconds = self._timeouts[0].deadline - nowpoll_timeout = min(milliseconds, poll_timeout)if not self._running:breakif self._blocking_signal_threshold is not None:# clear alarm so it doesn't fire while poll is waiting for# events. signal.setitimer(signal.ITIMER_REAL, 0, 0)try:event_pairs = self._impl.poll(poll_timeout)except Exception, e:# Depending on python version and IOLoop implementation,# different exception types may be thrown and there are# two ways EINTR might be signaled:# * e.errno == errno.EINTR# * e.args is like (errno.EINTR, 'Interrupted system call')if (getattr(e, 'errno', None) == errno.EINTR or(isinstance(getattr(e, 'args', None), tuple) andlen(e.args) == 2 and e.args[0] == errno.EINTR)):continueelse:raiseif self._blocking_signal_threshold is not None:signal.setitimer(signal.ITIMER_REAL,self._blocking_signal_threshold, 0)# Pop one fd at a time from the set of pending fds and run# its handler. Since that handler may perform actions on# other file descriptors, there may be reentrant calls to# this IOLoop that update self._events self._events.update(event_pairs)while self._events:fd, events = self._events.popitem()try:self._handlers[fd](fd, events)except (KeyboardInterrupt, SystemExit):raiseexcept (OSError, IOError), e:if e.args[0] == errno.EPIPE:# Happens when the client closes the connectionpasselse:logging.error("Exception in I/O handler for fd %d",fd, exc_info=True)except:logging.error("Exception in I/O handler for fd %d",fd, exc_info=True)# reset the stopped flag so another start/stop pair can be issuedself._stopped = Falseif self._blocking_signal_threshold is not None:signal.setitimer(signal.ITIMER_REAL, 0, 0) View Code對于上述代碼，執(zhí)行start方法后，程序就進(jìn)入“死循環(huán)”，也就是會一直不停的輪詢的去檢查是否有請求到來，如果有請求到達(dá)，則執(zhí)行封裝了HttpServer類的_handle_events方法和相關(guān)上下文的stack_context.wrap(handler)（其實(shí)就是執(zhí)行HttpServer類的_handle_events方法），詳細(xì)見下篇博文，簡要代碼如下：
結(jié)束
本篇博文介紹了“待請求階段”的所作所為，簡要來說其實(shí)就是三件事：其一、把setting中的各種配置以及url和Handler之間的映射關(guān)系封裝到來application對象中（application對象又被封裝到了HttpServer對象的request_callback字段中）；其二、結(jié)合epoll創(chuàng)建服務(wù)端socket；其三、當(dāng)請求到達(dá)時交由HttpServer類的_handle_events方法處理請求，即：處理請求的入口。對于處理請求的詳細(xì)，請參見下篇博客（客官莫急，加班編寫中...）
?
轉(zhuǎn)載于:https://www.cnblogs.com/wupeiqi/p/4375610.html
總結(jié)
以上是生活随笔為你收集整理的第二篇：白话tornado源码之待请求阶段的全部內(nèi)容，希望文章能夠幫你解決所遇到的問題。
- 上一篇： ubuntu sublime字体设置
- 下一篇： 求最小子数组之二维篇