首先分析糗百頁(yè)面：

目標(biāo)是獲取文字內(nèi)容和文章鏈接ID，并保存到文件。

#!/usr/bin/env/python #coding:utf-8if __name__ == '__main__':from urllib import request,parseimport re#抓取,獲取源代碼url='https://www.qiushibaike.com/textnew/'headers={"User-Agent":"Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/60.0.3112.90 Safari/537.36"}req=request.Request(url,headers=headers)resp=request.urlopen(req)html=resp.read().decode('utf-8')if resp.getcode()!=200:print('url open failed!')#解析數(shù)據(jù)pattern_id=re.compile(r'<a href="/article/(\d+)".*class=.contentHerf. >')pattern_content=re.compile(r'<div class="content">.*?<span>(.*?)</span>',re.S)ids=re.findall(pattern_id,html)contents=re.findall(pattern_content,html)data={}for i in range(len(ids)):pattern=r'<br/>'content=re.sub(pattern,'\n', contents[i])data[ids[i]]=content#保存數(shù)據(jù)directory='spider_qiubai/'for key in data.keys():f=open(directory+key+'.txt','w')try:f.write(data[key])except Exception as e:print("write file "+key+" failed! "+e)else:print("write file "+key+" successfully!")finally:f.close()

　　運(yùn)行結(jié)果：

轉(zhuǎn)載于:https://www.cnblogs.com/jasonhaven/p/7375503.html

總結(jié)

以上是生活随笔為你收集整理的小爬糗事百科的全部?jī)?nèi)容，希望文章能夠幫你解決所遇到的問(wèn)題。

如果覺(jué)得生活随笔網(wǎng)站內(nèi)容還不錯(cuò)，歡迎將生活随笔推薦給好友。