這里要注意，獲取貼吧ID有防爬蟲，他會把數(shù)據(jù)用<!-- -->這個注釋掉。

運行截圖如下：

注意要偽造成瀏覽器，不然獲取不到數(shù)據(jù)

代碼如下：

import requests import queue import time from bs4 import BeautifulSoupheader = {'Host' : 'tieba.baidu.com','Accept' : 'text/html,application/xhtml+xml,application/xml;q=0.9,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.9','Accept-Language' : 'zh-CN,zh;q=0.9','Cache-Control' : 'no-cache','Connection' : 'keep-alive','Cookie' : 'xxxxxxxxxxxxxxxx','Pragma' : 'no-cache','User-Agent' : 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/91.0.4472.114 Safari/537.36','sec-ch-ua' : '" Not;A Brand";v="99", "Google Chrome";v="91", "Chromium";v="91"' }baseUrl = "https://tieba.baidu.com/f?kw=%E5%B8%9D&ie=utf-8" urlQueue = queue.Queue(10000)def getRequest(url):response = requests.get(url, headers = header)return response.textpassif __name__ == '__main__':urlQueue.put(baseUrl)for i in range(100):url = urlQueue.get()content = getRequest(baseUrl + url)content = content.replace('-->', '').replace('<!--', '')contentSoup = BeautifulSoup(content, "html.parser")urlAllList = contentSoup.select("a")for urlTmp in urlAllList:if urlTmp.attrs.__contains__('href'):urlString = urlTmp['href']if '/p/' in urlString:testUrl = urlString.split('/p/')[1]urlQueue.put(testUrl)passpasspassprint('over')pass

?

總結(jié)

以上是生活随笔為你收集整理的Python笔记-获取某贴吧页面所有的贴吧id的全部內(nèi)容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網(wǎng)站內(nèi)容還不錯，歡迎將生活随笔推薦給好友。