當前位置：首頁 > 编程资源 > 编程问答 >内容正文

编程问答

利用jsoup解析网站网页

發布時間：2024/1/8 编程问答 30 豆豆

生活随笔收集整理的這篇文章主要介紹了利用jsoup解析网站网页小編覺得挺不錯的,現在分享給大家,幫大家做個參考.

用到的工具為 jsoup-1.7.2.jar包，具體jsoup的相關文檔，請去這邊看http://jsoup.org/，這里有全部Api可以查詢。

首先請求網頁，

Document doc = Jsoup.connect(search).timeout(5000).get();

獲取html：

<!DOCTYPE html>  <html><head> <meta charset="utf-8" /> <meta content="width=device-width, initial-scale=1.0, maximum-scale=1.0, user-scalable=no;" name="viewport" /> <meta name="apple-mobile-web-app-capable" content="yes" /> <meta name="apple-mobile-web-app-status-bar-style" content="black" /> <title>口袋_站內搜索</title> <link href="/static/css/mobileDefault.css?v=0.9" rel="stylesheet" type="text/css" /> <link href="/static/css/mobileGameresult.css?v=0.9" rel="stylesheet" type="text/css" /> <script type="text/javascript" src="/static/js/jquery-1.7.2.min.js"></script> <script type="text/javascript" src="/static/js/baidu-ajax.js"></script> <script type="text/javascript" src="/static/js/statistic.js"></script> </head> <body> <div id="wrap" style="position: relative;"> <div id="head"> <div id="head_d"> <form name="f" action="http://zhannei.baidu.com/cse/search" class="fm" id="bdcs-search-form"> <span class="s_back_wr"> <div onclick="backRefer();" class="head-a"> <img src="/static/img/back.png" width="12px" height="20px" border="0" class="s_back" id="s_back_i" /> </div> </span> <span class="s_ipt_wr" id="search_input_container" style="background-color:#f8f8f8;"> <b class="icon" id="icon_remove" onclick="removeText();"></b> <input name="q" id="kw" class="s_ipt" style="background-color:#f8f8f8;" value="口袋" maxlength="100" autocomplete="off" /> <input name="click" value="1" type="hidden" /> </span> <div onclick="submitMobile();" class="head-a"> <span class="s_btn_wr"> <input name="s" value="287293036948159515" type="hidden" /> <input name="nsid" value="" type="hidden" /> <img src="/static/img/fdj2.png" width="20px" height="20px" border="0" id="su" class="s_btn" /> </span> </div> </form> <ul id="bdcs-search-sug-list" class="bdcs-search-sug-list"></ul> </div> </div> <style type="text/css">.bdcs-search-sug-list {display:none;height:auto;border: 1px solid #DDD;position: absolute;z-index: 2147483647;background-color: #FFF;}.bdcs-search-hot-list {height:40%;border: 1px solid #DDD;position: absolute;z-index: 2147483647;background-color: #FFF;}.hotRadious{border:2px solid;border-radius:25px;-moz-border-radius:25px;}.bdcs-search-sug-list-item {display: block;list-style: none;cursor: pointer;padding-top:9px;padding-left:8px;vertical-align:middle;white-space: nowrap;text-overflow: ellipsis;overflow: hidden;height: 46px;line-height: 36px;font-family: arial;font-size: 18px;}.fdj{ display:block;vertical-align:middle;float:left;margin-top:4px;margin-right:8px;}.bdcs-search-sug-list-item-current {background-color: #EBEBEB;}</style> <script type="text/javascript">if ($("#kw").attr("value") == "") {$("#icon_remove").hide();}function removeText() {$("#kw").attr("value", "");$("#icon_remove").hide();}function submitMobile() {var frm = document.getElementById('bdcs-search-form');frm.submit();}function backRefer() {var refer = document.referrer;if (refer && refer.indexOf('cse/search') == -1) {window.location.href = refer;} else {history.go(-1);}}</script> <script>$("#kw").bind("input", function() {if ($("#kw").attr("value") == "" || $("#kw").attr("value").match(/^\s+$/)) {$("#icon_remove").hide();} else {$("#icon_remove").show();}});</script>  <div id="container" class="clearfix "> <div id="center" class="content"> <div id="results" class="content-main"> <div class="result-list" id="result-list-game"> <div class="result-item result-game-item"> <div onclick="window.location='http://www.biquge.com/32_32224/'" class="game-legend-a" ontouchstart="this.className = 'result_content_div'" ontouchend="this.className = 'game-legend-a result_content_end';" ontouchmove="this.className = 'game-legend-a result_content_end';"> <div class="result-game-item-pic" style="width:80px;height:110px;"> <img src="http://www.biquge.com/files/article/image/33/32224/32224.jpg" alt="夢想<em>口袋</em>" onerror="$(this).attr('src', '/static/img/novel-noimg.jpg')" style="border: 1px solid rgb(229,229,229);width:80px;height:110px;" class="result-game-item-pic-link-img" /> </div> <div class="result-game-item-detail"> <h3 class="result-item-title result-game-item-title"> 夢想<em>口袋</em> </h3> <p class="result-game-item-desc">第一百六十二章大結局（下）輪回！時光飛逝，又過了三十三年。Ｗ wＷ、Ｑb ⑸.C0M \ 天…</p> <div class="result-game-item-info"> <p class="result-game-item-info-tag"> <span class="result-game-item-info-tag-title">作者：</span> <span> 天天不休 </span> </p> <p class="result-game-item-info-tag"> <span class="result-game-item-info-tag-title">類型：</span> <span class="result-game-item-info-tag-title">都市言情</span> </p> <p class="result-game-item-info-tag"> <span class="result-game-item-info-tag-title">更新時間：</span> <span class="result-game-item-info-tag-title">2016-08-18</span> </p> <p class="result-game-item-info-tag"> <span class="result-game-item-info-tag-title">最新章節：</span> <span class="result-game-item-uspan" onclick="clickInner('http://www.biquge.com/32_32224/1796032.html');"> 第六十五章豪華,該死,龍過江 </span> </p> </div> </div>  <div style="clear: both;"></div>  </div> </div> <div class="result-item result-game-item"> <div onclick="window.location='http://www.biquge.com/20_20518/'" class="game-legend-a" ontouchstart="this.className = 'result_content_div'" ontouchend="this.className = 'game-legend-a result_content_end';" ontouchmove="this.className = 'game-legend-a result_content_end';"> <div class="result-game-item-pic" style="width:80px;height:110px;"> <img src="http://www.biquge.com/files/article/image/21/20518/20518.jpg" alt="超次元<em>口袋</em>" onerror="$(this).attr('src', '/static/img/novel-noimg.jpg')" style="border: 1px solid rgb(229,229,229);width:80px;height:110px;" class="result-game-item-pic-link-img" /> </div> <div class="result-game-item-detail"> <h3 class="result-item-title result-game-item-title"> 超次元<em>口袋</em> </h3> <p class="result-game-item-desc">平安夜，本想告別童貞，卻沒想到慘遭失戀。莫小天對著流星許下若干愿望，卻沒想到獲得一個超次元<em>口袋</em>，可以打開任何次元，并且從那里摸出一件東西。炮姐的安全褲！吾王的咖喱棒！藍胖子</p> <div class="result-game-item-info"> <p class="result-game-item-info-tag"> <span class="result-game-item-info-tag-title">作者：</span> <span> 秋子鳴 </span> </p> <p class="result-game-item-info-tag"> <span class="result-game-item-info-tag-title">類型：</span> <span class="result-game-item-info-tag-title">都市言情</span> </p> <p class="result-game-item-info-tag"> <span class="result-game-item-info-tag-title">更新時間：</span> <span class="result-game-item-info-tag-title">2016-09-04</span> </p> <p class="result-game-item-info-tag"> <span class="result-game-item-info-tag-title">最新章節：</span> <span class="result-game-item-uspan" onclick="clickInner('http://www.biquge.com/20_20518/1624811.html');"> 第一百八十章監控學校 </span> </p> </div> </div>  <div style="clear: both;"></div>  </div> </div> <div class="result-item result-game-item"> <div onclick="window.location='http://www.biquge.com/26_26070/'" class="game-legend-a" ontouchstart="this.className = 'result_content_div'" ontouchend="this.className = 'game-legend-a result_content_end';" ontouchmove="this.className = 'game-legend-a result_content_end';"> <div class="result-game-item-pic" style="width:80px;height:110px;"> <img src="http://www.biquge.com/files/article/image/27/26070/26070.jpg" alt...

再獲取查詢的小說列表，通過result-list篩選，獲取result-item：

es = doc.getElementsByClass("result-list");element = es.get(0);results = element.getElementsByClass("result-item");

item的HTML：

<div class="result-item result-game-item"> <div οnclick="window.location='http://www.biquge.com/32_32224/'" class="game-legend-a" ontouchstart="this.className = 'result_content_div'" ontouchend="this.className = 'game-legend-a result_content_end';" ontouchmove="this.className = 'game-legend-a result_content_end';"> <div class="result-game-item-pic" style="width:80px;height:110px;"> <img src="http://www.biquge.com/files/article/image/33/32224/32224.jpg" alt="夢想<em>口袋</em>" οnerrοr="$(this).attr('src', '/static/img/novel-noimg.jpg')" style="border: 1px solid rgb(229,229,229);width:80px;height:110px;" class="result-game-item-pic-link-img" /> </div> <div class="result-game-item-detail"> <h3 class="result-item-title result-game-item-title"> 夢想<em>口袋</em> </h3> <p class="result-game-item-desc">第一百六十二章大結局（下）輪回！時光飛逝，又過了三十三年。Ｗ wＷ、Ｑb ⑸.C0M \ 天…</p> <div class="result-game-item-info"> <p class="result-game-item-info-tag"> <span class="result-game-item-info-tag-title">作者：</span> <span> 天天不休 </span> </p> <p class="result-game-item-info-tag"> <span class="result-game-item-info-tag-title">類型：</span> <span class="result-game-item-info-tag-title">都市言情</span> </p> <p class="result-game-item-info-tag"> <span class="result-game-item-info-tag-title">更新時間：</span> <span class="result-game-item-info-tag-title">2016-08-18</span> </p> <p class="result-game-item-info-tag"> <span class="result-game-item-info-tag-title">最新章節：</span> <span class="result-game-item-uspan" οnclick="clickInner('http://www.biquge.com/32_32224/1796032.html');"> 第六十五章豪華,該死,龍過江 </span> </p> </div> </div>  <div style="clear: both;"></div>  </div> </div>

解析item：

for (Element result : results) {Book book = new Book();try {// 小說地址Element titleElement = result.getElementsByClass("game-legend-a").get(0);String onclick = titleElement.attr("onclick").trim();book.url = onclick.split("'")[1];} catch (Exception e) {e.printStackTrace();}try {// 圖片地址Element imgElement = result.getElementsByClass("result-game-item-pic").get(0);book.src = imgElement.select("img").attr("src").trim();} catch (Exception e) {e.printStackTrace();}try {// 小說標題Element titleElement = result.getElementsByClass("result-item-title").get(0);book.name = titleElement.text().replace("<em>", "").replace("</em>", "").trim();} catch (Exception e) {e.printStackTrace();}try {// 簡介Element descElement = result.getElementsByClass("result-game-item-desc").get(0);book.description = descElement.text().replace("<em>", "").replace("</em>", "").trim();} catch (Exception e) {e.printStackTrace();}try {// 作者Element autorElement = result.getElementsByClass("result-game-item-info-tag").get(0);book.autor = autorElement.select("span").get(1).text().trim();} catch (Exception e) {e.printStackTrace();}try {// 類型Element typeElement = result.getElementsByClass("result-game-item-info-tag").get(1);book.type = typeElement.select("span").get(1).text().trim();} catch (Exception e) {e.printStackTrace();}try {// 更新時間Element timeElement = result.getElementsByClass("result-game-item-info-tag").get(2);book.lastUpdateTime = timeElement.select("span").get(1).text().trim();} catch (Exception e) {e.printStackTrace();}try {// 最后章節Element chapElement = result.getElementsByClass("result-game-item-info-tag").get(3);book.lastUpdateChapter = chapElement.select("span").get(1).text().trim();} catch (Exception e) {e.printStackTrace();}list.add(book);}

這樣就解析完成了。

轉載于:https://www.cnblogs.com/chenlong-50954265/p/5846630.html

總結

以上是生活随笔為你收集整理的利用jsoup解析网站网页的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。

上一篇： Android基础知识（二十）：Noti
下一篇：调和数