文章目錄

• 概述
• TF/IDF
• 鏈接
• 示例
• • DSL
  • 普通查詢
  • dis_max 查詢
• best fields策略-dis_max

概述

繼續跟中華石杉老師學習ES，第十篇

課程地址： https://www.roncoo.com/view/55

---

TF/IDF

Apache Lucene默認評分機制

• TF (Term Frequency): 基于詞項(term vector), 用來表示一個詞項在某個文檔中出現了多少次。
  詞頻越高，文檔得分越高

• IDF (Inveres Dcoument Frequency): 基于詞項（term vector）,用來告訴評分公式該詞有多美的漢奸。
  逆文檔頻率越高，詞項就越罕見。 評分公式利用該因子為包含罕見詞項的文檔加權。

term vector : 詞項向量是一種針對每個文檔的微型倒排索引。詞項向量的每個維由詞項和出現頻率結對組成,還可以包含詞項的位置信息。 Lucene 和 ES都默認禁用詞項向量索引，如果實現某些功能比如高亮顯示等需要開啟該選項 。

---

鏈接

官方指導： https://www.elastic.co/guide/en/elasticsearch/guide/current/_tuning_best_fields_queries.html

https://www.elastic.co/guide/en/elasticsearch/reference/7.2/query-dsl-dis-max-query.html

---

數據量少的時候，dis_max不生效的問題： https://stackoverflow.com/questions/38065692/dis-max-query-isnt-looking-for-the-best-matching-clause

---

其他博主寫的相關文章：
https://blog.csdn.net/dm_vincent/article/details/41820537

---

示例

ES版本 6.4.1

為了演示效果，我們把之前的forum索引刪除了重建一下，

DSL如下

DSL

DELETE /forumPUT /forum { "settings" : { "number_of_shards" : 1 }}POST /forum/article/_bulk { "index": { "_id": 1 }} { "articleID" : "XHDK-A-1293-#fJ3", "userID" : 1, "hidden": false, "postDate": "2017-01-01" } { "index": { "_id": 2 }} { "articleID" : "KDKE-B-9947-#kL5", "userID" : 1, "hidden": false, "postDate": "2017-01-02" } { "index": { "_id": 3 }} { "articleID" : "JODL-X-1937-#pV7", "userID" : 2, "hidden": false, "postDate": "2017-01-01" } { "index": { "_id": 4 }} { "articleID" : "QQPX-R-3956-#aD8", "userID" : 2, "hidden": true, "postDate": "2017-01-02" }POST /forum/article/_bulk {"update":{"_id":"1"}} {"doc":{"tag":["java","hadoop"]}} {"update":{"_id":"2"}} {"doc":{"tag":["java"]}} {"update":{"_id":"3"}} {"doc":{"tag":["hadoop"]}} {"update":{"_id":"4"}} {"doc":{"tag":["java","elasticsearch"]}}POST /forum/article/_bulk {"update":{"_id":"1"}} {"doc":{"tag_cnt":2}} {"update":{"_id":"2"}} {"doc":{"tag_cnt":1}} {"update":{"_id":"3"}} {"doc":{"tag_cnt":1}} {"update":{"_id":"4"}} {"doc":{"tag_cnt":2}}POST /forum/article/_bulk {"update":{"_id":"1"}} {"doc":{"view_cnt":30}} {"update":{"_id":"2"}} {"doc":{"view_cnt":50}} {"update":{"_id":"3"}} {"doc":{"view_cnt":100}} {"update":{"_id":"4"}} {"doc":{"view_cnt":80}}POST /forum/article/_bulk {"index":{"_id":5}} {"articleID":"DHJK-B-1395-#Ky5","userID":3,"hidden":false,"postDate":"2019-06-01","tag":["elasticsearch"],"tag_cnt":1,"view_cnt":10}POST /forum/article/_bulk {"update":{"_id":"5"}} {"doc":{"postDate":"2019-05-01"}}POST /forum/article/_bulk {"update":{"_id":"1"}} {"doc":{"title":"this is java and elasticsearch blog"}} {"update":{"_id":"2"}} {"doc":{"title":"this is java blog"}} {"update":{"_id":"3"}} {"doc":{"title":"this is elasticsearch blog"}} {"update":{"_id":"4"}} {"doc":{"title":"this is java, elasticsearch, hadoop blog"}} {"update":{"_id":"5"}} {"doc":{"title":"this is spark blog"}}POST /forum/article/_bulk {"update":{"_id":"1"}} {"doc":{"content":"i like to write best elasticsearch article"}} {"update":{"_id":"2"}} {"doc":{"content":"i think java is the best programming language"}} {"update":{"_id":"3"}} {"doc":{"content":"i am only an elasticsearch beginner"}} {"update":{"_id":"4"}} {"doc":{"content":"elasticsearch and hadoop are all very good solution, i am a beginner"}} {"update":{"_id":"5"}} {"doc":{"content":"spark is best big data solution based on scala ,an programming language similar to java"}}

至此，數據構造完成 ，下面來看下dis_max是如何作用的吧

GET /forum/article/_search 數據如下： {"took": 0,"timed_out": false,"_shards": {"total": 1,"successful": 1,"skipped": 0,"failed": 0},"hits": {"total": 5,"max_score": 1,"hits": [{"_index": "forum","_type": "article","_id": "1","_score": 1,"_source": {"articleID": "XHDK-A-1293-#fJ3","userID": 1,"hidden": false,"postDate": "2017-01-01","tag": ["java","hadoop"],"tag_cnt": 2,"view_cnt": 30,"title": "this is java and elasticsearch blog","content": "i like to write best elasticsearch article"}},{"_index": "forum","_type": "article","_id": "2","_score": 1,"_source": {"articleID": "KDKE-B-9947-#kL5","userID": 1,"hidden": false,"postDate": "2017-01-02","tag": ["java"],"tag_cnt": 1,"view_cnt": 50,"title": "this is java blog","content": "i think java is the best programming language"}},{"_index": "forum","_type": "article","_id": "3","_score": 1,"_source": {"articleID": "JODL-X-1937-#pV7","userID": 2,"hidden": false,"postDate": "2017-01-01","tag": ["hadoop"],"tag_cnt": 1,"view_cnt": 100,"title": "this is elasticsearch blog","content": "i am only an elasticsearch beginner"}},{"_index": "forum","_type": "article","_id": "4","_score": 1,"_source": {"articleID": "QQPX-R-3956-#aD8","userID": 2,"hidden": true,"postDate": "2017-01-02","tag": ["java","elasticsearch"],"tag_cnt": 2,"view_cnt": 80,"title": "this is java, elasticsearch, hadoop blog","content": "elasticsearch and hadoop are all very good solution, i am a beginner"}},{"_index": "forum","_type": "article","_id": "5","_score": 1,"_source": {"articleID": "DHJK-B-1395-#Ky5","userID": 3,"hidden": false,"postDate": "2019-05-01","tag": ["elasticsearch"],"tag_cnt": 1,"view_cnt": 10,"title": "this is spark blog","content": "spark is best big data solution based on scala ,an programming language similar to java"}}]} }

---

普通查詢

先看下普通的DSL

GET /forum/article/_search {"query": {"bool": {"should": [{"match": {"title": "java solution"}},{"match": {"content": "java solution"}}],"minimum_should_match": 1}} }

返回：

{"took": 1,"timed_out": false,"_shards": {"total": 1,"successful": 1,"skipped": 0,"failed": 0},"hits": {"total": 4,"max_score": 1.5179626,"hits": [{"_index": "forum","_type": "article","_id": "2","_score": 1.5179626,"_source": {"articleID": "KDKE-B-9947-#kL5","userID": 1,"hidden": false,"postDate": "2017-01-02","tag": ["java"],"tag_cnt": 1,"view_cnt": 50,"title": "this is java blog","content": "i think java is the best programming language"}},{"_index": "forum","_type": "article","_id": "5","_score": 1.4233948,"_source": {"articleID": "DHJK-B-1395-#Ky5","userID": 3,"hidden": false,"postDate": "2019-05-01","tag": ["elasticsearch"],"tag_cnt": 1,"view_cnt": 10,"title": "this is spark blog","content": "spark is best big data solution based on scala ,an programming language similar to java"}},{"_index": "forum","_type": "article","_id": "4","_score": 1.2832261,"_source": {"articleID": "QQPX-R-3956-#aD8","userID": 2,"hidden": true,"postDate": "2017-01-02","tag": ["java","elasticsearch"],"tag_cnt": 2,"view_cnt": 80,"title": "this is java, elasticsearch, hadoop blog","content": "elasticsearch and hadoop are all very good solution, i am a beginner"}},{"_index": "forum","_type": "article","_id": "1","_score": 0.4889865,"_source": {"articleID": "XHDK-A-1293-#fJ3","userID": 1,"hidden": false,"postDate": "2017-01-01","tag": ["java","hadoop"],"tag_cnt": 2,"view_cnt": 30,"title": "this is java and elasticsearch blog","content": "i like to write best elasticsearch article"}}]} }

來分析一下結果

計算每個document的relevance score：每個query的分數，乘以matched query數量，除以總query數量

算一下doc2的分數

{ "match": { "title": "java solution" }}，針對doc2，是有一個分數的
{ "match": { "content": "java solution" }}，針對doc2，也是有一個分數的

假設分數如下 ， 所以是兩個分數加起來，比如說，1.1 + 1.2 = 2.3
matched query數量 = 2
總query數量 = 2

2.3 * 2 / 2 = 2.3

---

算一下doc5的分數

{ "match": { "title": "java solution" }}，針對doc5，是沒有分數的
{ "match": { "content": "java solution" }}，針對doc5，是有一個分數的

所以說，只有一個query是有分數的，比如2.3
matched query數量 = 1
總query數量 = 2

2.3 * 1 / 2 = 1.15

doc5的分數 = 1.15 < doc2的分數 = 2.3

---

id=2的數據排在了前面，其實我們希望id=5的排在前面，畢竟id=5的數據 content字段既有java又有solution. 那看下dis_max吧

---

dis_max 查詢

GET /forum/article/_search {"query": {"dis_max": {"queries": [{"match": {"title": "java solution"}},{"match": {"content": "java solution"}}]}} }

返回

{"took": 0,"timed_out": false,"_shards": {"total": 1,"successful": 1,"skipped": 0,"failed": 0},"hits": {"total": 4,"max_score": 1.4233948,"hits": [{"_index": "forum","_type": "article","_id": "5","_score": 1.4233948,"_source": {"articleID": "DHJK-B-1395-#Ky5","userID": 3,"hidden": false,"postDate": "2019-05-01","tag": ["elasticsearch"],"tag_cnt": 1,"view_cnt": 10,"title": "this is spark blog","content": "spark is best big data solution based on scala ,an programming language similar to java"}},{"_index": "forum","_type": "article","_id": "2","_score": 0.93952733,"_source": {"articleID": "KDKE-B-9947-#kL5","userID": 1,"hidden": false,"postDate": "2017-01-02","tag": ["java"],"tag_cnt": 1,"view_cnt": 50,"title": "this is java blog","content": "i think java is the best programming language"}},{"_index": "forum","_type": "article","_id": "4","_score": 0.79423964,"_source": {"articleID": "QQPX-R-3956-#aD8","userID": 2,"hidden": true,"postDate": "2017-01-02","tag": ["java","elasticsearch"],"tag_cnt": 2,"view_cnt": 80,"title": "this is java, elasticsearch, hadoop blog","content": "elasticsearch and hadoop are all very good solution, i am a beginner"}},{"_index": "forum","_type": "article","_id": "1","_score": 0.4889865,"_source": {"articleID": "XHDK-A-1293-#fJ3","userID": 1,"hidden": false,"postDate": "2017-01-01","tag": ["java","hadoop"],"tag_cnt": 2,"view_cnt": 30,"title": "this is java and elasticsearch blog","content": "i like to write best elasticsearch article"}}]} }

---

best fields策略-dis_max

best fields策略 : 搜索到的結果，應該是某一個field中匹配到了盡可能多的關鍵詞，被排在前面；而不是盡可能多的field匹配到了少數的關鍵詞，排在了前面.

dis_max語法，直接取多個query中，分數最高的那一個query的分數即可

舉個例子

{ "match": { "title": "java solution" }}，針對doc2，是有一個分數的，1.1
{ "match": { "content": "java solution" }}，針對doc2，也是有一個分數的，1.2

取最大分數，1.2

---

{ "match": { "title": "java solution" }}，針對doc5，是沒有分數的
{ "match": { "content": "java solution" }}，針對doc5，是有一個分數的，2.3

取最大分數，2.3

然后doc2的分數 = 1.2 < doc5的分數 = 2.3，所以doc5就可以排在更前面的地方.

總結

以上是生活随笔為你收集整理的白话Elasticsearch10-深度探秘搜索技术之基于dis_max实现best fields策略进行多字段搜索的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。