當前位置：首頁 > 编程资源 > 编程问答 >内容正文

编程问答

ES笔记

發(fā)布時間：2023/12/18 编程问答 35 豆豆

生活随笔收集整理的這篇文章主要介紹了 ES笔记小編覺得挺不錯的,現(xiàn)在分享給大家,幫大家做個參考.

1.1ES簡介

ES是使用java 語言并且基于lucence編寫的搜索引擎框架，他提供了分布式的全文搜索功能，提供了一個統(tǒng)一的基于restful風(fēng)格的web 接口。

lucence:一個搜索引擎底層

分布式：突出ES的橫向擴展能力

全文檢索：將一段詞語進行分詞，并將分出的詞語統(tǒng)一的放在一個分詞庫中，再搜索時，根據(jù)關(guān)鍵字取分詞庫中檢索，找到匹配的內(nèi)容（倒排索引）。

restful風(fēng)格的web 接口：只要發(fā)送一個http請求，并且根據(jù)請求方式的不同，攜帶參數(shù)的不同，執(zhí)行相應(yīng)的功能。

應(yīng)用廣泛：WIKI, github,Gold man

1.2ES的由來

回憶時光**許多年前，一個剛結(jié)婚的名叫 Shay Banon 的失業(yè)開發(fā)者，跟著他的妻子去了倫敦，他的妻子在那里學(xué)習(xí)廚師。在尋找一個賺錢的工作的時候，為了給他的妻子做一個食譜搜索引擎，他開始使用 Lucene 的一個早期版本。直接使用 Lucene 是很難的，因此 Shay 開始做一個抽象層，Java 開發(fā)者使用它可以很簡單的給他們的程序添加搜索功能。他發(fā)布了他的第一個開源項目 Compass。后來 Shay 獲得了一份工作，主要是高性能，分布式環(huán)境下的內(nèi)存數(shù)據(jù)網(wǎng)格。這個對于高性能，實時，分布式搜索引擎的需求尤為突出，他決定重寫 Compass，把它變?yōu)橐粋€獨立的服務(wù)并取名 Elasticsearch。第一個公開版本在2010年2月發(fā)布，從此以后，Elasticsearch 已經(jīng)成為了 Github 上最活躍的項目之一，他擁有超過300名 contributors(目前736名 contributors )。一家公司已經(jīng)開始圍繞 Elasticsearch 提供商業(yè)服務(wù)，并開發(fā)新的特性，但是，Elasticsearch 將永遠開源并對所有人可用。據(jù)說，Shay 的妻子還在等著她的食譜搜索引擎…

1.3ES和solr

1.solr 查詢死數(shù)據(jù)，速度比es快。但是數(shù)據(jù)如果是改變的，solr查詢速度會降低很多，ES的查詢速度沒有明顯的改變

2.solr搭建集群依賴ZK，ES本身就支持集群搭建

3.最開始solr 的社區(qū)很火爆，針對國內(nèi)文檔少，ES出現(xiàn)后，國內(nèi)社區(qū)火爆程度上升，，ES的文檔非常健全

4.ES對云計算和大數(shù)據(jù)支持很好

1.4倒排索引

[外鏈圖片轉(zhuǎn)存失敗,源站可能有防盜鏈機制,建議將圖片保存下來直接上傳(img-o2NLKDlo-1598685138000)(ES筆記.assets/image-20200727144457339.png)]

1.將存放的數(shù)據(jù)以一定的方式進行分詞，并將分詞的內(nèi)容存放到一個單獨的分詞庫中。

2.當用戶取查詢數(shù)據(jù)時，會將用戶的查詢關(guān)鍵字進行分詞，然后去分詞庫中匹配內(nèi)容，最終得到數(shù)據(jù)的id標識

3.根據(jù)id標識去存放數(shù)據(jù)的位置拉去指定數(shù)據(jù)

2 安裝

2.1 elasticsearch 安裝

http://hub.daocloud.io/ docker 鏡像工廠地址

version: "3.1" services:elasticsearch: image: daocloud.io/library/elasticsearch:6.5.4restart: alwayscontainer_name: elasticsearchports: - 9200:9200- 9300:9300kibana:image: daocloud.io/library/kibana:6.5.4restart: alwayscontainer_name: kibanaports: - 9200:9200environment:- elasticsearch_url=ip:9200depends_on:- elasticseatch

或者本地下載

2.2

https://github.com/medcl/elasticsearch-analysis-ik/archive/v6.8.10.zip

官方給的安裝的辦法 ./bin/elasticsearch-plugin install https://github.com/medcl/elasticsearch-analysis-ik/releases/download/v6.3.0/elasticsearch-analysis-ik-6.3.0.zip

本地自己安裝

https://github.com/medcl/elasticsearch-analysis-ik/archive/v6.8.10.zip 下載好后，執(zhí)行 mvn clean package 打包（注意pom文件中的es的版本，如果和自己的es的版本不一致，手動改下） elasticsearch-analysis-ik-6.8.10\target\releases 中壓縮包的內(nèi)容copy到 elasticsearch-6.8.10\plugins\ik 下

kibana 主要用到 Dev Tools 和 Management

POST _analyze {"analyzer": "ik_max_word","text":"我是中國人" }

3 es的基本操作

3.1 es的結(jié)構(gòu)

3.1.1索引indx，分片，備份

ES服務(wù)中會創(chuàng)建多個索引每個縮影默認被分成5個分片每個分片存在至少一個備份分片備份分片不會幫助檢索數(shù)據(jù)（當ES檢索壓力特別大的時候才，備份分片才會幫助檢索數(shù)據(jù)）備份的分片必須放在不同的服務(wù)器中

[外鏈圖片轉(zhuǎn)存失敗,源站可能有防盜鏈機制,建議將圖片保存下來直接上傳(img-fE98miFr-1598685138003)(ES筆記.assets/image-20200727174836230.png)]

3.1.2 類型type

一個索引下可以創(chuàng)建多個類型 PS:版本不同,類型的創(chuàng)建也不同

[外鏈圖片轉(zhuǎn)存失敗,源站可能有防盜鏈機制,建議將圖片保存下來直接上傳(img-nkXLIjq2-1598685138007)(ES筆記.assets/image-20200727175427524.png)]

3.1.3 文檔document

一個類型下可以有多個文檔，這個文檔就相當于mysql表中的多行數(shù)據(jù)

[外鏈圖片轉(zhuǎn)存失敗,源站可能有防盜鏈機制,建議將圖片保存下來直接上傳(img-KDirLSHo-1598685138014)(ES筆記.assets/image-20200727175655572.png)]

3.1.4 屬性field

一個文檔中可以包含多個屬性，類似于mysql 表中的一行數(shù)據(jù)有多個列

[外鏈圖片轉(zhuǎn)存失敗,源站可能有防盜鏈機制,建議將圖片保存下來直接上傳(img-1JZLPpGl-1598685138025)(ES筆記.assets/image-20200727180642583.png)]

3.2操作ES的restful語法

GET請求：http://ip:port/index :查詢索引信息http://ip:port/index/type/doc_id :查詢指定的文檔信息 POST請求：http://ip:port/index/type/_search: 查詢文檔，可以在請求體中添加json字符串來代表查詢條件http://ip:port/index/type/doc_id/_update: 修改文檔，在請求體中添加json字符串來代表修改的信息 PUT請求：http://ip:port/index : 創(chuàng)建一個索引，需要在請求體中指定索引的信息http://ip:port/index/type/_mappings:代表創(chuàng)建索引時，指定索引文檔存儲屬性的信息 DELETE 請求：http://ip:port/index：刪除跑路http://ip:port/index/type/doc_id: 刪除指定的文檔

3.3 索引的操作

3.3.1 創(chuàng)建一個索引

#創(chuàng)建一個索引 #number_of_shards 分片 #number_of_replicas 備份 PUT /person {"settings": {"number_of_shards": 5, "number_of_replicas": 1} }

3.3.2 查看一個索引

1.management2. #查看索引信息 GET /person

3.3.3 刪除索引

1.management2. #刪除索引 DELETE /person

3.4 ES中Field可以指定的類型

https://www.elastic.co/guide/en/elasticsearch/reference/6.8/mapping-types.html 官方文檔

字符串類型:text: 一般用于全文檢索，將當前field 進行分詞keyword:當前field 不會進行分詞數(shù)值類型：long:Intger:short:byte:double:float:half_float: 精度比float 小一半scaled_float:根據(jù)一個long 和scaled 來表達一個浮點型 long-345, -scaled 100 ->3.45 時間類型：date類型,根據(jù)時間類型指定具體的格式PUT my_index{"mappings": {"_doc": {"properties": {"date": {"type": "date","format": "yyyy-MM-dd HH:mm:ss||yyyy-MM-dd||epoch_millis"}}}}} 布爾類型：boolean 類型，表達true 和false 二進制類型：binary類型暫時支持Base64編碼的字符串范圍類型：integer_range：float_range：long_range：賦值時，無需指定具體的內(nèi)容，只需存儲一個范圍即可，gte,lte,gt,lt,double_range：date_range：ip_range：PUT range_index{"settings": {"number_of_shards": 2},"mappings": {"_doc": {"properties": {"expected_attendees": {"type": "integer_range"},"time_frame": {"type": "date_range", "format": "yyyy-MM-dd HH:mm:ss||yyyy-MM-dd||epoch_millis"}}}}}PUT range_index/_doc/1?refresh{"expected_attendees" : { "gte" : 10,"lte" : 20},"time_frame" : { "gte" : "2015-10-31 12:00:00", "lte" : "2015-11-01"}} 經(jīng)緯度類型：geo_point:用來存儲經(jīng)緯度 IP類型：ip:可以存儲IPV4 和IPV6 其他的數(shù)據(jù)類型，參考官網(wǎng)

3.5 創(chuàng)建索引并指定數(shù)據(jù)結(jié)構(gòu)

#創(chuàng)建索引，指定數(shù)據(jù)類型 PUT /book {"settings": {#分片數(shù)"number_of_shards": 5,#備份數(shù)"number_of_replicas": 1},#指定數(shù)據(jù)類型"mappings": {#類型 Type"novel":{#文檔存儲的field"properties":{#field屬性名"name":{#類型"type":"text",#指定分詞器"analyzer":"ik_max_word",#指定當前的field可以被作為查詢的條件"index":true,#是否需要額外存儲"store":false},"author":{"type":"keyword"},"count":{"type":"long"},"on-sale":{"type":"date",#指定時間類型的格式化方式"format": "yyyy-MM-dd HH:mm:ss||yyyy-MM-dd||epoch_millis"},"descr":{"type":"text","analyzer":"ik_max_word"}}}} }

3.6 文檔操作

文檔在ES服務(wù)中的唯一標識， _indx ,_type,_id 三個內(nèi)容為組合，鎖定一個文檔，操作時添加還時修改操作，

3.6.1 新建文檔

自動生成id #添加文檔，自動生成id POST /book/novel {"name":"盤龍","author":"我吃西紅柿","count":100000,"on-sale":"2001-01-01","descr":"大小的血睛鬃毛獅，力大無窮的紫睛金毛猿，毀天滅地的九頭蛇皇，攜帶著毀滅雷電的恐怖雷龍……這里無奇不有，這是一個廣博的魔幻世界。強者可以站在黑色巨龍的頭頂遨游天際，恐怖的魔法可以焚燒江河，可以毀滅城池，可以夷平山岳……" }#添加文檔,手動指定id PUT /book/novel/1 {"name":"紅樓夢","author":"曹雪芹","count":10000000,"on-sale":"2501-01-01","descr":"中國古代章回體長篇小說，中國古典四大名著之一，一般認為是清代作家曹雪芹所著。小說以賈、史、王、薛四大家族的興衰為背景，以富貴公子賈寶玉為視角，以賈寶玉與林黛玉、薛寶釵的愛情婚姻悲劇為主線，描繪了一批舉止見識出于須眉之上的閨閣佳人的人生百態(tài)，展現(xiàn)了真正的人性美和悲劇美" }

3.6.2 修改文檔

1.覆蓋式修改

#添加文檔,手動指定id PUT /book/novel/1 {"name":"紅樓夢","author":"曹雪芹","count":1000444,"on-sale":"2501-01-01","descr":"中國古代章回體長篇小說，中國古典四大名著之一，一般認為是清代作家曹雪芹所著。小說以賈、史、王、薛四大家族的興衰為背景，以富貴公子賈寶玉為視角，以賈寶玉與林黛玉、薛寶釵的愛情婚姻悲劇為主線，描繪了一批舉止見識出于須眉之上的閨閣佳人的人生百態(tài)，展現(xiàn)了真正的人性美和悲劇美" }

2.使用doc修改方式

#修改文檔，使用doc 方式 POST /book/novel/1/_update {"doc":{#指定需要修改的field和對應(yīng)的值"count":566666} }

3.6.3 刪除文檔

#根據(jù)id刪除文檔 DELETE /book/novel/3mEnk3MBaSKoGN4T2olw

4.java 操作ElasticSearch

4.1 java 連接ES

創(chuàng)建maven工程導(dǎo)入依賴 <dependency><groupId>org.elasticsearch</groupId><artifactId>elasticsearch</artifactId><version>6.8.10</version></dependency> <dependency><groupId>org.elasticsearch.client</groupId><artifactId>elasticsearch-rest-high-level-client</artifactId><version>6.8.10</version></dependency> <dependency><groupId>junit</groupId><artifactId>junit</artifactId><version>4.12</version></dependency> <dependency><groupId>org.projectlombok</groupId><artifactId>lombok</artifactId><version>1.16.22</version></dependency>

創(chuàng)建client鏈接

package com.utils;import org.apache.http.HttpHost; import org.elasticsearch.client.RestClient; import org.elasticsearch.client.RestClientBuilder; import org.elasticsearch.client.RestHighLevelClient;public class EsClient {public static RestHighLevelClient getClient(){// 創(chuàng)建 HttpHostHttpHost httpHost = new HttpHost("127.0.0.1",9200);// 創(chuàng)建 RestClientBuilderRestClientBuilder builder = RestClient.builder(httpHost);// 創(chuàng)建 RestHighLevelClientRestHighLevelClient client = new RestHighLevelClient(builder);return client;} }

4.2創(chuàng)建索引

package com.test;import com.utils.EsClient; import org.elasticsearch.action.admin.indices.create.CreateIndexRequest; import org.elasticsearch.action.admin.indices.create.CreateIndexResponse; import org.elasticsearch.client.RequestOptions; import org.elasticsearch.client.RestHighLevelClient; import org.elasticsearch.common.settings.Settings; import org.elasticsearch.common.xcontent.XContentBuilder; import org.elasticsearch.common.xcontent.json.JsonXContent; import org.junit.Test;public class Demo2 {RestHighLevelClient client = EsClient.getClient();String index = "person";String type="man";@Testpublic void createIndx() throws Exception{// 1.準備關(guān)于索引的settingSettings.Builder settings = Settings.builder().put("number_of_shards", 2).put("number_of_replicas", 1);// 2.準備關(guān)于索引的mappingXContentBuilder mappings = JsonXContent.contentBuilder().startObject().startObject("properties").startObject("name").field("type", "text").endObject().startObject("age").field("type", "integer").endObject().startObject("birthday").field("type", "date").field("format", "yyyy-MM-dd").endObject().endObject().endObject();// 3.將settings和mappings 封裝到到一個Request對象中CreateIndexRequest request = new CreateIndexRequest(index).settings(settings).mapping(type,mappings);// 4.使用client 去連接ESCreateIndexResponse response = client.indices().create(request, RequestOptions.DEFAULT);System.out.println("response:"+response.toString());} }

4.3 檢查索引是否存在，刪除索引

4.3.1 檢查索引存在

4.3.2 刪除索引

package com.test;import com.utils.EsClient; import org.elasticsearch.action.admin.indices.create.CreateIndexRequest; import org.elasticsearch.action.admin.indices.create.CreateIndexResponse; import org.elasticsearch.action.admin.indices.delete.DeleteIndexRequest; import org.elasticsearch.action.support.master.AcknowledgedResponse; import org.elasticsearch.client.RequestOptions; import org.elasticsearch.client.RestHighLevelClient; import org.elasticsearch.client.indices.GetIndexRequest; import org.elasticsearch.common.settings.Settings; import org.elasticsearch.common.xcontent.XContentBuilder; import org.elasticsearch.common.xcontent.json.JsonXContent; import org.junit.Test;import java.io.IOException;public class Demo2 {RestHighLevelClient client = EsClient.getClient();String index = "person";String type="man";@Testpublic void testDelete() throws IOException {// 1.獲取requestDeleteIndexRequest request = new DeleteIndexRequest(index);// 2.使用client 操作requestAcknowledgedResponse delete = client.indices().delete(request, RequestOptions.DEFAULT);// 3.輸出結(jié)果System.out.println(delete.isAcknowledged());} }

4.4 Java操作文檔

4.4.1 添加文檔操作

public class Demo3 {ObjectMapper mapper = new ObjectMapper();RestHighLevelClient client = EsClient.getClient();String index = "person";String type="man";@Testpublic void createDocTest() throws IOException {// 1.準備一個json數(shù)據(jù)Person person = new Person(1,"張三",33,new Date());String json = mapper.writeValueAsString(person);// 2.創(chuàng)建一個request對象(手動指定的方式創(chuàng)建)IndexRequest request = new IndexRequest(index,type,person.getId().toString());request.source(json, XContentType.JSON);// 3.使用client 操作request對象生成docIndexResponse response = client.index(request, RequestOptions.DEFAULT);// 4.輸出返回結(jié)果System.out.println(response.getResult().toString());} }

4.4.2 修改文檔

public class Demo3 {ObjectMapper mapper = new ObjectMapper();RestHighLevelClient client = EsClient.getClient();String index = "person";String type="man";@Testpublic void updateDocTest() throws Exception{// 1.創(chuàng)建要跟新的MapMap<String,Object> doc = new HashMap<>();doc.put("name","張三三");// 2.創(chuàng)建request, 將doc 封裝進去UpdateRequest request = new UpdateRequest(index,type,"1");request.doc(doc);// 3. client 去操作 requestUpdateResponse response = client.update(request, RequestOptions.DEFAULT);// 4.輸出更新結(jié)果System.out.println(response.getResult());} }

4.4.3 刪除文檔

public class Demo3 {ObjectMapper mapper = new ObjectMapper();RestHighLevelClient client = EsClient.getClient();String index = "person";String type="man";@Testpublic void deleteDocTest() throws Exception{// 1.封裝刪除對象DeleteRequest request = new DeleteRequest(index,type,"1");// 2 client 操作 request對象DeleteResponse response = client.delete(request, RequestOptions.DEFAULT);// 3.輸出結(jié)果System.out.println(response.getResult().toString());} }

4.5 java批量操作文檔

批量操作新增

@Testpublic void bulkCreateDoc() throws Exception{// 1.準備多個json 對象Person p1 = new Person(1,"張三",23,new Date());Person p2 = new Person(2,"里斯",24,new Date());Person p3 = new Person(3,"王武",24,new Date());String json1 = mapper.writeValueAsString(p1);String json2 = mapper.writeValueAsString(p2);String json3 = mapper.writeValueAsString(p3);// 2.創(chuàng)建requestBulkRequest bulkRequest = new BulkRequest();bulkRequest.add(new IndexRequest(index,type,p1.getId().toString()).source(json1,XContentType.JSON)).add(new IndexRequest(index,type,p2.getId().toString()).source(json2,XContentType.JSON)).add(new IndexRequest(index,type,p3.getId().toString()).source(json3,XContentType.JSON));// 3.client 執(zhí)行BulkResponse responses = client.bulk(bulkRequest, RequestOptions.DEFAULT);// 4.輸出結(jié)果System.out.println(responses.getItems().toString());}

批量刪除

public void bulkDelete() throws Exception{// 1.創(chuàng)建Request 對象BulkRequest bulkRequest = new BulkRequest();bulkRequest.add(new DeleteRequest(index,type,"1"));bulkRequest.add(new DeleteRequest(index,type,"2"));bulkRequest.add(new DeleteRequest(index,type,"3"));// 2.執(zhí)行BulkResponse re = client.bulk(bulkRequest, RequestOptions.DEFAULT);// 3.輸出結(jié)果System.out.println(re.toString());}

5.ES 練習(xí)

索引：sms-logs-index 類型：sms-logs-type

[外鏈圖片轉(zhuǎn)存失敗,源站可能有防盜鏈機制,建議將圖片保存下來直接上傳(img-b2R80xn8-1598685138030)(ES筆記.assets/image-20200728173057412.png)]

public class Demo4 {ObjectMapper mapper = new ObjectMapper();RestHighLevelClient client = EsClient.getClient();String index = "sms-logs-index";String type="sms-logs-type";@Testpublic void createIndex() throws Exception{// 1.準備關(guān)于索引的settingSettings.Builder settings = Settings.builder().put("number_of_shards", 3).put("number_of_replicas", 1);// 2.準備關(guān)于索引的mappingXContentBuilder mappings = JsonXContent.contentBuilder().startObject().startObject("properties").startObject("corpName").field("type", "keyword").endObject().startObject("createDate").field("type", "date").field("format", "yyyy-MM-dd").endObject().startObject("fee").field("type", "long").endObject().startObject("ipAddr").field("type", "ip").endObject().startObject("longCode").field("type", "keyword").endObject().startObject("mobile").field("type", "keyword").endObject().startObject("operatorId").field("type", "integer").endObject().startObject("province").field("type", "keyword").endObject().startObject("replyTotal").field("type", "integer").endObject().startObject("sendDate").field("type", "date").field("format", "yyyy-MM-dd").endObject().startObject("smsContent").field("type", "text").field("analyzer", "ik_max_word").endObject().startObject("state").field("type", "integer").endObject().endObject().endObject();// 3.將settings和mappings 封裝到到一個Request對象中CreateIndexRequest request = new CreateIndexRequest(index).settings(settings).mapping(type,mappings);// 4.使用client 去連接ESCreateIndexResponse response = client.indices().create(request, RequestOptions.DEFAULT);System.out.println("response:"+response.toString());}@Testpublic void bulkCreateDoc() throws Exception{// 1.準備多個json 對象String longcode = "1008687";String mobile ="138340658";List<String> companies = new ArrayList<>();companies.add("騰訊課堂");companies.add("阿里旺旺");companies.add("海爾電器");companies.add("海爾智家公司");companies.add("格力汽車");companies.add("蘇寧易購");List<String> provinces = new ArrayList<>();provinces.add("北京");provinces.add("重慶");provinces.add("上海");provinces.add("晉城");BulkRequest bulkRequest = new BulkRequest();for (int i = 1; i <16 ; i++) {Thread.sleep(1000);SmsLogs s1 = new SmsLogs();s1.setId(i);s1.setCreateDate(new Date());s1.setSendDate(new Date());s1.setLongCode(longcode+i);s1.setMobile(mobile+2*i);s1.setCorpName(companies.get(i%5));s1.setSmsContent(SmsLogs.doc.substring((i-1)*100,i*100));s1.setState(i%2);s1.setOperatorId(i%3);s1.setProvince(provinces.get(i%4));s1.setIpAddr("127.0.0."+i);s1.setReplyTotal(i*3);s1.setFee(i*6+"");String json1 = mapper.writeValueAsString(s1);bulkRequest.add(new IndexRequest(index,type,s1.getId().toString()).source(json1, XContentType.JSON));System.out.println("數(shù)據(jù)"+i+s1.toString());}// 3.client 執(zhí)行BulkResponse responses = client.bulk(bulkRequest, RequestOptions.DEFAULT);// 4.輸出結(jié)果System.out.println(responses.getItems().toString());} }

6.ElasticSearch的各種查詢

6.1 term 和terms 查詢

6.1.1 term 查詢

term 查詢是代表完全匹配，搜索之前不會對你搜索的關(guān)鍵字進行分詞，直接拿關(guān)鍵字去文檔分詞庫中匹配內(nèi)容 #term查詢 POST /sms-logs-index/sms-logs-type/_search {#limit ?"from": 0, #limit x,?"size":5,"query": {"term": {"province": {"value": "北京"}}} } public class TermSearch {ObjectMapper mapper = new ObjectMapper();RestHighLevelClient client = EsClient.getClient();String index = "sms-logs-index";String type="sms-logs-type";@Testpublic void termSearchTest() throws IOException {// 1.創(chuàng)建request對象SearchRequest request = new SearchRequest(index);request.types(type);// 2.創(chuàng)建查詢條件SearchSourceBuilder builder = new SearchSourceBuilder();builder.from(0);builder.size(5);builder.query(QueryBuilders.termQuery("province","北京"));request.source(builder);// 3.執(zhí)行查詢SearchResponse response = client.search(request, RequestOptions.DEFAULT);// 4.輸出查詢結(jié)果for (SearchHit hit : response.getHits().getHits()) {Map<String, Object> sourceAsMap = hit.getSourceAsMap();System.out.println(sourceAsMap);}} }

6.1.2 查詢

terms 和 term 查詢的機制一樣，搜索之前不會對你搜索的關(guān)鍵字進行分詞，直接拿關(guān)鍵字去文檔分詞庫中匹配內(nèi)容 terms:是針對一個字段包含多個值 term : where province =北京 terms: where province = 北京 or province =? (類似于mysql 中的 in) 也可針對 text, 只是在分詞庫中查詢的時候不會進行分詞 #terms 查詢 POST /sms-logs-index/sms-logs-type/_search {"query": {"terms": {"province": ["北京","晉城"]}} } public class TermSearch {ObjectMapper mapper = new ObjectMapper();RestHighLevelClient client = EsClient.getClient();String index = "sms-logs-index";String type="sms-logs-type";@Testpublic void termsSearchTest() throws IOException {// 1.創(chuàng)建request對象SearchRequest request = new SearchRequest(index);request.types(type);// 2.創(chuàng)建查詢條件SearchSourceBuilder builder = new SearchSourceBuilder();builder.query(QueryBuilders.termsQuery("province","北京","晉城"));request.source(builder);// 3.執(zhí)行查詢SearchResponse response = client.search(request, RequestOptions.DEFAULT);// 輸出查詢結(jié)果for (SearchHit hit : response.getHits().getHits()) {System.out.println(hit.getSourceAsMap());}} }

6.2 match

match 查詢屬于高級查詢，會根據(jù)你查詢字段的類型不一樣，采用不同的查詢方式查詢的是日期或者數(shù)值，他會將你基于字符串的查詢內(nèi)容轉(zhuǎn)換為日期或數(shù)值對待如果查詢的內(nèi)容是一個不能被分詞的內(nèi)容（keyword）,match 不會將你指定的關(guān)鍵字進行分詞如果查詢的內(nèi)容是一個可以被分詞的內(nèi)容（text）,match 查詢會將你指定的內(nèi)容根據(jù)一定的方式進行分詞，去分詞庫中匹配指定的內(nèi)容 match 查詢，實際底層就是多個term 查詢，將多個term查詢的結(jié)果給你封裝到一起

6.2.1 math_all

查詢?nèi)績?nèi)容，不指定查詢條件 #match_all 查詢 POST /sms-logs-index/sms-logs-type/_search {"query":{"match_all": {}} } public class MatchSearch {ObjectMapper mapper = new ObjectMapper();RestHighLevelClient client = EsClient.getClient();String index = "sms-logs-index";String type="sms-logs-type";@Testpublic void matchAllSearch() throws IOException {// 1.創(chuàng)建request對象SearchRequest request = new SearchRequest(index);request.types(type);// 2.創(chuàng)建查詢條件SearchSourceBuilder builder = new SearchSourceBuilder();builder.query(QueryBuilders.matchAllQuery());// ES 默認只查詢10條數(shù)據(jù)builder.size(20);request.source(builder);// 3.執(zhí)行查詢SearchResponse response = client.search(request, RequestOptions.DEFAULT);// 4.輸出查詢結(jié)果for (SearchHit hit : response.getHits().getHits()) {System.out.println(hit.getSourceAsMap());}System.out.println(response.getHits().getHits().length);} }

6.2.2 match 查詢

指定一個field 作為查詢條件

#match 查詢 POST /sms-logs-index/sms-logs-type/_search {"query": {"match": {"smsContent": "偉大戰(zhàn)士"}} } public class MatchSearch {ObjectMapper mapper = new ObjectMapper();RestHighLevelClient client = EsClient.getClient();String index = "sms-logs-index";String type="sms-logs-type";@Testpublic void matchSearch() throws IOException {// 1.創(chuàng)建request對象SearchRequest request = new SearchRequest(index);request.types(type);// 2.創(chuàng)建查詢條件SearchSourceBuilder builder = new SearchSourceBuilder();//--------------------------------------------------------------builder.query(QueryBuilders.matchQuery("smsContent","偉大戰(zhàn)士"));//--------------------------------------------------------------builder.size(20);request.source(builder);// 3.執(zhí)行查詢SearchResponse response = client.search(request, RequestOptions.DEFAULT);// 4.輸出查詢結(jié)果for (SearchHit hit : response.getHits().getHits()) {System.out.println(hit.getSourceAsMap());}System.out.println(response.getHits().getHits().length);}}

6.2.3 布爾match 查詢

基于一個field 匹配的內(nèi)容，按照 and 或者or的方式連接 #布爾match查詢 POST /sms-logs-index/sms-logs-type/_search {"query": {"match": {"smsContent": {# 既包含戰(zhàn)士也包含團隊"query": "戰(zhàn)士團隊","operator": "and"}}} }#布爾match查詢 POST /sms-logs-index/sms-logs-type/_search {"query": {"match": {"smsContent": {# 既包含戰(zhàn)士或者團隊"query": "戰(zhàn)士團隊","operator": "or"}}} } @Testpublic void booleanMatchSearch() throws IOException {// 1.創(chuàng)建request對象SearchRequest request = new SearchRequest(index);request.types(type);// 2.創(chuàng)建查詢條件SearchSourceBuilder builder = new SearchSourceBuilder();//--------------------------------------------------------------builder.query(QueryBuilders.matchQuery("smsContent","戰(zhàn)士團隊").operator(Operator.AND));//--------------------------------------------------------------builder.size(20);request.source(builder);// 3.執(zhí)行查詢SearchResponse response = client.search(request, RequestOptions.DEFAULT);// 4.輸出查詢結(jié)果for (SearchHit hit : response.getHits().getHits()) {System.out.println(hit.getSourceAsMap());}System.out.println(response.getHits().getHits().length);}

6.2.4 multi_match

match 針對一個field 做檢索，multi_math 針對多個field 進行檢索，多個field對應(yīng)一個文本。 #multi_math 查詢 POST /sms-logs-index/sms-logs-type/_search {"query":{"multi_match": {"query": "北京","fields": ["province","smsContent"]}} } public void multiMatchSearch() throws IOException {// 1.創(chuàng)建request對象SearchRequest request = new SearchRequest(index);request.types(type);// 2.創(chuàng)建查詢條件SearchSourceBuilder builder = new SearchSourceBuilder();//--------------------------------------------------------------builder.query(QueryBuilders.multiMatchQuery("北京","province","smsContent"));//--------------------------------------------------------------builder.size(20);request.source(builder);// 3.執(zhí)行查詢SearchResponse response = client.search(request, RequestOptions.DEFAULT);// 4.輸出查詢結(jié)果for (SearchHit hit : response.getHits().getHits()) {System.out.println(hit.getSourceAsMap());}System.out.println(response.getHits().getHits().length);}

6.3 其他查詢

6.3.1id 查詢

#id 查詢 GET /sms-logs-index/sms-logs-type/1 public class IdGetSearch {ObjectMapper mapper = new ObjectMapper();RestHighLevelClient client = EsClient.getClient();String index = "sms-logs-index";String type="sms-logs-type";@Testpublic void findById() throws IOException {// 創(chuàng)建GetRequest對象GetRequest request = new GetRequest(index,type,"1");// 執(zhí)行查詢GetResponse response = client.get(request, RequestOptions.DEFAULT);// 輸出結(jié)果System.out.println(response.getSourceAsMap());} }

6.3.2 ids查詢

根據(jù)多個id 查詢,類似 mysql 中的 where in (id1,id2...) #ids 查詢 POST /sms-logs-index/sms-logs-type/_search {"query": {"ids": {"values": ["1","2","3"]}} } public void findByIds() throws IOException {// 創(chuàng)建request對象SearchRequest request = new SearchRequest(index);request.types(type);// 指定查詢條件SearchSourceBuilder builder = new SearchSourceBuilder();//--------------------------------------------------builder.query(QueryBuilders.idsQuery().addIds("1","2","3"));//------------------------------------------------------request.source(builder);// 執(zhí)行SearchResponse response = client.search(request, RequestOptions.DEFAULT);// 輸出結(jié)果for (SearchHit hit : response.getHits().getHits()) {System.out.println(hit.getSourceAsMap());}}

6.3.3 prefix 查詢

前綴查詢，可以通過一個關(guān)鍵字去指定一個field 的前綴，從而查詢到指定文檔

#prefix 查詢 POST /sms-logs-index/sms-logs-type/_search {"query": {"prefix": {"corpName": {"value": "海"}}} } #match 查詢在這里是什么都查不到的和上邊的prefix 做比較 POST /sms-logs-index/sms-logs-type/_search {"query": {"match": {"corpName": "海"}} } public void findByPrefix() throws IOException {// 創(chuàng)建request對象SearchRequest request = new SearchRequest(index);request.types(type);// 指定查詢條件SearchSourceBuilder builder = new SearchSourceBuilder();//--------------------------------------------------builder.query(QueryBuilders.prefixQuery("corpName","阿"));//------------------------------------------------------request.source(builder);// 執(zhí)行SearchResponse response = client.search(request, RequestOptions.DEFAULT);// 輸出結(jié)果for (SearchHit hit : response.getHits().getHits()) {System.out.println(hit.getSourceAsMap());}}

6.3.4 fuzzy 查詢

模糊查詢，我們可以輸入一個字符的大概，ES 可以根據(jù)輸入的大概去匹配內(nèi)容。查詢結(jié)果不穩(wěn)定 #fuzzy 查詢 POST /sms-logs-index/sms-logs-type/_search {"query": {"fuzzy": {"corpName": {"value": "騰訊客堂",#指定前邊幾個字符是不允許出現(xiàn)錯誤的"prefix_length": 2}}} } public void findByFuzzy() throws IOException {// 創(chuàng)建request對象SearchRequest request = new SearchRequest(index);request.types(type);// 指定查詢條件SearchSourceBuilder builder = new SearchSourceBuilder();//--------------------------------------------------builder.query(QueryBuilders.fuzzyQuery("corpName","騰訊客堂").prefixLength(2));//------------------------------------------------------request.source(builder);// 執(zhí)行SearchResponse response = client.search(request, RequestOptions.DEFAULT);// 輸出結(jié)果for (SearchHit hit : response.getHits().getHits()) {System.out.println(hit.getSourceAsMap());}}

6.3.5 wildcard 查詢

通配查詢，同mysql中的like 是一樣的，可以在查詢時，在字符串中指定通配符*和占位符？ #wildcard 查詢 POST /sms-logs-index/sms-logs-type/_search {"query": {"wildcard": {"corpName": {"value": "海爾*"}}} }#wildcard 查詢 POST /sms-logs-index/sms-logs-type/_search {"query": {"wildcard": {"corpName": {"value": "海爾??"}}} } public void findByWildCard() throws IOException {// 創(chuàng)建request對象SearchRequest request = new SearchRequest(index);request.types(type);// 指定查詢條件SearchSourceBuilder builder = new SearchSourceBuilder();//--------------------------------------------------builder.query(QueryBuilders.wildcardQuery("corpName","海爾*"));//------------------------------------------------------request.source(builder);// 執(zhí)行SearchResponse response = client.search(request, RequestOptions.DEFAULT);// 輸出結(jié)果for (SearchHit hit : response.getHits().getHits()) {System.out.println(hit.getSourceAsMap());}}

6.3.6 rang 查詢

范圍查詢，只針對數(shù)值類型，對一個field 進行大于或者小于的范圍指定

#rang 查詢 POST /sms-logs-index/sms-logs-type/_search {"query": {"range": {"fee": {"gte": 10,"lte": 20}}} } public void findByRang() throws IOException {// 創(chuàng)建request對象SearchRequest request = new SearchRequest(index);request.types(type);// 指定查詢條件SearchSourceBuilder builder = new SearchSourceBuilder();//--------------------------------------------------builder.query(QueryBuilders.rangeQuery("fee").gt(10).lte(30));//------------------------------------------------------request.source(builder);// 執(zhí)行SearchResponse response = client.search(request, RequestOptions.DEFAULT);// 輸出結(jié)果for (SearchHit hit : response.getHits().getHits()) {System.out.println(hit.getSourceAsMap());}}

6.3.7 regexp 查詢

正則查詢，通過你編寫的正則表達式去匹配內(nèi)容 Ps:prefix wildcard fuzzy 和regexp 查詢效率比較低 ,在要求效率比較高時，避免使用 #regexp 查詢 POST /sms-logs-index/sms-logs-type/_search {"query": {"regexp": {"mobile": "138[0-9]{8}"}} } public void findByRegexp() throws IOException {// 創(chuàng)建request對象SearchRequest request = new SearchRequest(index);request.types(type);// 指定查詢條件SearchSourceBuilder builder = new SearchSourceBuilder();//--------------------------------------------------builder.query(QueryBuilders.regexpQuery("mobile","138[0-9]{8}"));//------------------------------------------------------request.source(builder);// 執(zhí)行SearchResponse response = client.search(request, RequestOptions.DEFAULT);// 輸出結(jié)果for (SearchHit hit : response.getHits().getHits()) {System.out.println(hit.getSourceAsMap());} }

6.4 深分頁 scrol l

ES 對from +size時又限制的，from +size 之和不能大于1W,超過后效率會十分低下原理：from+size ES查詢數(shù)據(jù)的方式，第一步將用戶指定的關(guān)鍵詞進行分詞，第二部將詞匯去分詞庫中進行檢索，得到多個文檔id,第三步去各個分片中拉去數(shù)據(jù)，耗時相對較長第四步根據(jù)score 將數(shù)據(jù)進行排序，耗時相對較長第五步根據(jù)from 和size 的值將部分數(shù)據(jù)舍棄，第六步，返回結(jié)果。scroll +size ES 查詢數(shù)據(jù)的方式第一步將用戶指定的關(guān)鍵詞進行分詞，第二部將詞匯去分詞庫中進行檢索，得到多個文檔id,第三步，將文檔的id放在一個上下文中第四步，根據(jù)指定的size去ES中檢索指定個數(shù)數(shù)據(jù)，拿完數(shù)據(jù)的文檔id,會從上下文中移除第五步，如果需要下一頁的數(shù)據(jù)，直接去ES的上下文中找后續(xù)內(nèi)容。第六步，循環(huán)第四步和第五步scroll 不適合做實時查詢。 #scroll 查詢,返回第一頁數(shù)據(jù)，并將文檔id信息存放在ES上下文中，并指定生存時間 POST /sms-logs-index/sms-logs-type/_search?scroll=1m {"query": {"match_all": {}},"size": 2,"sort": [{"fee": {"order": "desc"}}] }#根據(jù)scroll 查詢下一頁數(shù)據(jù) POST _search/scroll {"scroll_id":"DnF1ZXJ5VGhlbkZldGNoAwAAAAAAABbqFk04VlZ1cjlUU2t1eHpsQWNRY1YwWWcAAAAAAAAW7BZNOFZWdXI5VFNrdXh6bEFjUWNWMFlnAAAAAAAAFusWTThWVnVyOVRTa3V4emxBY1FjVjBZZw==","scroll":"1m" }#刪除scroll上下文中的數(shù)據(jù) DELETE _search/scroll/DnF1ZXJ5VGhlbkZldGNoAwAAAAAAABchFk04VlZ1cjlUU2t1eHpsQWNRY1YwWWcAAAAAAAAXIBZNOFZWdXI5VFNrdXh6bEFjUWNWMFlnAAAAAAAAFx8WTThWVnVyOVRTa3V4emxBY1FjVjBZZw== public class ScrollSearch {ObjectMapper mapper = new ObjectMapper();RestHighLevelClient client = EsClient.getClient();String index = "sms-logs-index";String type="sms-logs-type";@Testpublic void scrollSearch() throws IOException {// 1.創(chuàng)建requestSearchRequest searchRequest = new SearchRequest(index);searchRequest.types(type);// 2.指定scroll信息,過期時間searchRequest.scroll(TimeValue.timeValueMinutes(1L));// 3.指定查詢條件SearchSourceBuilder builder = new SearchSourceBuilder();builder.size(4);builder.sort("fee", SortOrder.DESC);searchRequest.source(builder);// 4.獲取返回結(jié)果scrollId,獲取sourceSearchResponse response = client.search(searchRequest, RequestOptions.DEFAULT);String scrollId = response.getScrollId();System.out.println("-------------首頁數(shù)據(jù)---------------------");for (SearchHit hit : response.getHits().getHits()) {System.out.println(hit.getSourceAsMap());}while (true){// 5.創(chuàng)建scroll requestSearchScrollRequest scrollRequest = new SearchScrollRequest(scrollId);// 6.指定scroll 有效時間scrollRequest.scroll(TimeValue.timeValueMinutes(1L));// 7.執(zhí)行查詢，返回查詢結(jié)果SearchResponse scroll = client.scroll(scrollRequest, RequestOptions.DEFAULT);// 8.判斷是否查詢到數(shù)據(jù)，查詢到輸出SearchHit[] searchHits = scroll.getHits().getHits();if(searchHits!=null && searchHits.length >0){System.out.println("-------------下一頁數(shù)據(jù)---------------------");for (SearchHit hit : searchHits) {System.out.println(hit.getSourceAsMap());}}else{// 9.沒有數(shù)據(jù)，結(jié)束System.out.println("-------------結(jié)束---------------------");break;}}// 10.創(chuàng)建 clearScrollRequestClearScrollRequest clearScrollRequest = new ClearScrollRequest();// 11.指定scrollIdclearScrollRequest.addScrollId(scrollId);//12.刪除scrollClearScrollResponse clearScrollResponse = client.clearScroll(clearScrollRequest, RequestOptions.DEFAULT);// 13.輸出結(jié)果System.out.println("刪除scroll:"+clearScrollResponse.isSucceeded());} }

6.5 delete-by-query

根據(jù)term,match 等查詢方式去刪除大量索引 PS:如果你要刪除的內(nèi)容，時index下的大部分數(shù)據(jù)，推薦創(chuàng)建一個新的index,然后把保留的文檔內(nèi)容，添加到全新的索引 #Delet-by-query 刪除 POST /sms-logs-index/sms-logs-type/_delete_by_query {"query": {"range": {"fee": {"lt": 20}}} } public void deleteByQuery() throws IOException {// 1.創(chuàng)建DeleteByQueryRequestDeleteByQueryRequest request = new DeleteByQueryRequest(index);request.types(type);// 2.指定條件request.setQuery(QueryBuilders.rangeQuery("fee").lt(20));// 3.執(zhí)行BulkByScrollResponse response = client.deleteByQuery(request, RequestOptions.DEFAULT);// 4.輸出返回結(jié)果System.out.println(response.toString());}

6.6 復(fù)合查詢

復(fù)合過濾器，將你的多個查詢條件以一定的邏輯組合在一起，must:所有條件組合在一起，表示 and 的意思 must_not: 將must_not中的條件，全部都不匹配，表示not的意思 should:所有條件用should 組合在一起，表示or 的意思 #省是晉城或者北京 # 運營商不能是聯(lián)通 #smsContent 包含戰(zhàn)士和的 POST /sms-logs-index/sms-logs-type/_search {"query": {"bool": {"should": [{"term": {"province": {"value": "晉城"}}},{"term": {"province": {"value": "北京"}}}],"must_not": [{"term": {"operatorId": {"value": "2"}}}],"must": [{"match": {"smsContent": "戰(zhàn)士"}},{"match": {"smsContent": "的"}}]}} } public void boolSearch() throws IOException {// 1.創(chuàng)建 searchRequestSearchRequest request = new SearchRequest(index);request.types(type);// 2.指定查詢條件SearchSourceBuilder builder = new SearchSourceBuilder();BoolQueryBuilder boolQueryBuilder = new BoolQueryBuilder();// #省是晉城或者北京boolQueryBuilder.should(QueryBuilders.termQuery("province","北京"));boolQueryBuilder.should(QueryBuilders.termQuery("province","晉城"));//# 運營商不能是聯(lián)通boolQueryBuilder.mustNot(QueryBuilders.termQuery("operatorId",2));//#smsContent 包含戰(zhàn)士和的boolQueryBuilder.must(QueryBuilders.matchQuery("smsContent","戰(zhàn)士"));boolQueryBuilder.must(QueryBuilders.matchQuery("smsContent","的"));builder.query(boolQueryBuilder);request.source(builder);// 3.執(zhí)行查詢SearchResponse response = client.search(request, RequestOptions.DEFAULT);// 4.輸出結(jié)果for (SearchHit hit : response.getHits().getHits()) {System.out.println(hit.getSourceAsMap());}}

6.6.2 boosting 查詢

boosting 查詢可以幫助我們?nèi)ビ绊懖樵兒蟮膕corepositive:只有匹配上positive 查詢的內(nèi)容，才會被放到返回的結(jié)果集中negative: 如果匹配上了positive 也匹配上了negative, 就可以降低這樣的文檔score.negative_boost:指定系數(shù),必須小于1 0.5 關(guān)于查詢時，分數(shù)時如何計算的：搜索的關(guān)鍵字再文檔中出現(xiàn)的頻次越高，分數(shù)越高指定的文檔內(nèi)容越短，分數(shù)越高。我們再搜索時，指定的關(guān)鍵字也會被分詞，這個被分詞的內(nèi)容，被分詞庫匹配的個數(shù)越多，分數(shù)就越高。 #boosting 查詢 POST /sms-logs-index/sms-logs-type/_search {"query": {"boosting": {"positive": {"match": {"smsContent": "戰(zhàn)士"}}, "negative": {"match": {"smsContent": "團隊"}},"negative_boost": 0.2}} } public void boostSearch() throws IOException {// 1.創(chuàng)建 searchRequestSearchRequest request = new SearchRequest(index);request.types(type);// 2.指定查詢條件SearchSourceBuilder builder = new SearchSourceBuilder();BoostingQueryBuilder boost = QueryBuilders.boostingQuery(QueryBuilders.matchQuery("smsContent", "戰(zhàn)士"),QueryBuilders.matchQuery("smsContent", "團隊")).negativeBoost(0.2f);builder.query(boost);request.source(builder);// 3.執(zhí)行查詢SearchResponse response = client.search(request, RequestOptions.DEFAULT);// 4.輸出結(jié)果for (SearchHit hit : response.getHits().getHits()) {System.out.println(hit.getSourceAsMap());}}

6.7 filter 查詢

query 查詢：根據(jù)你的查詢條件，去計算文檔的匹配度得到一個分數(shù)，并根據(jù)分數(shù)排序，不會做緩存的。

filter 查詢：根據(jù)查詢條件去查詢文檔，不去計算分數(shù)，而且filter會對經(jīng)常被過濾的數(shù)據(jù)進行緩存。

#filter 查詢 POST /sms-logs-index/sms-logs-type/_search {"query": {"bool": {"filter": [{"term": {"corpName": "海爾智家公司"}},{"range":{"fee":{"lte":50}}}]}} } public void filter() throws IOException {// 1.searchRequestSearchRequest searchRequest = new SearchRequest(index);searchRequest.types(type);// 2.指定查詢條件SearchSourceBuilder sourceBuilder = new SearchSourceBuilder();BoolQueryBuilder boolBuilder = QueryBuilders.boolQuery();boolBuilder.filter(QueryBuilders.termQuery("corpName","海爾智家公司"));boolBuilder.filter(QueryBuilders.rangeQuery("fee").gt(20));sourceBuilder.query(boolBuilder);searchRequest.source(sourceBuilder);// 3.執(zhí)行SearchResponse response = client.search(searchRequest, RequestOptions.DEFAULT);// 4. 輸出結(jié)果for (SearchHit hit : response.getHits().getHits()) {System.out.println(hit.getSourceAsMap());System.out.println(hit.getId()+"的分數(shù)是："+hit.getScore());}}

6.8 高亮查詢

高亮查詢就是用戶輸入的關(guān)鍵字，以一定特殊樣式展示給用戶，讓用戶知道為什么這個結(jié)果被檢索出來高亮展示的數(shù)據(jù)，本身就是文檔中的一個field,單獨將field以highlight的形式返回給用戶 ES提供了一個highlight 屬性，他和query 同級別。frament_size: 指定高亮數(shù)據(jù)展示多少個字符回來pre_tags:指定前綴標簽<front color="red">post_tags:指定后綴標簽 #highlight 高亮查詢 POST /sms-logs-index/sms-logs-type/_search {"query": {"match": {"smsContent": "團隊"}},"highlight": {"fields": {"smsContent":{}},"pre_tags":"","post_tags":"","fragment_size":10} } public void highLightQuery() throws IOException {// 1.創(chuàng)建requestSearchRequest request = new SearchRequest(index);request.types(type);// 2.指定查詢條件，指定高亮SearchSourceBuilder builder = new SearchSourceBuilder();builder.query(QueryBuilders.matchQuery("smsContent","團隊"));HighlightBuilder highlightBuilder = new HighlightBuilder();highlightBuilder.field("smsContent",10).preTags("").postTags("");builder.highlighter(highlightBuilder);request.source(builder);// 3.執(zhí)行SearchResponse response = client.search(request, RequestOptions.DEFAULT);//4. 輸出結(jié)果for (SearchHit hit : response.getHits().getHits()) {System.out.println(hit.getHighlightFields().get("smsContent"));}}

6.9 聚合查詢

ES的聚合查詢和mysql 的聚合查詢類似，ES的聚合查詢相比mysql 要強大得多。ES提供的統(tǒng)計數(shù)據(jù)的方式多種多樣。 #ES 聚合查詢的RSTFul 語法 POST /index/type/_search {"aggs":{"(名字)agg":{"agg_type":{"屬性"："值"}}} }

6.9.1 去重計數(shù)聚合查詢

去重計數(shù)，cardinality 先將返回的文檔中的一個指定的field進行去重，統(tǒng)計一共有多少條 # 去重計數(shù) 查詢 province POST /sms-logs-index/sms-logs-type/_search {"aggs": {"provinceAgg": {"cardinality": {"field": "province"}}} } public void aggCardinalityC() throws IOException {// 1.創(chuàng)建requestSearchRequest request = new SearchRequest(index);request.types(type);// 2. 指定使用聚合查詢方式SearchSourceBuilder builder = new SearchSourceBuilder();builder.aggregation(AggregationBuilders.cardinality("provinceAgg").field("province"));request.source(builder);// 3.執(zhí)行查詢SearchResponse response = client.search(request, RequestOptions.DEFAULT);// 4.輸出返回結(jié)果Cardinality agg = response.getAggregations().get("provinceAgg");System.out.println(agg.getValue());}

6.9.2 范圍統(tǒng)計

統(tǒng)計一定范圍內(nèi)出現(xiàn)的文檔個數(shù)，比如，針對某一個field 的值再0~100,100~200,200~300 之間文檔出現(xiàn)的個數(shù)分別是多少范圍統(tǒng)計可以針對普通的數(shù)值，針對時間類型，針對ip類型都可以響應(yīng)。數(shù)值 rang 時間 date_rang ip ip_rang #針對數(shù)值方式的范圍統(tǒng)計 from 帶等于效果，to 不帶等于效果 POST /sms-logs-index/sms-logs-type/_search {"aggs": {"agg": {"range": {"field": "fee","ranges": [{"to": 30},{"from": 30,"to": 60},{"from": 60}]}}} } #時間方式統(tǒng)計 POST /sms-logs-index/sms-logs-type/_search {"aggs": {"agg": {"date_range": {"field": "sendDate","format": "yyyy", "ranges": [{"to": "2000"},{"from": "2000"}]}}} } #ip 方式范圍統(tǒng)計 POST /sms-logs-index/sms-logs-type/_search {"aggs": {"agg": {"ip_range": {"field": "ipAddr","ranges": [{"to": "127.0.0.8"},{"from": "127.0.0.8"}]}}} } public void aggRang() throws IOException {// 1.創(chuàng)建requestSearchRequest request = new SearchRequest(index);request.types(type);// 2. 指定使用聚合查詢方式SearchSourceBuilder builder = new SearchSourceBuilder();builder.aggregation(AggregationBuilders.range("agg").field("fee").addUnboundedTo(30).addRange(30,60).addUnboundedFrom(60));request.source(builder);// 3.執(zhí)行查詢SearchResponse response = client.search(request, RequestOptions.DEFAULT);// 4.輸出返回結(jié)果Range agg = response.getAggregations().get("agg");for (Range.Bucket bucket : agg.getBuckets()) {String key = bucket.getKeyAsString();Object from = bucket.getFrom();Object to = bucket.getTo();long docCount = bucket.getDocCount();System.out.println(String.format("key: %s ,from: %s ,to: %s ,docCount: %s",key,from,to,docCount));}}

6.9.3 統(tǒng)計聚合

他可以幫你查詢指定field 的最大值，最小值，平均值，平方和... 使用 extended_stats #統(tǒng)計聚合查詢 extended_stats POST /sms-logs-index/sms-logs-type/_search {"aggs": {"agg": {"extended_stats": {"field": "fee"}}} } // java實現(xiàn) public void aggExtendedStats() throws IOException {// 1.創(chuàng)建requestSearchRequest request = new SearchRequest(index);request.types(type);// 2. 指定使用聚合查詢方式SearchSourceBuilder builder = new SearchSourceBuilder();builder.aggregation(AggregationBuilders.extendedStats("agg").field("fee"));request.source(builder);// 3.執(zhí)行查詢SearchResponse response = client.search(request, RequestOptions.DEFAULT);// 4.輸出返回結(jié)果ExtendedStats extendedStats = response.getAggregations().get("agg");System.out.println("最大值："+extendedStats.getMaxAsString()+",最小值："+extendedStats.getMinAsString());}

6.9.4 其他聚合查詢查看官方文檔

https://www.elastic.co/guide/en/elasticsearch/reference/6.8/search-aggregations-metrics-weight-avg-aggregation.html

6.10 地圖經(jīng)緯度搜索

#創(chuàng)建一個經(jīng)緯度索引,指定一個 name ,一個location PUT /map {"settings": {"number_of_shards": 5,"number_of_replicas": 1},"mappings": {"map":{"properties":{"name":{"type":"text"},"location":{"type":"geo_point"}}}} }#添加測試數(shù)據(jù) PUT /map/map/1 {"name":"天安門","location":{"lon": 116.403694,"lat":39.914492} }PUT /map/map/2 {"name":"百望山","location":{"lon": 116.26284,"lat":40.036576} }PUT /map/map/3 {"name":"北京動物園","location":{"lon": 116.347352,"lat":39.947468} }

6.10.1 ES 的地圖檢索方式

geo_distance :直線距離檢索方式 geo_bounding_box: 以2個點確定一個矩形，獲取再矩形內(nèi)的數(shù)據(jù) geo_polygon:以多個點，確定一個多邊形，獲取多邊形的全部數(shù)據(jù)

6.10.2 基于RESTFul 實現(xiàn)地圖檢索

geo_distance

#geo_distance POST /map/map/_search {"query": {"geo_distance":{#確定一個點"location":{"lon":116.434739,"lat":39.909843},#確定半徑"distance":20000,#指定形狀為圓形"distance_type":"arc"}} } #geo_bounding_box POST /map/map/_search {"query":{"geo_bounding_box":{"location":{"top_left":{"lon":116.327805,"lat":39.95499},"bottom_right":{"lon": 116.363162,"lat":39.938395}}}} } #geo_polygon POST /map/map/_search {"query":{"geo_polygon":{"location":{# 指定多個點確定位置"points":[{"lon":116.220296,"lat":40.075013},{"lon":116.346777,"lat":40.044751},{"lon":116.236106,"lat":39.981533} ]}}} }

6.10.3 java 實現(xiàn) geo_polygon

public class GeoDemo {RestHighLevelClient client = EsClient.getClient();String index = "map";String type="map";@Testpublic void GeoPolygon() throws IOException {// 1.創(chuàng)建searchRequestSearchRequest request = new SearchRequest(index);request.types(type);// 2.指定檢索方式SearchSourceBuilder builder = new SearchSourceBuilder();List<GeoPoint> points = new ArrayList<>();points.add(new GeoPoint(40.075013,116.220296));points.add(new GeoPoint(40.044751,116.346777));points.add(new GeoPoint(39.981533,116.236106));builder.query(QueryBuilders.geoPolygonQuery("location",points));request.source(builder);// 3.執(zhí)行SearchResponse response = client.search(request, RequestOptions.DEFAULT);// 4.輸出結(jié)果for (SearchHit hit : response.getHits().getHits()) {System.out.println(hit.getSourceAsMap());}} }

總結(jié)

以上是生活随笔為你收集整理的ES笔记的全部內(nèi)容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網(wǎng)站內(nèi)容還不錯，歡迎將生活随笔推薦給好友。

笔记
es

上一篇：关于pipe的使用
下一篇： 16进制转10进制c 语言算法,16进制