elasticsearch 使用
目錄
RestAPI 簡介
1.前提工作? --? 創建索引庫
2.RestClient的簡單使用
創建索引庫
刪除索引庫
索引庫是否存在
創建文檔
刪除文檔
修改文檔
批量操作文檔
match_all(全部查詢)
match查詢
term(精準查詢)
range(范圍查詢)
geo(坐標查詢)
functionScore(算分函數)
bool(布爾查詢-多條件)
排序、分頁
高亮
聚合
Bucket聚合(分組)
Metric聚合函數(min、max、avg)
詞條(漢字 & 拼音)自動補全
SpringBoot 整合 EleasticSearch 8.0.X 棄用high-level-client
RestAPI 簡介
ES官方提供了各種不同語言的客戶端,用來操作ES。這些客戶端的本質就是組裝DSL語句,通過http請求發送給ES。官方文檔地址:Elasticsearch Clients | Elastic
1.前提工作? --? 創建索引庫
我們根據一個mysql的數據庫表來創建對應的索引庫
CREATE?TABLE?`tb_hotel`?(`id`?bigint(20)?NOT?NULL?COMMENT?'酒店id',`name`?varchar(255)?NOT?NULL?COMMENT?'酒店名稱;例:7天酒店',`address`?varchar(255)?NOT?NULL?COMMENT?'酒店地址;例:航頭路',`price`?int(10)?NOT?NULL?COMMENT?'酒店價格;例:329',`score`?int(2)?NOT?NULL?COMMENT?'酒店評分;例:45,就是4.5分',`brand`?varchar(32)?NOT?NULL?COMMENT?'酒店品牌;例:如家',`city`?varchar(32)?NOT?NULL?COMMENT?'所在城市;例:上海',`star_name`?varchar(16)?DEFAULT?NULL?COMMENT?'酒店星級,從低到高分別是:1星到5星,1鉆到5鉆',`business`?varchar(255)?DEFAULT?NULL?COMMENT?'商圈;例:虹橋',`latitude`?varchar(32)?NOT?NULL?COMMENT?'緯度;例:31.2497',`longitude`?varchar(32)?NOT?NULL?COMMENT?'經度;例:120.3925',`pic`?varchar(255)?DEFAULT?NULL?COMMENT?'酒店圖片;例:/img/1.jpg',PRIMARY?KEY?(`id`) )?ENGINE=InnoDB?DEFAULT?CHARSET=utf8mb4;創建的索引庫如下:
PUT /hotel {"mappings": {"properties": {"id": {"type": "keyword"},"name":{"type": "text","analyzer": "ik_max_word","copy_to": "all"},"address":{"type": "keyword","index": false},"price":{"type": "integer"},"score":{"type": "integer"},"brand":{"type": "keyword","copy_to": "all"},"city":{"type": "keyword","copy_to": "all"},"starName":{"type": "keyword"},"business":{"type": "keyword"},"location":{"type": "geo_point"},"pic":{"type": "keyword","index": false},"all":{"type": "text","analyzer": "ik_max_word"}}} }幾個特殊字段說明:
-
location:地理坐標,里面包含精度、緯度
-
all:一個組合字段,其目的是將多字段的值 利用copy_to合并,提供給用戶搜索
地理坐標說明:
?copy_to說明:
2.RestClient的簡單使用
在elasticsearch提供的API中,與elasticsearch一切交互都封裝在一個名為RestHighLevelClient的類中,必須先完成這個對象的初始化,建立與elasticsearch的連接。
分為三步:
1)引入es的RestHighLevelClient依賴:
<dependency><groupId>org.elasticsearch.client</groupId><artifactId>elasticsearch-rest-high-level-client</artifactId> </dependency>2)因為SpringBoot默認的ES版是7.6.2,所以我們需要覆蓋默認的ES版本:
<properties><java.version>1.8</java.version><elasticsearch.version>7.12.1</elasticsearch.version> </properties>3)初始化RestHighLevelClient:
初始化的代碼如下:
RestHighLevelClient client = new RestHighLevelClient(RestClient.builder(HttpHost.create("http://192.168.150.101:9200") ));創建索引庫
代碼分為三步:
-
1)創建Request對象。因為是創建索引庫的操作,因此Request是CreateIndexRequest。
-
2)添加請求參數,其實就是DSL的JSON參數部分。因為json字符串很長,這里是定義了靜態字符串常量MAPPING_TEMPLATE,讓代碼看起來更加優雅。
-
3)發送請求,client.indices()方法的返回值是IndicesClient類型,封裝了所有與索引庫操作有關的方法。
code:
@Testvoid createHotelIndex() throws IOException {CreateIndexRequest request = new CreateIndexRequest("hotel");request.source(MAPPING_TEMPLATE, XContentType.JSON);client.indices().create(request, RequestOptions.DEFAULT);}public class HotelConstants {public static final String MAPPING_TEMPLATE = "{\n" +" \"mappings\": {\n" +" \"properties\": {\n" +" \"id\":{\n" +" \"type\": \"keyword\"\n" +" },\n" +" \"name\":{\n" +" \"type\": \"text\",\n" +" \"analyzer\": \"ik_max_word\",\n" +" \"copy_to\": \"all\"\n" +" },\n" +" \"address\":{\n" +" \"type\": \"keyword\",\n" +" \"index\": false\n" +" },\n" +" \"price\":{\n" +" \"type\": \"integer\"\n" +" },\n" +" \"score\":{\n" +" \"type\": \"integer\"\n" +" },\n" +" \"brand\":{\n" +" \"type\": \"keyword\",\n" +" \"copy_to\": \"all\"\n" +" },\n" +" \"city\":{\n" +" \"type\": \"keyword\",\n" +" \"copy_to\": \"all\"\n" +" },\n" +" \"starName\":{\n" +" \"type\": \"keyword\"\n" +" },\n" +" \"business\":{\n" +" \"type\": \"keyword\"\n" +" },\n" +" \"location\":{\n" +" \"type\": \"geo_point\"\n" +" },\n" +" \"pic\":{\n" +" \"type\": \"keyword\",\n" +" \"index\": false\n" +" },\n" +" \"all\":{\n" +" \"type\": \"text\",\n" +" \"analyzer\": \"ik_max_word\"\n" +" }\n" +" }\n" +" }\n" +"}"; }刪除索引庫
刪除索引庫的DSL語句非常簡單
DELETE /hotel @Test void testDeleteHotelIndex() throws IOException {// 1.創建Request對象DeleteIndexRequest request = new DeleteIndexRequest("hotel");// 2.發送請求client.indices().delete(request, RequestOptions.DEFAULT); }索引庫是否存在
判斷索引庫是否存在,本質就是查詢,對應的DSL是:
GET /hotel @Test void testExistsHotelIndex() throws IOException {// 1.創建Request對象GetIndexRequest request = new GetIndexRequest("hotel");// 2.發送請求boolean exists = client.indices().exists(request, RequestOptions.DEFAULT);// 3.輸出System.err.println(exists ? "索引庫已經存在!" : "索引庫不存在!"); }創建文檔
創建文檔的DSL語句如下:
POST /{索引庫名}/_doc/1 {"name": "Jack","age": 21 }?code:
@Data @NoArgsConstructor public class HotelDoc {private Long id;private String name;private String address;private Integer price;private Integer score;private String brand;private String city;private String starName;private String business;private String location;private String pic;public HotelDoc(Hotel hotel) {this.id = hotel.getId();this.name = hotel.getName();this.address = hotel.getAddress();this.price = hotel.getPrice();this.score = hotel.getScore();this.brand = hotel.getBrand();this.city = hotel.getCity();this.starName = hotel.getStarName();this.business = hotel.getBusiness();this.location = hotel.getLatitude() + ", " + hotel.getLongitude();this.pic = hotel.getPic();} } @Testvoid addHotelDoc() throws IOException {Hotel hotel = hotelService.getById(36934L);HotelDoc doc= new HotelDoc(hotel);String json = JSON.toJSONString(doc);IndexRequest request = new IndexRequest("hotel").id(hotel.getId().toString());request.source(json, XContentType.JSON);client.index(request, RequestOptions.DEFAULT);}Hotel在數據庫中經緯度字段是分開的,es中經緯度是一個字段location,geo_point,所以用HotelDoc和es的屬性做對應
刪除文檔
刪除的DSL為是這樣的:
DELETE /hotel/_doc/{id} @Test void testDeleteDocument() throws IOException {// 1.準備RequestDeleteRequest request = new DeleteRequest("hotel", "61083");// 2.發送請求client.delete(request, RequestOptions.DEFAULT); }修改文檔
修改我們講過兩種方式:
-
全量修改:本質是先根據id刪除,再新增
-
增量修改:修改文檔中的指定字段值
在RestClient的API中,全量修改與新增的API完全一致,判斷依據是ID:
-
如果新增時,ID已經存在,則修改
-
如果新增時,ID不存在,則新增
這里是增量修改
@Testvoid updateHotelDoc() throws IOException {UpdateRequest request = new UpdateRequest("hotel","36934");request.doc("price", 333,"starName","三鉆");client.update(request, RequestOptions.DEFAULT);}批量操作文檔
批量處理BulkRequest,其本質就是將多個普通的CRUD請求組合在一起發送。
其中提供了一個add方法,用來添加其他請求:
可以看到,能添加的請求包括:
-
IndexRequest,也就是新增
-
UpdateRequest,也就是修改
-
DeleteRequest,也就是刪除
因此Bulk中添加了多個IndexRequest,就是批量新增功能了。:
@Testvoid bulkHotelDoc() throws IOException {List<Hotel> list = hotelService.list();BulkRequest request = new BulkRequest();for (Hotel hotel : list) {HotelDoc doc = new HotelDoc(hotel);request.add(new IndexRequest("hotel").id(doc.getId().toString()).source(JSON.toJSONString(doc), XContentType.JSON));}client.bulk(request, RequestOptions.DEFAULT);}match_all(全部查詢)
@Test void testMatchAll() throws IOException {// 1.準備RequestSearchRequest request = new SearchRequest("hotel");// 2.準備DSLrequest.source().query(QueryBuilders.matchAllQuery());// 3.發送請求SearchResponse response = client.search(request, RequestOptions.DEFAULT);// 4.解析響應handleResponse(response); }private void handleResponse(SearchResponse response) {// 4.解析響應SearchHits searchHits = response.getHits();// 4.1.獲取總條數long total = searchHits.getTotalHits().value;System.out.println("共搜索到" + total + "條數據");// 4.2.文檔數組SearchHit[] hits = searchHits.getHits();// 4.3.遍歷for (SearchHit hit : hits) {// 獲取文檔sourceString json = hit.getSourceAsString();// 反序列化HotelDoc hotelDoc = JSON.parseObject(json, HotelDoc.class);System.out.println("hotelDoc = " + hotelDoc);} }match查詢
@Testvoid searchMatch() throws IOException {SearchRequest request = new SearchRequest("hotel");request.source().query(QueryBuilders.matchQuery("all", "如家"));SearchResponse searchResponse = client.search(request, RequestOptions.DEFAULT);printSearchResponse(searchResponse);} @Testvoid searchMatchMulti() throws IOException {SearchRequest request = new SearchRequest("hotel");request.source().query(QueryBuilders.multiMatchQuery("外灘", "business", "name"));SearchResponse searchResponse = client.search(request, RequestOptions.DEFAULT);printSearchResponse(searchResponse);}term(精準查詢)
-
term:詞條精確匹配
range(范圍查詢)
@Testvoid searchRange() throws IOException {SearchRequest request = new SearchRequest("hotel");request.source().query(QueryBuilders.rangeQuery("price").lte(200));SearchResponse searchResponse = client.search(request, RequestOptions.DEFAULT);printSearchResponse(searchResponse);}geo(坐標查詢)
@Testvoid searchGeo() throws IOException {SearchRequest request = new SearchRequest("hotel");request.source().query(QueryBuilders.geoDistanceQuery("location").point(31.21, 121.5).distance(15, DistanceUnit.KILOMETERS));SearchResponse searchResponse = client.search(request, RequestOptions.DEFAULT);printSearchResponse(searchResponse);}functionScore(算分函數)
@Testvoid searchFunctionScore() throws IOException {SearchRequest request = new SearchRequest("hotel");QueryBuilder queryBuilder = QueryBuilders.matchQuery("all", "外灘");FunctionScoreQueryBuilder.FilterFunctionBuilder[] filterFunctionBuilders = new FunctionScoreQueryBuilder.FilterFunctionBuilder[1];ScoreFunctionBuilder<WeightBuilder> scoreFunctionBuilder = new WeightBuilder();scoreFunctionBuilder.setWeight(10);QueryBuilder termQuery = QueryBuilders.termQuery("brand", "如家");FunctionScoreQueryBuilder.FilterFunctionBuilder category = new FunctionScoreQueryBuilder.FilterFunctionBuilder(termQuery, scoreFunctionBuilder);filterFunctionBuilders[0] = category;request.source().query(QueryBuilders.functionScoreQuery(queryBuilder,filterFunctionBuilders) // .scoreMode(FunctionScoreQuery.ScoreMode.SUM).boostMode(CombineFunction.SUM));SearchResponse searchResponse = client.search(request, RequestOptions.DEFAULT);printSearchResponse(searchResponse);}或者:
?bool 查詢 搭配 算分函數
// 1.構建BooleanQueryBoolQueryBuilder boolQuery = QueryBuilders.boolQuery();// 關鍵字搜索String key = params.getKey();if (key == null || "".equals(key)) {boolQuery.must(QueryBuilders.matchAllQuery());} else {boolQuery.must(QueryBuilders.matchQuery("all", key));}// 城市條件if (params.getCity() != null && !params.getCity().equals("")) {boolQuery.filter(QueryBuilders.termQuery("city", params.getCity()));}// 品牌條件if (params.getBrand() != null && !params.getBrand().equals("")) {boolQuery.filter(QueryBuilders.termQuery("brand", params.getBrand()));}// 星級條件if (params.getStarName() != null && !params.getStarName().equals("")) {boolQuery.filter(QueryBuilders.termQuery("starName", params.getStarName()));}// 價格if (params.getMinPrice() != null && params.getMaxPrice() != null) {boolQuery.filter(QueryBuilders.rangeQuery("price").gte(params.getMinPrice()).lte(params.getMaxPrice()));}// 2.算分控制FunctionScoreQueryBuilder functionScoreQuery =QueryBuilders.functionScoreQuery(// 原始查詢,相關性算分的查詢boolQuery,// function score的數組new FunctionScoreQueryBuilder.FilterFunctionBuilder[]{// 其中的一個function score 元素new FunctionScoreQueryBuilder.FilterFunctionBuilder(// 過濾條件QueryBuilders.termQuery("isAD", true),// 算分函數ScoreFunctionBuilders.weightFactorFunction(10)).boostMode(CombineFunction.SUM)});request.source().query(functionScoreQuery);bool(布爾查詢-多條件)
布爾查詢是用must、must_not、filter等方式組合其它查詢,代碼示例如下:
?可以看到,API與其它查詢的差別同樣是在查詢條件的構建,QueryBuilders,結果解析等其他代碼完全不變。
@Testvoid searchBool() throws IOException {SearchRequest request = new SearchRequest("hotel");QueryBuilder mustBuilder = QueryBuilders.matchQuery("name", "如家");QueryBuilder mustNotBuilder = QueryBuilders.rangeQuery("price").gt(400);QueryBuilder geoBuilder = QueryBuilders.geoDistanceQuery("location").point(31.21, 121.5).distance(10, DistanceUnit.KILOMETERS);request.source().query(QueryBuilders.boolQuery().must(mustBuilder).mustNot(mustNotBuilder).filter(geoBuilder));SearchResponse searchResponse = client.search(request, RequestOptions.DEFAULT);printSearchResponse(searchResponse);}排序、分頁
搜索結果的排序和分頁是與query同級的參數,因此同樣是使用request.source()來設置。
@Test void testPageAndSort() throws IOException {// 頁碼,每頁大小int page = 1, size = 5;// 1.準備RequestSearchRequest request = new SearchRequest("hotel");// 2.準備DSL// 2.1.queryrequest.source().query(QueryBuilders.matchAllQuery());// 2.2.排序 sortrequest.source().sort("price", SortOrder.ASC);// 2.3.分頁 from、sizerequest.source().from((page - 1) * size).size(5);// 3.發送請求SearchResponse response = client.search(request, RequestOptions.DEFAULT);// 4.解析響應handleResponse(response);}?距離排序
request.source().sort(SortBuilders.geoDistanceSort("location", new GeoPoint(location)).order(SortOrder.ASC).unit(DistanceUnit.KILOMETERS));高亮
代碼解讀:
-
第一步:從結果中獲取source。hit.getSourceAsString(),這部分是非高亮結果,json字符串。還需要反序列為HotelDoc對象
-
第二步:獲取高亮結果。hit.getHighlightFields(),返回值是一個Map,key是高亮字段名稱,值是HighlightField對象,代表高亮值
-
第三步:從map中根據高亮字段名稱,獲取高亮字段值對象HighlightField
-
第四步:從HighlightField中獲取Fragments,并且轉為字符串。這部分就是真正的高亮字符串了
-
第五步:用高亮的結果替換HotelDoc中的非高亮結果
聚合
Bucket聚合(分組)
void searchAggs() throws IOException {SearchRequest request = new SearchRequest("hotel");//不返回文檔數據request.source().size(0);request.source().aggregation(AggregationBuilders.terms("brandAgg").field("brand").size(20).order(BucketOrder.aggregation("_count",true)));SearchResponse searchResponse = client.search(request, RequestOptions.DEFAULT);Aggregations aggregations = searchResponse.getAggregations();Terms brandTerms = aggregations.get("brandAgg");List<? extends Terms.Bucket> buckets = brandTerms.getBuckets();for (Terms.Bucket bucket : buckets) {System.out.println(bucket.getKeyAsString() + " : " + bucket.getDocCount());} }?獲取多個聚合結果:
GET /hotel/_search {"size": 0,"aggs": {"brandAgg": {"terms": {"field": "brand","size": 2}}, "cityAgg": {"terms": {"field": "city","size": 2}}} } request.source().aggregation(AggregationBuilders.terms("brandAgg").field("brand").size(2));request.source().aggregation(AggregationBuilders.terms("cityAgg").field("city").size(2));Metric聚合函數(min、max、avg)
現在我們需要對桶內的酒店做運算,獲取每個品牌的用戶評分的min、max、avg等值。
SearchRequest request = new SearchRequest("hotel");request.source().size(0);request.source().aggregation(AggregationBuilders.terms("brandAgg").field("brand").size(20).order(BucketOrder.aggregation("score_stats.avg",false)).subAggregation(AggregationBuilders.stats("score_stats").field("score")));SearchResponse searchResponse = client.search(request, RequestOptions.DEFAULT);結果:
@Testvoid searchAggsMetric() throws IOException {SearchRequest request = new SearchRequest("hotel");request.source().size(0);request.source().aggregation(AggregationBuilders.terms("brandAgg").field("brand").size(20).order(BucketOrder.aggregation("score_stats.avg",false)).subAggregation(AggregationBuilders.stats("score_stats").field("score")));SearchResponse searchResponse = client.search(request, RequestOptions.DEFAULT);Aggregations aggregations = searchResponse.getAggregations();Terms brandTerms = aggregations.get("brandAgg");List<? extends Terms.Bucket> buckets = brandTerms.getBuckets();for (Terms.Bucket bucket : buckets) {Aggregations bucketAggs = bucket.getAggregations();Stats stats = bucketAggs.get("score_stats");System.out.println(bucket.getKeyAsString() +" : " + JSON.toJSONString(stats));}}詞條(漢字 & 拼音)自動補全
目標根據酒店品牌 和 商圈的關鍵字,或者拼音,補全關聯到的內容。
此處以(keyword)精準詞語為例
1. 支持拼音關聯搜索,則需要安裝拼音分詞器。
拼音分詞器的安裝請參考:elasticsearch 安裝(docker)_naki_bb的博客-CSDN博客
不需要拼音補全的課參考?elasticsearch DSL命令_naki_bb的博客-CSDN博客的 自動補全
2.詞條補全必須要有一個字段類型為suggestion。
索引庫DSL語句如下
// 酒店數據索引庫 PUT /hotel {"settings": {"analysis": {"analyzer": {"text_anlyzer": {"tokenizer": "ik_max_word","filter": "py"},"completion_analyzer": {"tokenizer": "keyword","filter": "py"}},"filter": {"py": {"type": "pinyin","keep_full_pinyin": false,"keep_joined_full_pinyin": true,"keep_original": true,"limit_first_letter_length": 16,"remove_duplicated_term": true,"none_chinese_pinyin_tokenize": false}}}},"mappings": {"properties": {"id":{"type": "keyword"},"name":{"type": "text","analyzer": "text_anlyzer","search_analyzer": "ik_smart","copy_to": "all"},"address":{"type": "keyword","index": false},"price":{"type": "integer"},"score":{"type": "integer"},"brand":{"type": "keyword","copy_to": "all"},"city":{"type": "keyword"},"starName":{"type": "keyword"},"business":{"type": "keyword","copy_to": "all"},"location":{"type": "geo_point"},"pic":{"type": "keyword","index": false},"all":{"type": "text","analyzer": "text_anlyzer","search_analyzer": "ik_smart"},"suggestion":{"type": "completion","analyzer": "completion_analyzer"}}} }實體類:
HotelDoc中要添加一個字段,用來做自動補全,內容可以是酒店品牌、商圈等信息。按照自動補全字段的要求,最好是這些字段的數組。
import lombok.Data; import lombok.NoArgsConstructor;import java.util.ArrayList; import java.util.Arrays; import java.util.Collections; import java.util.List;@Data @NoArgsConstructor public class HotelDoc {private Long id;private String name;private String address;private Integer price;private Integer score;private String brand;private String city;private String starName;private String business;private String location;private String pic;private Object distance;private Boolean isAD;private List<String> suggestion;public HotelDoc(Hotel hotel) {this.id = hotel.getId();this.name = hotel.getName();this.address = hotel.getAddress();this.price = hotel.getPrice();this.score = hotel.getScore();this.brand = hotel.getBrand();this.city = hotel.getCity();this.starName = hotel.getStarName();this.business = hotel.getBusiness();this.location = hotel.getLatitude() + ", " + hotel.getLongitude();this.pic = hotel.getPic();// 組裝suggestionif(this.business.contains("/")){// business有多個值,需要切割String[] arr = this.business.split("/");// 添加元素this.suggestion = new ArrayList<>();this.suggestion.add(this.brand);Collections.addAll(this.suggestion, arr);}else {this.suggestion = Arrays.asList(this.brand, this.business);}} }重新導入數據功能,可以看到新的酒店數據中包含了suggestion:
?API 示例
?結果解析示例:
?案例的實現如下:
@Override public List<String> getSuggestions(String prefix) {try {// 1.準備RequestSearchRequest request = new SearchRequest("hotel");// 2.準備DSLrequest.source().suggest(new SuggestBuilder().addSuggestion("suggestions",SuggestBuilders.completionSuggestion("suggestion").prefix(prefix).skipDuplicates(true).size(10)));// 3.發起請求SearchResponse response = client.search(request, RequestOptions.DEFAULT);// 4.解析結果Suggest suggest = response.getSuggest();// 4.1.根據補全查詢名稱,獲取補全結果CompletionSuggestion suggestions = suggest.getSuggestion("suggestions");// 4.2.獲取optionsList<CompletionSuggestion.Entry.Option> options = suggestions.getOptions();// 4.3.遍歷List<String> list = new ArrayList<>(options.size());for (CompletionSuggestion.Entry.Option option : options) {String text = option.getText().toString();list.add(text);}return list;} catch (IOException e) {throw new RuntimeException(e);} }SpringBoot 整合 EleasticSearch 8.0.X 棄用high-level-client
請參考
springboot集成elasticsearch 8.0以上版本_weixin_46501729的博客-CSDN博客
總結
以上是生活随笔為你收集整理的elasticsearch 使用的全部內容,希望文章能夠幫你解決所遇到的問題。
- 上一篇: win8系统中chm文件打不开怎么办
- 下一篇: Openbravo ERP介绍