當前位置：首頁 > 编程资源 > 编程问答 >内容正文

编程问答

es elasticsearch 几种常见查询场景二次分组 java读取es的查询json文件

發布時間：2023/12/10 编程问答 24 豆豆

生活随笔收集整理的這篇文章主要介紹了 es elasticsearch 几种常见查询场景二次分组 java读取es的查询json文件小編覺得挺不錯的,現在分享給大家,幫大家做個參考.

大家好，我是烤鴨：

es中幾種常見的查詢場景,使用java讀取es的json文件進行查詢。

es 中文使用手冊。
https://www.elastic.co/guide/cn/elasticsearch/guide/current/foreword_id.html

1. ?從最簡單的查詢開始

GET /_search

{"hits" : {"total" : 14,"hits" : [{"_index": "us","_type": "tweet","_id": "7","_score": 1,"_source": {"date": "2014-09-17","name": "John Smith","tweet": "The Query DSL is really powerful and flexible","user_id": 2}},... 9 RESULTS REMOVED ...],"max_score" : 1},"took" : 4,"_shards" : {"failed" : 0,"successful" : 10,"total" : 10},"timed_out" : false }

從官方給的例子看下這些參數的含義。
hits 結果集
?? ?- _index document的屬性
?? ?- type document的屬性
?? ?- _id document的屬性
?? ?- _score 對搜索的匹配性(越匹配,得分越高)
?? ?- _source 就是存進去的value
took 請求花費時間
timeout 超時時間(默認不超時,可以指定 GET /_search?timeout=10ms)
shards 有多少個響應的分片,多少成功/失敗
之所以hits 要多返回幾個document的屬性，避免需要的時候再去查詢。

2. ?稍微麻煩的查詢

sql 一般都會寫，介紹一個神器
神器：?? ?Sql 轉 Es 查詢語句?
http://www.ischoolbar.com/EsParser/

舉個例子
sql：

select name,number from sys_user order by create_time desc limit 10

es query：

{"from":0,"size":"10","sort":[{"create_time":{"order":"DESC"}}],"_source":{"include":["name","number"]}}

如圖所示：

上面的例子還是太簡單了，如果想做分組后求和呢。
如圖所示：

3. ?es數據展示

這次模擬的是二次分組的數據情況,某個時間段的按分鐘分組,獲取有多少分鐘,再獲取這一分鐘內數據的分組情況(每分鐘有多少數據)。
es sql ：uid_aggs 代表第一次分組的字段,keyword_aggs?是第二次分組后的詳情數據

{"aggs": {"all": {"terms": {"field": "currentMinutes.keyword","size": 100}},"keyword_aggs": {"aggs": {"top": {"top_hits": {"size": 100}}},"terms": {"field": "currentMinutes.keyword","size": 100}},"uid_aggs": {"cardinality": {"precision_threshold": 120,"field": "currentMinutes.keyword"}}},"query": {"bool": {"must": [{"range": {"currentTime.keyword": {"gte": "2020-08-05 14:43","lte": "2020-08-05 14:45"}}}],"must_not": [],"should": []}},"sort": [{"currentTime.keyword": {"order": "desc"}}] }

結果：
我把返回的詳細數據hits刪掉了，主要看一下 aggregations 里邊的內容。
返回的參數說明：
uid_aggs 就是請求查詢傳入的參數,這里返回的是,按分鐘分組的數量。
all的buckets 的key為二次分組字段,doc_count?為二次分組對應的數據量。
keyword_aggs的 buckets 展示了all的buckets 對應的數據詳情。

{"took": 5,"timed_out": false,"_shards": {"total": 5,"successful": 5,"skipped": 0,"failed": 0},"hits": {"total": 5,"max_score": null,"hits": [{// 數據暫時刪掉了...}]},"aggregations": {"all": {"doc_count_error_upper_bound": 0,"sum_other_doc_count": 0,"buckets": [{"key": "2020-08-05 14:44","doc_count": 3},{"key": "2020-08-05 14:43","doc_count": 2}]},"uid_aggs": {"value": 2},"keyword_aggs": {"doc_count_error_upper_bound": 0,"sum_other_doc_count": 0,"buckets": [{"key": "2020-08-05 14:44","doc_count": 3,"top": {"hits": {"total": 3,"max_score": 1.0,"hits": [{"_index": "apm-report-app2020-08-05","_type": "info","_id": "7cpdvXMBjkzeFkDgwB3t","_score": 1.0,"_source": {"dvid": "D3B6D5FBC668B731E52BC08D9FE23EAB","showName": "車系列表","appVer": "10.36.0","groupId": 3,"weight": 1,"pageId": 30001,"apiTime": 541.3210391998291,"platform": 2,"currentTime": "2020-08-05 14:44:53.292","loadTime": 584.99908447265625,"@timestamp": "2020-08-05 14:44:53","currentMinutes": "2020-08-05 14:44","viewTime": 43.678998947143555,"requestId": "BD43986C-7995-4F93-84F7-B4D65E8BE5EB","cName": "BPCSerialListRootVC"}},{"_index": "apm-report-app2020-08-05","_type": "info","_id": "7spdvXMBjkzeFkDgxh2B","_score": 1.0,"_source": {"dvid": "D3B6D5FBC668B731E52BC08D9FE23EAB","showName": "車型綜述頁","appVer": "10.36.0","groupId": 3,"weight": 1,"pageId": 30007,"apiTime": 456.7570686340332,"platform": 2,"currentTime": "2020-08-05 14:44:54.721","loadTime": 720.36707401275635,"@timestamp": "2020-08-05 14:44:54","currentMinutes": "2020-08-05 14:44","viewTime": 263.61000537872314,"requestId": "DCE5F4A4-40AC-4EB4-BCCF-ABACCFC7FF01","cName": "BPCOverPagesViewController"}},{"_index": "apm-report-app2020-08-05","_type": "info","_id": "78pdvXMBjkzeFkDgyB1A","_score": 1.0,"_source": {"dvid": "D3B6D5FBC668B731E52BC08D9FE23EAB","showName": "車型-圖片列表","appVer": "10.36.0","groupId": 3,"weight": 1,"pageId": 30015,"apiTime": 129.27305698394775,"platform": 2,"currentTime": "2020-08-05 14:44:55.168","loadTime": 150.16889572143555,"@timestamp": "2020-08-05 14:44:55","currentMinutes": "2020-08-05 14:44","viewTime": 20.89691162109375,"requestId": "A1DA2749-A56B-4AFE-BF83-ECDF82806152","cName": "BPImageVideoViewController"}}]}}},{"key": "2020-08-05 14:43","doc_count": 2,"top": {"hits": {"total": 2,"max_score": 1.0,"hits": [{"_index": "apm-report-app2020-08-05","_type": "info","_id": "7MpcvXMBjkzeFkDg2x17","_score": 1.0,"_source": {"dvid": "D3B6D5FBC668B731E52BC08D9FE23EAB","showName": "車型-圖片列表","appVer": "10.36.0","groupId": 3,"weight": 1,"pageId": 30015,"apiTime": 269.10901069641113,"platform": 2,"currentTime": "2020-08-05 14:43:54.553","loadTime": 334.06901359558105,"@timestamp": "2020-08-05 14:43:54","currentMinutes": "2020-08-05 14:43","viewTime": 64.961075782775879,"requestId": "E5105DA7-5914-41ED-A76E-5DF39CF38588","cName": "BPImageVideoViewController"}},{"_index": "apm-report-app2020-08-05","_type": "info","_id": "68pcvXMBjkzeFkDg1h2t","_score": 1.0,"_source": {"dvid": "D3B6D5FBC668B731E52BC08D9FE23EAB","showName": "車型綜述頁","appVer": "10.36.0","groupId": 3,"weight": 1,"pageId": 30007,"apiTime": 488.01600933074951,"platform": 2,"currentTime": "2020-08-05 14:43:53.324","loadTime": 539.05403614044189,"@timestamp": "2020-08-05 14:43:53","currentMinutes": "2020-08-05 14:43","viewTime": 51.03909969329834,"requestId": "B370AB84-3A6E-4C48-9DF2-4F6E1855567B","cName": "BPCOverPagesViewController"}}]}}}]}} }

4. ?Java代碼轉換

將寫好的es json放到 resouces目錄下，變量用唯一字符表示，等讀取sql的時候再替換。
讀取resources下json的代碼。

/*** 加載查詢* @param path 路徑* @return string*/ private String getQueryJsonStr(String path) {//加載json文件ClassPathResource classPathResource = new ClassPathResource(path);String queryJson=null;try (InputStream in = classPathResource.getInputStream();BufferedReader br = new BufferedReader(new InputStreamReader(in))) {StringBuilder message = new StringBuilder();String line = null;while ((line = br.readLine()) != null) {message.append(line);}queryJson = message.toString();} catch (Exception e) {}return queryJson; }

拼裝替換的map參數,這里以時間為例

/*** es查詢時間* @param beginTime beginTime* @param endTime endTime* @return Map*/private Map<String,String> buildEsQueryMap(String beginTime, String endTime){Map<String,String> params = new HashMap<>();//初始化時間if (StringUtils.isEmpty(beginTime)){beginTime = DateUtil.getCurrDate()+" 00:00:00";}if (StringUtils.isEmpty(endTime)){endTime = DateUtil.getCurrTime();//當前時間}params.put("{beginTime}", beginTime);params.put("{endTime}", endTime);return params;}

遍歷替換json

for (Map.Entry<String, String> entry : params.entrySet()) {queryJson=queryJson.replace(entry.getKey(), entry.getValue()); }

發送http請求查詢,queryJson就是替換后的es查詢json

HttpHeaders headers = new HttpHeaders(); MediaType type = MediaType.parseMediaType("application/json; charset=UTF-8"); headers.setContentType(type); headers.add("Accept", MediaType.APPLICATION_JSON.toString()); HttpEntity<String> formEntity = new HttpEntity<>(queryJson, headers); //查詢 result= restTemplate.postForEntity(url,formEntity,JSONObject.class).getBody();

5. 總結

es的學習成本確實有點高,文檔又多又雜。更多的看官方文檔吧。
英文版：
https://www.elastic.co/guide/en/elasticsearch/reference/current/search-search.html
中文版：（基于 Elasticsearch 2.x 版本）
https://www.elastic.co/guide/cn/elasticsearch/guide/current/query-dsl-intro.html

總結

以上是生活随笔為你收集整理的es elasticsearch 几种常见查询场景二次分组 java读取es的查询json文件的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。