一、分詞器

Elasticsearch中，內置了很多分詞器（analyzers），例如standard（標準分詞器）、english（英文分詞）和chinese（中文分詞），默認是standard.

standard tokenizer：以單詞邊界進行切分
standard token filter：什么都不做
lowercase token filter：將所有字母轉換為小寫
stop token filer（默認被禁用）：移除停用詞，比如a the it等等

?

二、修改分詞器設置

啟用english，停用詞token filter

PUT /my_index {"settings": {"analysis": {"analyzer": {"es_std":{"type":"standard","stopwords":"_english_"}}}} }

三、標準分詞測試代碼

GET /my_index/_analyze {"analyzer": "standard","text":"a dog is in the house" }

結果：

{"tokens": [{"token": "a","start_offset": 0,"end_offset": 1,"type": "<ALPHANUM>","position": 0},{"token": "dog","start_offset": 2,"end_offset": 5,"type": "<ALPHANUM>","position": 1},{"token": "is","start_offset": 6,"end_offset": 8,"type": "<ALPHANUM>","position": 2},{"token": "in","start_offset": 9,"end_offset": 11,"type": "<ALPHANUM>","position": 3},{"token": "the","start_offset": 12,"end_offset": 15,"type": "<ALPHANUM>","position": 4},{"token": "house","start_offset": 16,"end_offset": 21,"type": "<ALPHANUM>","position": 5}] }

四、設置的英文分詞測試代碼

GET /my_index/_analyze {"analyzer": "es_std","text":"a dog is in the house"}

結果：

{"tokens": [{"token": "dog","start_offset": 2,"end_offset": 5,"type": "<ALPHANUM>","position": 1},{"token": "house","start_offset": 16,"end_offset": 21,"type": "<ALPHANUM>","position": 5}] }

五、自定義分詞器

PUT /my_index {"settings": {"analysis": {"char_filter": {"&_to_and": {"type": "mapping","mappings": ["&=> and"]}},"filter": {"my_stopwords": {"type": "stop","stopwords": ["the", "a"]}},"analyzer": {"my_analyzer": {"type": "custom","char_filter": ["html_strip", "&_to_and"],"tokenizer": "standard","filter": ["lowercase", "my_stopwords"]}}}} }

測試：

GET /my_index/_analyze {"text": "tom&jerry are a friend in the house, <a>, HAHA!!","analyzer": "my_analyzer" }

結果：

{"tokens": [{"token": "tomandjerry","start_offset": 0,"end_offset": 9,"type": "<ALPHANUM>","position": 0},{"token": "are","start_offset": 10,"end_offset": 13,"type": "<ALPHANUM>","position": 1},{"token": "friend","start_offset": 16,"end_offset": 22,"type": "<ALPHANUM>","position": 3},{"token": "in","start_offset": 23,"end_offset": 25,"type": "<ALPHANUM>","position": 4},{"token": "house","start_offset": 30,"end_offset": 35,"type": "<ALPHANUM>","position": 6},{"token": "haha","start_offset": 42,"end_offset": 46,"type": "<ALPHANUM>","position": 7}] }

六、type中的使用

PUT /my_index/_mapping/my_type {"properties": {"content": {"type": "text","analyzer": "my_analyzer"}} }

?

轉載于:https://www.cnblogs.com/yfb918/p/10718712.html

創作挑戰賽新人創作獎勵來咯，堅持創作打卡瓜分現金大獎

總結

以上是生活随笔為你收集整理的ElasticSearch教程——自定义分词器（转学习使用）的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。