仿牛客論壇項目

• 一、Elasticsearch入門
• • 1.1 elasticsearch安裝
  • 1.2 修改config目錄下的elasticsearch.yml配置文件
  • 1.3 配置環境變量
  • 1.4 下載ik中文分詞器
  • 1.5 下載postman
  • 1.6 啟動es
  • 1.7 測試是否正確啟動
  • 1.8 操作ES的RESTful語法
  • 1.9 es彈性搜索指南
  • 1.10 es中的數據類型
• 二、 Spring整合Elasticsearch
• • 2.1 導入依賴
  • 2.2 修改配置文件
  • 2.3 修改主啟動類
  • 2.4 實體類添加es注解
  • 2.5 數據操作
  • • DiscussPostRepository
  • 2.6 測試
• 三、開發社區搜索功能
• • 3.1 業務層
  • • ElasticSearchService(封裝對es的增刪改查操作)
  • 3.2 控制層
  • • DiscussPostController
    • CommentController
    • SearchController
  • 3.3 新增事件
  • • EventConsumer
  • 3.4 靜態資源
  • • index.html
    • serch.html
  • 3.4 測試

一、Elasticsearch入門

1.1 elasticsearch安裝

https://artifacts.elastic.co/downloads/elasticsearch/elasticsearch-6.4.3.zip

• 下載后直接解壓

1.2 修改config目錄下的elasticsearch.yml配置文件

1.3 配置環境變量

• 把對應的bin目錄配置到環境變量

1.4 下載ik中文分詞器

https://github.com/medcl/elasticsearch-analysis-ik/releases/download/v6.4.3/elasticsearch-analysis-ik-6.4.3.zip

• 下載之后解壓到es安裝目錄下的plugins目錄下的ik(自己建)目錄下
  
• 如果需要引入自己建的字典,則自己建一個dic文件,把需要的新詞加入進去,然后在下面文件引入即可
  
  

1.5 下載postman

https://www.getpostman.com

1.6 啟動es

1.7 測試是否正確啟動

1.8 操作ES的RESTful語法

1.9 es彈性搜索指南

https://www.elastic.co/guide/en/elasticsearch/reference/6.4/index.html

1.10 es中的數據類型

https://www.elastic.co/guide/en/elasticsearch/reference/6.4/mapping-types.html

二、 Spring整合Elasticsearch

2.1 導入依賴

<!--整合elasticsearch--><dependency><groupId>org.springframework.boot</groupId><artifactId>spring-boot-starter-data-elasticsearch</artifactId></dependency>

2.2 修改配置文件

# ElasticsearchProperties # 集群名 spring.data.elasticsearch.cluster-name=nowcoder # tcp端口:9300, http訪問端口:9200 spring.data.elasticsearch.cluster-nodes=127.0.0.1:9300

2.3 修改主啟動類

• 避免es和redis同時依賴netty起沖突

@SpringBootApplication @MapperScan("com.nowcoder.community.mapper") public class CommunityApplication {@PostConstructpublic void init() {// 解決netty啟動沖突問題// see Netty4Utils.setAvailableProcessors()System.setProperty("es.set.netty.runtime.available.processors", "false");}public static void main(String[] args) {SpringApplication.run(CommunityApplication.class, args);}}

2.4 實體類添加es注解

• 在程序啟動時會自動在es中創建索引

@Data //index必須為lowercase不然會報錯 index:類似mysql中的數據庫,type:類似mysql中的表,shards:分片(將數據分成幾片),replicas:副本(3份) @Document(indexName = "discusspost", type = "_doc", shards = 6, replicas = 3) public class DiscussPost {/*** 自增主鍵,插入數據后將自增主鍵設置到對象中*/@TableId(type = IdType.AUTO)@Idprivate Integer id;@Field(type = FieldType.Integer)private Integer userId;//文本類型,將對應的內容拆分成最多的詞,搜索時搜索最常見的分詞即可@Field(type = FieldType.Text, analyzer = "ik_max_word", searchAnalyzer = "ik_smart")private String title;@Field(type = FieldType.Text, analyzer = "ik_max_word", searchAnalyzer = "ik_smart")private String content;//0-普通; 1-置頂;@Field(type = FieldType.Integer)private Integer type;//0-正常; 1-精華; 2-拉黑;@Field(type = FieldType.Integer)private Integer status;@Field(type = FieldType.Date)private Date createTime;@Field(type = FieldType.Integer)private Integer commentCount;@Field(type = FieldType.Double)private Double score;}

2.5 數據操作

DiscussPostRepository

@Repository public interface DiscussPostRepository extends ElasticsearchRepository<DiscussPost, Integer> { }

2.6 測試

@RunWith(SpringRunner.class) @SpringBootTest @ContextConfiguration(classes = CommunityApplication.class) public class ElasticsearchTests {@Resourceprivate DiscussPostMapper discussPostMapper;@Resourceprivate DiscussPostService discussPostService;@Resourceprivate DiscussPostRepository discussRepository;@Resourceprivate ElasticsearchTemplate elasticTemplate;/*** 每次添加單條數據*/@Testpublic void testInsert() {discussRepository.save(discussPostMapper.selectById(241));discussRepository.save(discussPostMapper.selectById(242));discussRepository.save(discussPostMapper.selectById(243));}/*** 每次添加多條數據*/@Testpublic void testInsertList() {discussRepository.saveAll(discussPostService.findDiscussPosts(101, 0, 100));discussRepository.saveAll(discussPostService.findDiscussPosts(102, 0, 100));discussRepository.saveAll(discussPostService.findDiscussPosts(103, 0, 100));discussRepository.saveAll(discussPostService.findDiscussPosts(111, 0, 100));discussRepository.saveAll(discussPostService.findDiscussPosts(112, 0, 100));discussRepository.saveAll(discussPostService.findDiscussPosts(131, 0, 100));discussRepository.saveAll(discussPostService.findDiscussPosts(132, 0, 100));discussRepository.saveAll(discussPostService.findDiscussPosts(133, 0, 100));discussRepository.saveAll(discussPostService.findDiscussPosts(134, 0, 100));}/*** 測試更新*/@Testpublic void testUpdate() {DiscussPost post = discussPostMapper.selectById(231);post.setContent("我是新人,使勁灌水.");discussRepository.save(post);}/*** 測試刪除*/@Testpublic void testDelete() {// discussRepository.deleteById(231);discussRepository.deleteAll();}/*** 測試查詢*/@Testpublic void testSearchByRepository() {SearchQuery searchQuery = new NativeSearchQueryBuilder().withQuery(QueryBuilders.multiMatchQuery("互聯網寒冬", "title", "content")).withSort(SortBuilders.fieldSort("type").order(SortOrder.DESC)).withSort(SortBuilders.fieldSort("score").order(SortOrder.DESC)).withSort(SortBuilders.fieldSort("createTime").order(SortOrder.DESC)).withPageable(PageRequest.of(0, 10)).withHighlightFields(new HighlightBuilder.Field("title").preTags("<em>").postTags("</em>"),new HighlightBuilder.Field("content").preTags("<em>").postTags("</em>")).build();// elasticTemplate.queryForPage(searchQuery, class, SearchResultMapper)// 底層獲取得到了高亮顯示的值, 但是沒有返回.Page<DiscussPost> page = discussRepository.search(searchQuery);System.out.println(page.getTotalElements());System.out.println(page.getTotalPages());System.out.println(page.getNumber());System.out.println(page.getSize());for (DiscussPost post : page) {System.out.println(post);}}@Testpublic void testSearchByTemplate() {SearchQuery searchQuery = new NativeSearchQueryBuilder().withQuery(QueryBuilders.multiMatchQuery("互聯網寒冬", "title", "content")).withSort(SortBuilders.fieldSort("type").order(SortOrder.DESC)).withSort(SortBuilders.fieldSort("score").order(SortOrder.DESC)).withSort(SortBuilders.fieldSort("createTime").order(SortOrder.DESC)).withPageable(PageRequest.of(0, 10)).withHighlightFields(new HighlightBuilder.Field("title").preTags("<em>").postTags("</em>"),new HighlightBuilder.Field("content").preTags("<em>").postTags("</em>")).build();Page<DiscussPost> page = elasticTemplate.queryForPage(searchQuery, DiscussPost.class, new SearchResultMapper() {@Overridepublic <T> AggregatedPage<T> mapResults(SearchResponse response, Class<T> aClass, Pageable pageable) {SearchHits hits = response.getHits();if (hits.getTotalHits() <= 0) {return null;}List<DiscussPost> list = new ArrayList<>();for (SearchHit hit : hits) {DiscussPost post = new DiscussPost();String id = hit.getSourceAsMap().get("id").toString();post.setId(Integer.valueOf(id));String userId = hit.getSourceAsMap().get("userId").toString();post.setUserId(Integer.valueOf(userId));String title = hit.getSourceAsMap().get("title").toString();post.setTitle(title);String content = hit.getSourceAsMap().get("content").toString();post.setContent(content);String status = hit.getSourceAsMap().get("status").toString();post.setStatus(Integer.valueOf(status));String createTime = hit.getSourceAsMap().get("createTime").toString();post.setCreateTime(new Date(Long.valueOf(createTime)));String commentCount = hit.getSourceAsMap().get("commentCount").toString();post.setCommentCount(Integer.valueOf(commentCount));// 處理高亮顯示的結果HighlightField titleField = hit.getHighlightFields().get("title");if (titleField != null) {post.setTitle(titleField.getFragments()[0].toString());}HighlightField contentField = hit.getHighlightFields().get("content");if (contentField != null) {post.setContent(contentField.getFragments()[0].toString());}list.add(post);}return new AggregatedPageImpl(list, pageable,hits.getTotalHits(), response.getAggregations(), response.getScrollId(), hits.getMaxScore());}});System.out.println(page.getTotalElements());System.out.println(page.getTotalPages());System.out.println(page.getNumber());System.out.println(page.getSize());for (DiscussPost post : page) {System.out.println(post);}}}

• 測試testinsert()方法和testInsertList()方法
  
  

• GET: localhost:9200/discusspost/_search: 查詢discusspost索引下的文檔
  

• 查詢指定的文檔數據
  

• 調用修改方法修改上面這條數據的content
  
  

• 測試刪除方法
  

三、開發社區搜索功能

3.1 業務層

ElasticSearchService(封裝對es的增刪改查操作)

@Service public class ElasticsearchService {@Resourceprivate DiscussPostRepository discussRepository;@Resourceprivate ElasticsearchTemplate elasticTemplate;public void saveDiscussPost(DiscussPost post) {discussRepository.save(post);}public void deleteDiscussPost(int id) {discussRepository.deleteById(id);}public Page<DiscussPost> searchDiscussPost(String keyword, int current, int limit) {SearchQuery searchQuery = new NativeSearchQueryBuilder().withQuery(QueryBuilders.multiMatchQuery(keyword, "title", "content")).withSort(SortBuilders.fieldSort("type").order(SortOrder.DESC)).withSort(SortBuilders.fieldSort("score").order(SortOrder.DESC)).withSort(SortBuilders.fieldSort("createTime").order(SortOrder.DESC)).withPageable(PageRequest.of(current, limit)).withHighlightFields(new HighlightBuilder.Field("title").preTags("<em>").postTags("</em>"),new HighlightBuilder.Field("content").preTags("<em>").postTags("</em>")).build();return elasticTemplate.queryForPage(searchQuery, DiscussPost.class, new SearchResultMapper() {@Overridepublic <T> AggregatedPage<T> mapResults(SearchResponse response, Class<T> aClass, Pageable pageable) {SearchHits hits = response.getHits();if (hits.getTotalHits() <= 0) {return null;}List<DiscussPost> list = new ArrayList<>();for (SearchHit hit : hits) {DiscussPost post = new DiscussPost();String id = hit.getSourceAsMap().get("id").toString();post.setId(Integer.valueOf(id));String userId = hit.getSourceAsMap().get("userId").toString();post.setUserId(Integer.valueOf(userId));String title = hit.getSourceAsMap().get("title").toString();post.setTitle(title);String content = hit.getSourceAsMap().get("content").toString();post.setContent(content);String status = hit.getSourceAsMap().get("status").toString();post.setStatus(Integer.valueOf(status));String createTime = hit.getSourceAsMap().get("createTime").toString();post.setCreateTime(new Date(Long.valueOf(createTime)));String commentCount = hit.getSourceAsMap().get("commentCount").toString();post.setCommentCount(Integer.valueOf(commentCount));// 處理高亮顯示的結果HighlightField titleField = hit.getHighlightFields().get("title");if (titleField != null) {post.setTitle(titleField.getFragments()[0].toString());}HighlightField contentField = hit.getHighlightFields().get("content");if (contentField != null) {post.setContent(contentField.getFragments()[0].toString());}list.add(post);}return new AggregatedPageImpl(list, pageable,hits.getTotalHits(), response.getAggregations(), response.getScrollId(), hits.getMaxScore());}});}}

3.2 控制層

DiscussPostController

@PostMapping("/add")@ResponseBodypublic String addDiscussPost(String title, String content) {User user = UserThreadLocal.getUser();if (user == null) {return CommunityUtil.getJSONString(403, "你還沒有登錄喔!");}DiscussPost post = new DiscussPost();post.setUserId(user.getId());post.setTitle(title);post.setContent(content);post.setCreateTime(new Date());post.setType(0);post.setStatus(0);post.setCommentCount(0);discussPostService.addDiscussPost(post);// 觸發發帖事件Event event = new Event().setTopic(TOPIC_PUBLISH).setUserId(user.getId()).setEntityType(ENTITY_TYPE_POST).setEntityId(post.getId());eventProducer.fireEvent(event);// 報錯的情況,將來統一處理.return CommunityUtil.getJSONString(0, "發布成功!");}

CommentController

@PostMapping("/add/{discussPostId}")public String addComment(@PathVariable("discussPostId") int discussPostId, Comment comment) {comment.setUserId(UserThreadLocal.getUser().getId());comment.setStatus(0);comment.setCreateTime(new Date());commentService.addComment(comment);// 觸發評論事件Event event = new Event().setTopic(TOPIC_COMMENT).setUserId(UserThreadLocal.getUser().getId()).setEntityType(comment.getEntityType()).setEntityId(comment.getEntityId()).setData("postId", discussPostId);if (comment.getEntityType() == ENTITY_TYPE_POST) {DiscussPost target = discussPostService.findDiscussPostById(comment.getEntityId());event.setEntityUserId(target.getUserId());} else if (comment.getEntityType() == ENTITY_TYPE_COMMENT) {Comment target = commentService.findCommentById(comment.getEntityId());event.setEntityUserId(target.getUserId());}eventProducer.fireEvent(event);if (comment.getEntityType() == ENTITY_TYPE_POST) {// 觸發發帖事件event = new Event().setTopic(TOPIC_PUBLISH).setUserId(comment.getUserId()).setEntityType(ENTITY_TYPE_POST).setEntityId(discussPostId);eventProducer.fireEvent(event);}return "redirect:/discuss/detail/" + discussPostId;}

SearchController

@Controller public class SearchController implements CommunityConstant {@Resourceprivate ElasticsearchService elasticsearchService;@Resourceprivate UserService userService;@Resourceprivate LikeService likeService;// search?keyword=xxx@GetMapping("/search")public String search(String keyword, Page page, Model model) {// 搜索帖子org.springframework.data.domain.Page<DiscussPost> searchResult =elasticsearchService.searchDiscussPost(keyword, page.getCurrent() - 1, page.getLimit());// 聚合數據List<Map<String, Object>> discussPosts = new ArrayList<>();if (searchResult != null) {for (DiscussPost post : searchResult) {Map<String, Object> map = new HashMap<>();// 帖子map.put("post", post);// 作者map.put("user", userService.findUserById(post.getUserId()));// 點贊數量map.put("likeCount", likeService.findEntityLikeCount(ENTITY_TYPE_POST, post.getId()));discussPosts.add(map);}}model.addAttribute("discussPosts", discussPosts);model.addAttribute("keyword", keyword);// 分頁信息page.setPath("/search?keyword=" + keyword);page.setRows(searchResult == null ? 0 : (int) searchResult.getTotalElements());return "/site/search";}}

3.3 新增事件

EventConsumer

// 消費發帖事件@KafkaListener(topics = {TOPIC_PUBLISH})public void handlePublishMessage(ConsumerRecord record) {if (record == null || record.value() == null) {logger.error("消息的內容為空!");return;}Event event = JSONObject.parseObject(record.value().toString(), Event.class);if (event == null) {logger.error("消息格式錯誤!");return;}DiscussPost post = discussPostService.findDiscussPostById(event.getEntityId());elasticsearchService.saveDiscussPost(post);}

3.4 靜態資源

index.html

<!-- 搜索 --><form class="form-inline my-2 my-lg-0" method="get" th:action="@{/search}"><input class="form-control mr-sm-2" type="search" aria-label="Search" name="keyword" th:value="${keyword}"/><button class="btn btn-outline-light my-2 my-sm-0" type="submit">搜索</button></form>

serch.html

<!doctype html> <html lang="en" xmlns:th="http://www.thymeleaf.org"> <head><meta charset="utf-8"><meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no"><link rel="icon" href="https://static.nowcoder.com/images/logo_87_87.png"/><link rel="stylesheet" href="https://stackpath.bootstrapcdn.com/bootstrap/4.3.1/css/bootstrap.min.css" crossorigin="anonymous"><link rel="stylesheet" th:href="@{/css/global.css}" /><title>牛客網-搜索結果</title> </head> <body> <div class="nk-container"><!-- 頭部 --><header class="bg-dark sticky-top" th:replace="index::header"><div class="container"><!-- 導航 --><nav class="navbar navbar-expand-lg navbar-dark"><!-- logo --><a class="navbar-brand" href="#"></a><button class="navbar-toggler" type="button" data-toggle="collapse" data-target="#navbarSupportedContent" aria-controls="navbarSupportedContent" aria-expanded="false" aria-label="Toggle navigation"><span class="navbar-toggler-icon"></span></button><!-- 功能 --><div class="collapse navbar-collapse" id="navbarSupportedContent"><ul class="navbar-nav mr-auto"><li class="nav-item ml-3 btn-group-vertical"><a class="nav-link" href="../index.html">首頁</a></li><li class="nav-item ml-3 btn-group-vertical"><a class="nav-link position-relative" href="letter.html">消息<span class="badge badge-danger">12</span></a></li><li class="nav-item ml-3 btn-group-vertical"><a class="nav-link" href="register.html">注冊</a></li><li class="nav-item ml-3 btn-group-vertical"><a class="nav-link" href="login.html">登錄</a></li><li class="nav-item ml-3 btn-group-vertical dropdown"><a class="nav-link dropdown-toggle" href="#" id="navbarDropdown" role="button" data-toggle="dropdown" aria-haspopup="true" aria-expanded="false"><img src="http://images.nowcoder.com/head/1t.png" class="rounded-circle" style="width:30px;"/></a><div class="dropdown-menu" aria-labelledby="navbarDropdown"><a class="dropdown-item text-center" href="profile.html">個人主頁</a><a class="dropdown-item text-center" href="setting.html">賬號設置</a><a class="dropdown-item text-center" href="login.html">退出登錄</a><div class="dropdown-divider"></div><span class="dropdown-item text-center text-secondary">nowcoder</span></div></li></ul><!-- 搜索 --><form class="form-inline my-2 my-lg-0" action="search.html"><input class="form-control mr-sm-2" type="search" aria-label="Search" /><button class="btn btn-outline-light my-2 my-sm-0" type="submit">搜索</button></form></div></nav></div></header><!-- 內容 --><div class="main"><div class="container"><h6><b class="square"></b> 相關帖子</h6><!-- 帖子列表 --><ul class="list-unstyled mt-4"><li class="media pb-3 pt-3 mb-3 border-bottom" th:each="map:${discussPosts}"><img th:src="${map.user.headerUrl}" class="mr-4 rounded-circle" alt="用戶頭像" style="width:50px;height:50px;"><div class="media-body"><h6 class="mt-0 mb-3"><a th:href="@{|/discuss/detail/${map.post.id}|}" th:utext="${map.post.title}">備戰<em>春招</em>，面試刷題跟他復習，一個月全搞定！</a></h6><div class="mb-3" th:utext="${map.post.content}">金三銀四的金三已經到了，你還沉浸在過年的喜悅中嗎？ 如果是，那我要讓你清醒一下了：目前大部分公司已經開啟了內推，正式網申也將在3月份陸續開始，金三銀四，<em>春招</em>的求職黃金時期已經來啦！！！ 再不準備，作為19應屆生的你可能就找不到工作了。。。作為20屆實習生的你可能就找不到實習了。。。 現階段時間緊，任務重，能做到短時間內快速提升的也就只有算法了， 那么算法要怎么復習？重點在哪里？常見筆試面試算法題型和解題思路以及最優代碼是怎樣的？ 跟左程云老師學算法，不僅能解決以上所有問題，還能在短時間內得到最大程度的提升！！！</div><div class="text-muted font-size-12"><u class="mr-3" th:utext="${map.user.username}">寒江雪</u>發布于 <b th:text="${#dates.format(map.post.createTime,'yyyy-MM-dd HH:mm:ss')}">2019-04-15 15:32:18</b><ul class="d-inline float-right"><li class="d-inline ml-2">贊 <i th:text="${map.likeCount}">11</i></li><li class="d-inline ml-2">|</li><li class="d-inline ml-2">回復 <i th:text="${map.post.commentCount}">7</i></li></ul></div></div></li></ul><!-- 分頁 --><nav class="mt-5" th:replace="index::pagination"><ul class="pagination justify-content-center"><li class="page-item"><a class="page-link" href="#">首頁</a></li><li class="page-item disabled"><a class="page-link" href="#">上一頁</a></li><li class="page-item active"><a class="page-link" href="#">1</a></li><li class="page-item"><a class="page-link" href="#">2</a></li><li class="page-item"><a class="page-link" href="#">3</a></li><li class="page-item"><a class="page-link" href="#">4</a></li><li class="page-item"><a class="page-link" href="#">5</a></li><li class="page-item"><a class="page-link" href="#">下一頁</a></li><li class="page-item"><a class="page-link" href="#">末頁</a></li></ul></nav></div></div><!-- 尾部 --><footer class="bg-dark" th:replace="index::footer"><div class="container"><div class="row"><!-- 二維碼 --><div class="col-4 qrcode"><img src="https://uploadfiles.nowcoder.com/app/app_download.png" class="img-thumbnail" style="width:136px;" /></div><!-- 公司信息 --><div class="col-8 detail-info"><div class="row"><div class="col"><ul class="nav"><li class="nav-item"><a class="nav-link text-light" href="#">關于我們</a></li><li class="nav-item"><a class="nav-link text-light" href="#">加入我們</a></li><li class="nav-item"><a class="nav-link text-light" href="#">意見反饋</a></li><li class="nav-item"><a class="nav-link text-light" href="#">企業服務</a></li><li class="nav-item"><a class="nav-link text-light" href="#">聯系我們</a></li><li class="nav-item"><a class="nav-link text-light" href="#">免責聲明</a></li><li class="nav-item"><a class="nav-link text-light" href="#">友情鏈接</a></li></ul></div></div><div class="row"><div class="col"><ul class="nav btn-group-vertical company-info"><li class="nav-item text-white-50">公司地址：北京市朝陽區大屯路東金泉時代3-2708北京牛客科技有限公司</li><li class="nav-item text-white-50">聯系方式：010-60728802(電話)&nbsp;&nbsp;&nbsp;&nbsp;admin@nowcoder.com</li><li class="nav-item text-white-50">牛客科技?2018 All rights reserved</li><li class="nav-item text-white-50">京ICP備14055008號-4 &nbsp;&nbsp;&nbsp;&nbsp;<img src="http://static.nowcoder.com/company/images/res/ghs.png" style="width:18px;" />京公網安備 11010502036488號</li></ul></div></div></div></div></div></footer> </div><script src="https://code.jquery.com/jquery-3.3.1.min.js" crossorigin="anonymous"></script> <script src="https://cdnjs.cloudflare.com/ajax/libs/popper.js/1.14.7/umd/popper.min.js" crossorigin="anonymous"></script> <script src="https://stackpath.bootstrapcdn.com/bootstrap/4.3.1/js/bootstrap.min.js" crossorigin="anonymous"></script> <script th:src="@{/js/global.js}"></script> </body> </html>

3.4 測試

• 每次發帖和對帖子評論都會使用kafka異步更新es中對應的數據,我們直接新增一個帖子,然后搜索我們剛創建的帖子
  
  
  

總結

以上是生活随笔為你收集整理的仿牛客论坛项目(4)的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。