一、Elasticsearch 集群環(huán)境準(zhǔn)備 ### --- hadoop01~03修改系統(tǒng)配置：修改/etc/sysctl.conf~~~ # 修改/etc/sysctl.conf [root@hadoop01 ~]# vim /etc/sysctl.conf ~~~在文件末尾添加如下參數(shù) vm.max_map_count=655360 ~~~ # 執(zhí)行sysctl -p 讓其生效[root@hadoop01 ~]# sysctl -p ### --- 修改/etc/security/limits.conf~~~ # 修改linux系統(tǒng)對文件描述符的限制級別 [root@hadoop01 ~]# vim /etc/security/limits.conf ~~~在文件末尾添加如下參數(shù) * soft nofile 65536 * hard nofile 65536 * soft nproc 4096 * hard nproc 4096 ### --- 準(zhǔn)備工作：創(chuàng)建es用戶~~~ # hadoop01~03創(chuàng)建es用戶(必須)，因為es不能用root用戶啟動 [root@hadoop01 ~]# useradd es [root@hadoop01 ~]# passwd es New password:123456 ~~~ # hadoop01~03使用root用戶執(zhí)行visudo然后為es用戶添加權(quán)限[root@hadoop01 ~]# vim /etc/sudoers ~~~第101行添加如下內(nèi)容 es ALL=(ALL) ALL ### --- 使用root用戶創(chuàng)建工作目錄~~~ # hadoop01~03創(chuàng)建es環(huán)境目錄 [root@hadoop01 ~]# mkdir -p /opt/yanqi/servers/es [root@hadoop01 ~]# mkdir -p /opt/yanqi/servers/es/data/es [root@hadoop01 ~]# mkdir -p /opt/yanqi/servers/es/logs/es [root@hadoop01 ~]# chown -R es /opt/yanqi/servers/es [root@hadoop01 ~]# chown -R es /opt/yanqi/servers/es/data/es [root@hadoop01 ~]# chown -R es /opt/yanqi/servers/es/logs/es 二、elasticsearch分布式部署：Hadoop01 ### --- 分布式安裝部署：# 使用es用戶連接服務(wù)器~~~ # hadoop01解壓版本包 [es@hadoop01 software]$ tar -zxvf elasticsearch-7.3.0-linux-x86_64.tar.gz -C ../servers/es/ ~~~ # 修改版本包名稱 [es@hadoop01 ~]$ mv /opt/yanqi/servers/es/elasticsearch-7.3.0/ /opt/yanqi/servers/es/elasticsearch ### --- 修改配置文件：elasticsearch.yml配置文件說明：~~~ # hadoop01下修改es配置文件參數(shù)： [es@hadoop01 ~]$ vim /opt/yanqi/servers/es/elasticsearch/config/elasticsearch.yml~~~ # 第17行：集群名字 cluster.name: myes~~~ # 第23行：集群中當(dāng)前的節(jié)點(diǎn) node.name: hadoop01~~~ # 第33行：數(shù)據(jù)目錄 path.data: /opt/yanqi/servers/es/data/es~~~ # 第37行：日志目錄 path.logs: /opt/yanqi/servers/es/logs/es~~~ # 第55/59行：當(dāng)前主機(jī)的ip地址：ECS使用內(nèi)網(wǎng)地址 network.host: hadoop01 http.port: 9200~~~ # 第72行：初始化一個新的集群時需要此配置來選舉master：ECS使用內(nèi)網(wǎng)地址 cluster.initial_master_nodes: ["hadoop01","hadoop02","hadoop03"]~~~ # 第68行：寫入候選主節(jié)點(diǎn)的設(shè)備地址 discovery.seed_hosts: ["hadoop01","hadoop02","hadoop03"]~~~ # 第89/90行：行尾添加： 跨域，適用于所有域名 http.cors.enabled: true http.cors.allow-origin: "*" ### --- 安裝包分發(fā)到其他兩臺機(jī)器上~~~ # hadoop01節(jié)點(diǎn)發(fā)送es版本包到其它主機(jī) [es@hadoop01 ~]$ cd /opt/yanqi/servers/es/ [es@hadoop01 es]$ scp -r elasticsearch/ hadoop02:$PWD [es@hadoop01 es]$ scp -r elasticsearch/ hadoop03:$PWD 三、elasticsearch分布式部署：Hadoop02/hadoop03 ### --- hadoop02主機(jī)修改elasticsearch.yaml~~~ # hadoop02主機(jī)修改elasticsearch.yaml配置文件 [es@hadoop02 ~]$ vim /opt/yanqi/servers/es/elasticsearch/config/elasticsearch.yml~~~ # 第17行：集群名字 cluster.name: myes~~~ # 第23行：集群中當(dāng)前的節(jié)點(diǎn) node.name: hadoop02~~~ # 第33行：數(shù)據(jù)目錄 path.data: /opt/yanqi/servers/es/data/es~~~ # 第37行：日志目錄 path.logs: /opt/yanqi/servers/es/logs/es~~~ # 第55/59行：當(dāng)前主機(jī)的ip地址 network.host: hadoop02 http.port: 9200~~~ # 第72行：初始化一個新的集群時需要此配置來選舉master cluster.initial_master_nodes: ["hadoop01","hadoop02","hadoop03"]~~~ # 第68行：寫入候選主節(jié)點(diǎn)的設(shè)備地址 discovery.seed_hosts: ["hadoop01","hadoop02","hadoop03"]~~~ # 第89/90行：行尾添加： 跨域，適用于所有域名 http.cors.enabled: true http.cors.allow-origin: "*" ### --- hadoop03主機(jī)修改elasticsearch.yaml~~~ # hadoop03主機(jī)修改elasticsearch.yaml配置文件 [es@hadoop03 ~]$ vim /opt/yanqi/servers/es/elasticsearch/config/elasticsearch.yml~~~ # 第17行：集群名字17 cluster.name: myes~~~ # 第23行：集群中當(dāng)前的節(jié)點(diǎn)23 node.name: hadoop03~~~ # 第33行：數(shù)據(jù)目錄33 path.data: /opt/yanqi/servers/es/data/es~~~ # 第37行：日志目錄37 path.logs: /opt/yanqi/servers/es/logs/es~~~ # 第55/59行：當(dāng)前主機(jī)的ip地址55 network.host: hadoop0359 http.port: 9200~~~ # 第72行：初始化一個新的集群時需要此配置來選舉master72 cluster.initial_master_nodes: ["hadoop01","hadoop02","hadoop03"]~~~ # 第68行：寫入候選主節(jié)點(diǎn)的設(shè)備地址68 discovery.seed_hosts: ["hadoop01","hadoop02","hadoop03"]~~~ # 第89/90行：行尾添加： 跨域，適用于所有域名89 http.cors.enabled: true90 http.cors.allow-origin: "*" 四、修改jvm內(nèi)存參數(shù)大小 ### --- hadoop01~03修改jvm內(nèi)存大小[es@hadoop01 ~]$ vim /opt/yanqi/servers/es/elasticsearch/config/jvm.options ~~~ # 第22~23行：修改內(nèi)存 -Xms2g -Xmx2g 五、啟動es服務(wù) ### --- 啟動es服務(wù)~~~ # hadoop01~03：后臺啟動es服務(wù)，要發(fā)現(xiàn)錯誤的話，去/opt/yanqi/servers/logs/es目錄下查看。 [es@hadoop01 ~]$ nohup /opt/yanqi/servers/es/elasticsearch/bin/elasticsearch >/dev/null 2>&1 & ~~~ # 查看es服務(wù)進(jìn)程 [es@hadoop01 ~]$ jps 1500 Elasticsearch[es@hadoop01 ~]$ ps -ef | grep 1500 es 1500 1279 8 14:49 pts/1 00:00:37 /opt/yanqi/servers/jdk1.8.0_231/bin/java -Xms512m -Xmx512m -XX:+UseConcMarkSweepGC -XX:CMSInitiatingOccupancyFraction=75 -XX:+UseCMSInitiatingOccupancyOnly -Des.networkaddress.cache.ttl=60 -Des.networkaddress.cache.negative.ttl=10 -XX:+AlwaysPreTouch -Xss1m -Djava.awt.headless=true -Dfile.encoding=UTF-8 -Djna.nosys=true -XX:-OmitStackTraceInFastThrow -Dio.netty.noUnsafe=true -Dio.netty.noKeySetOptimization=true -Dio.netty.recycler.maxCapacityPerThread=0 -Dlog4j.shutdownHookEnabled=false -Dlog4j2.disable.jmx=true -Djava.io.tmpdir=/tmp/elasticsearch-7630453916949373570 -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=data -XX:ErrorFile=logs/hs_err_pid%p.log -XX:+PrintGCDetails -XX:+PrintGCDateStamps -XX:+PrintTenuringDistribution -XX:+PrintGCApplicationStoppedTime -Xloggc:logs/gc.log -XX:+UseGCLogFileRotation -XX:NumberOfGCLogFiles=32 -XX:GCLogFileSize=64m -Dio.netty.allocator.type=unpooled -XX:MaxDirectMemorySize=268435456 -Des.path.home=/opt/yanqi/servers/es/elasticsearch -Des.path.conf=/opt/yanqi/servers/es/elasticsearch/config -Des.distribution.flavor=default -Des.distribution.type=tar -Des.bundled_jdk=true -cp /opt/yanqi/servers/es/elasticsearch/lib/* org.elasticsearch.bootstrap.Elasticsearch es 1570 1500 0 14:49 pts/1 00:00:00 /opt/yanqi/servers/es/elasticsearch/modules/x-pack-ml/platform/linux-x86_64/bin/controller ### --- 訪問es服務(wù)~~~ # http://hadoop01:9200/?pretty ；pretty：格式化的，漂亮的。 ~~~ # UI.輸出參數(shù) {"name" : "hadoop01","cluster_name" : "myes","cluster_uuid" : "dkCOn-auSUCJH42iBUSO1w","version" : {"number" : "7.3.0","build_flavor" : "default","build_type" : "tar","build_hash" : "de777fa","build_date" : "2019-07-24T18:30:11.767338Z","build_snapshot" : false,"lucene_version" : "8.1.0","minimum_wire_compatibility_version" : "6.8.0","minimum_index_compatibility_version" : "6.0.0-beta1"},"tagline" : "You Know, for Search" } ~~~ # http://hadoop02:9200/?pretty ；pretty：格式化的，漂亮的。~~~ # UI.輸出參數(shù) {"name" : "hadoop02","cluster_name" : "myes","cluster_uuid" : "dkCOn-auSUCJH42iBUSO1w","version" : {"number" : "7.3.0","build_flavor" : "default","build_type" : "tar","build_hash" : "de777fa","build_date" : "2019-07-24T18:30:11.767338Z","build_snapshot" : false,"lucene_version" : "8.1.0","minimum_wire_compatibility_version" : "6.8.0","minimum_index_compatibility_version" : "6.0.0-beta1"},"tagline" : "You Know, for Search" } # --- http://hadoop03:9200/?pretty ；pretty：格式化的，漂亮的。~~~ # UI.輸出參數(shù) {"name" : "hadoop03","cluster_name" : "myes","cluster_uuid" : "dkCOn-auSUCJH42iBUSO1w","version" : {"number" : "7.3.0","build_flavor" : "default","build_type" : "tar","build_hash" : "de777fa","build_date" : "2019-07-24T18:30:11.767338Z","build_snapshot" : false,"lucene_version" : "8.1.0","minimum_wire_compatibility_version" : "6.8.0","minimum_index_compatibility_version" : "6.0.0-beta1"},"tagline" : "You Know, for Search" } 附錄一：elasticsearch.yml配置文件說明：

配置項	作用
cluster.name	集群名稱，相同名稱為一個集群
node.name?	節(jié)點(diǎn)名稱，集群模式下每個節(jié)點(diǎn)名稱唯一
node.master	當(dāng)前節(jié)點(diǎn)是否可以被選舉為master節(jié)點(diǎn)，是：true、否：false
node.data	當(dāng)前節(jié)點(diǎn)是否用于存儲數(shù)據(jù)，是：true、否：false
path.data	索引數(shù)據(jù)存放的位置
path.logs	日志文件存放的位置
bootstrap.memory_lock	需求鎖住物理內(nèi)存，是：true、否：false
network.host	監(jiān)聽地址，用于訪問該es
http.port	es對外提供的http端口，默認(rèn) 9200
transport.port	節(jié)點(diǎn)選舉的通信端口 默認(rèn)是9300
discovery.seed_hosts	es7.x 之后新增的配置， 寫入候選主節(jié)點(diǎn)的設(shè)備地址，在開啟服務(wù)后可以被選為主節(jié)點(diǎn)
cluster.initial_master_nodes	es7.x 之后新增的配置， 初始化一個新的集群時需要此配置來選舉master
http.cors.enabled	是否支持跨域，是：true，在使用head插件時需要此配置
http.cors.allow-origin ""	表示支持所有域名

總結(jié)

以上是生活随笔為你收集整理的CC00006.elasticsearch——|HadoopElasticSearch.V06|——|ELK.v06|集群|ElasticSearch集群部署.V1|的全部內(nèi)容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網(wǎng)站內(nèi)容還不錯，歡迎將生活随笔推薦給好友。