當前位置：首頁 > 编程资源 > 编程问答 >内容正文

编程问答

log4net直切ElasticSearch，小步快跑首选

發布時間：2023/12/4 编程问答 38 豆豆

生活随笔收集整理的這篇文章主要介紹了 log4net直切ElasticSearch，小步快跑首选小編覺得挺不錯的,現在分享給大家,幫大家做個參考.

很多小步快跑的公司，開發人員可能就3-4個，面對巨大業務壓力，日連夜的趕著上線，快速試錯，自然就沒時間搭建一些基礎設施，比如說logCenter，但初期項目不穩定，bug又多，每次都跑到生產去找日志，確實也不大方便，用elk或者用hadoop做日志中心，雖然都是沒問題的，但基于成本和人手還是怎么簡化怎么來，

本篇就來說說直接使用log4net的ElasticSearchAppender擴展直接將log寫入到es中。

一：從nuget下載

nuget下來的都是開箱可用，如果看源碼的話可以在github上找一下log4net.ElasticSearch項目。https://github.com/jptoto/log4net.ElasticSearch

1. App.config配置

nuget包下來之后，就可以配置config文件了，其實還是蠻簡單的，大家可以根據自己的項目合理的配置里面的各項參數，為了方便大家理解，我在每個配置項上加了詳細的注釋，大家可以仔細看看。

<?xml?version="1.0"?encoding="utf-8"??> <configuration><configSections><p?name="log4net"?type="log4net.Config.Log4NetConfigurationSectionHandler,?log4net"?/></configSections><log4net><appender?name="ElasticSearchAppender"?type="log4net.ElasticSearch.ElasticSearchAppender,?log4net.ElasticSearch"><connectionString?value="Scheme=http;Server=192.168.23.145;Index=datamiprm_log;Port=9200;rolling=true"/><lossy?value="false"?/><evaluator?type="log4net.Core.LevelEvaluator"><threshold?value="ERROR"?/></evaluator><bufferSize?value="50"?/></appender><root><level?value="ALL"/><appender-ref?ref="ElasticSearchAppender"?/></root></log4net><startup><supportedRuntime?version="v4.0"?sku=".NETFramework,Version=v4.6.1"?/></startup> </configuration>

上面有幾點要特別注意一下：

bufferSize

這個大家理解成緩存區，方便批量提交到es中，否則的話，過于頻繁的和es進行交互，對帶寬，對application，es都是比較大的壓力。

有時候我們有這樣的需求，我希望ERROR，Fault這種級別的錯誤不要走buffer，直接提交給es，這樣更容易讓初創團隊發現問題，找到問題，恭喜你，上面這個配置就是解決這個事的。

2. 在AssemblyInfo中追加如下配置

//?可以指定所有值，也可以使用以下所示的?"*"?預置版本號和修訂號 //?方法是按如下所示使用“*”:?: //?[assembly:?AssemblyVersion("1.0.*")] [assembly:?AssemblyVersion("1.0.0.0")] [assembly:?AssemblyFileVersion("1.0.0.0")] [assembly:?log4net.Config.XmlConfigurator(Watch?=?true)]

3. 然后就可以寫一段測試代碼啦。

namespace?ConsoleApp1 {class?Program{private?static?readonly?ILog?_log?=?LogManager.GetLogger(typeof(Program));static?void?Main(string[]?args){????????????for?(int?i?=?0;?i?<?1000;?i++){try{var?m?=?"1";var?result?=?100?/?Convert.ToInt32(m);_log.Info("我要開始記錄日志啦");}catch?(Exception?ex){_log.Error("調用失敗"?+?i,?ex);//_log.Info("調用失敗"?+?i,?ex);//Console.WriteLine(i);}}Console.Read();}} }

然后通過elasticsearch-head 插件進行查看，各種數據就都在es中了。

二：elasticsearch-header 插件

關于這個插件的下載，因為google官網被屏蔽，大家如果有VPN的話，可以自行在chrome商店搜索，當然也可以直接下載我的zip包，elasticsearch-head。在瀏覽器中加載插件的時候選擇0.1.3.0 文件夾即可

完了之后點擊右上邊的 “放大鏡“ 按鈕就可以看到你想看到的UI了。

三：下載elasticsearch

這是一個基于luncene的分布式搜索框架，用起來還是挺順手的，你可以下載5.6.4版本。

[elsearch@localhost?myapp]$?ls elasticsearch-5.6.4.tar.gz??

一般來說，安裝的過程中你可能會遇到3個坑。

1. ?不能用root賬號登錄

[root@localhost?bin]#?./elasticsearch [2018-05-31T04:01:59,402][WARN?][o.e.b.ElasticsearchUncaughtExceptionHandler]?[]?uncaught?exception?in?thread?[main] org.elasticsearch.bootstrap.StartupException:?java.lang.RuntimeException:?can?not?run?elasticsearch?as?rootat?org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:136)?~[elasticsearch-5.6.4.jar:5.6.4]at?org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:123)?~[elasticsearch-5.6.4.jar:5.6.4]at?org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:70)?~[elasticsearch-5.6.4.jar:5.6.4]at?org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:134)?~[elasticsearch-5.6.4.jar:5.6.4]at?org.elasticsearch.cli.Command.main(Command.java:90)?~[elasticsearch-5.6.4.jar:5.6.4]at?org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:91)?~[elasticsearch-5.6.4.jar:5.6.4]at?org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:84)?~[elasticsearch-5.6.4.jar:5.6.4] Caused?by:?java.lang.RuntimeException:?can?not?run?elasticsearch?as?rootat?org.elasticsearch.bootstrap.Bootstrap.initializeNatives(Bootstrap.java:106)?~[elasticsearch-5.6.4.jar:5.6.4]at?org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:195)?~[elasticsearch-5.6.4.jar:5.6.4]at?org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:342)?~[elasticsearch-5.6.4.jar:5.6.4]at?org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:132)?~[elasticsearch-5.6.4.jar:5.6.4]...?6?more

這個簡單，增加一個elasearch用戶就可以了。

groupadd?elsearch???????????????????#新建elsearch組 useradd?elsearch?-g?elsearch?-p?elasticsearch??#新建一個elsearch用戶 chown?-R?elsearch:elsearch??./elasticsearch????#指定elasticsearch所屬elsearch組

2. ?內存不足的問題。

[elsearch@localhost?bin]$?./elasticsearch Java?HotSpot(TM)?64-Bit?Server?VM?warning:?INFO:?os::commit_memory(0x00000000a9990000,?1449590784,?0)?failed;?error='Cannot?allocate?memory'?(errno=12) # #?There?is?insufficient?memory?for?the?Java?Runtime?Environment?to?continue. #?Native?memory?allocation?(mmap)?failed?to?map?1449590784?bytes?for?committing?reserved?memory. #?An?error?report?file?with?more?information?is?saved?as: #?/usr/myapp/elasticsearch/bin/hs_err_pid33341

這是ES默認分配的堆內存是2g，如果出現這個問題，一般來說是你的虛擬機擁有的內存小于2g，只需要在 jvm.options 中將2g修改1g就可以了。

[root@localhost?config]#?ls elasticsearch.yml??jvm.options??log4j2.properties??scripts#?Xms?represents?the?initial?size?of?total?heap?space #?Xmx?represents?the?maximum?size?of?total?heap?space-Xms1g -Xmx1g

3. max file descriptors 太少和 max virtual memory 虛擬內存太低。

[elsearch@localhost?bin]$?./elasticsearch [2018-05-30T20:44:56,484][INFO?][o.e.n.Node???????????????]?[]?initializing?... [2018-05-30T20:44:56,632][INFO?][o.e.e.NodeEnvironment????]?[f9t2Sfl]?using?[1]?data?paths,?mounts?[[/?(rootfs)]],?net?usable_space?[14.4gb],?net?total_space?[22.1gb],?spins??[unknown],?types?[rootfs] [2018-05-30T20:44:56,632][INFO?][o.e.e.NodeEnvironment????]?[f9t2Sfl]?heap?size?[989.8mb],?compressed?ordinary?object?pointers?[true] [2018-05-30T20:44:56,634][INFO?][o.e.n.Node???????????????]?node?name?[f9t2Sfl]?derived?from?node?ID?[f9t2SfljReiND4XeMLUbyA];?set?[node.name]?to?override [2018-05-30T20:44:56,634][INFO?][o.e.n.Node???????????????]?version[5.6.4],?pid[33546],?build[8bbedf5/2017-10-31T18:55:38.105Z],?OS[Linux/3.10.0-327.el7.x86_64/amd64],?JVM[Oracle?Corporation/Java?HotSpot(TM)?64-Bit?Server?VM/1.8.0_144/25.144-b01] [2018-05-30T20:44:56,634][INFO?][o.e.n.Node???????????????]?JVM?arguments?[-Xms1g,?-Xmx1g,?-XX:+UseConcMarkSweepGC,?-XX:CMSInitiatingOccupancyFraction=75,?-XX:+UseCMSInitiatingOccupancyOnly,?-XX:+AlwaysPreTouch,?-Xss1m,?-Djava.awt.headless=true,?-Dfile.encoding=UTF-8,?-Djna.nosys=true,?-Djdk.io.permissionsUseCanonicalPath=true,?-Dio.netty.noUnsafe=true,?-Dio.netty.noKeySetOptimization=true,?-Dio.netty.recycler.maxCapacityPerThread=0,?-Dlog4j.shutdownHookEnabled=false,?-Dlog4j2.disable.jmx=true,?-Dlog4j.skipJansi=true,?-XX:+HeapDumpOnOutOfMemoryError,?-Des.path.home=/usr/myapp/elasticsearch] [2018-05-30T20:44:57,530][INFO?][o.e.p.PluginsService?????]?[f9t2Sfl]?loaded?module?[aggs-matrix-stats] [2018-05-30T20:44:57,530][INFO?][o.e.p.PluginsService?????]?[f9t2Sfl]?loaded?module?[ingest-common] [2018-05-30T20:44:57,530][INFO?][o.e.p.PluginsService?????]?[f9t2Sfl]?loaded?module?[lang-expression] [2018-05-30T20:44:57,531][INFO?][o.e.p.PluginsService?????]?[f9t2Sfl]?loaded?module?[lang-groovy] [2018-05-30T20:44:57,531][INFO?][o.e.p.PluginsService?????]?[f9t2Sfl]?loaded?module?[lang-mustache] [2018-05-30T20:44:57,531][INFO?][o.e.p.PluginsService?????]?[f9t2Sfl]?loaded?module?[lang-painless] [2018-05-30T20:44:57,531][INFO?][o.e.p.PluginsService?????]?[f9t2Sfl]?loaded?module?[parent-join] [2018-05-30T20:44:57,531][INFO?][o.e.p.PluginsService?????]?[f9t2Sfl]?loaded?module?[percolator] [2018-05-30T20:44:57,531][INFO?][o.e.p.PluginsService?????]?[f9t2Sfl]?loaded?module?[reindex] [2018-05-30T20:44:57,531][INFO?][o.e.p.PluginsService?????]?[f9t2Sfl]?loaded?module?[transport-netty3] [2018-05-30T20:44:57,531][INFO?][o.e.p.PluginsService?????]?[f9t2Sfl]?loaded?module?[transport-netty4] [2018-05-30T20:44:57,532][INFO?][o.e.p.PluginsService?????]?[f9t2Sfl]?no?plugins?loaded [2018-05-30T20:44:59,469][INFO?][o.e.d.DiscoveryModule????]?[f9t2Sfl]?using?discovery?type?[zen] [2018-05-30T20:45:00,107][INFO?][o.e.n.Node???????????????]?initialized [2018-05-30T20:45:00,107][INFO?][o.e.n.Node???????????????]?[f9t2Sfl]?starting?... [2018-05-30T20:45:00,339][INFO?][o.e.t.TransportService???]?[f9t2Sfl]?publish_address?{192.168.23.143:9300},?bound_addresses?{[::]:9300} [2018-05-30T20:45:00,356][INFO?][o.e.b.BootstrapChecks????]?[f9t2Sfl]?bound?or?publishing?to?a?non-loopback?or?non-link-local?address,?enforcing?bootstrap?checks ERROR:?[2]?bootstrap?checks?failed [1]:?max?file?descriptors?[4096]?for?elasticsearch?process?is?too?low,?increase?to?at?least?[65536] [2]:?max?virtual?memory?areas?vm.max_map_count?[65530]?is?too?low,?increase?to?at?least?[262144] [2018-05-30T20:45:00,365][INFO?][o.e.n.Node???????????????]?[f9t2Sfl]?stopping?... [2018-05-30T20:45:00,444][INFO?][o.e.n.Node???????????????]?[f9t2Sfl]?stopped [2018-05-30T20:45:00,444][INFO?][o.e.n.Node???????????????]?[f9t2Sfl]?closing?... [2018-05-30T20:45:00,455][INFO?][o.e.n.Node???????????????]?[f9t2Sfl]?closed

這個max file descriptors 的問題，我只需要修改 vim /etc/security/limits.conf 文件增加句柄值即可。

*?hard?nofile?65536 *?soft?nofile?65536

虛擬內存的 max virtual memory areas vm.max_map_count [65530] is too low, increase to at least [262144] ?只需要在sysctl.conf 中修改max即可。

[root@localhost?config]#?vim?/etc/sysctl.confvm.max_map_count=655360

這些坑解決的話，es就可以正常啟動了，最后記得在elasticsearch.yml 中將host設為0.0.0.0 讓遠程機器可以訪問。

[elsearch@localhost?config]$?vim?elasticsearch.ymlnetwork.host:?0.0.0.0

最后執行 ./elasticsearch -d ?讓es在后臺執行。

[elsearch@localhost?bin]$?./elasticsearch?-d[elsearch@localhost?bin]$?netstat?-tlnp (Not?all?processes?could?be?identified,?non-owned?process?infowill?not?be?shown,?you?would?have?to?be?root?to?see?it?all.) Active?Internet?connections?(only?servers) Proto?Recv-Q?Send-Q?Local?Address???????????Foreign?Address?????????State???????PID/Program?name???? tcp????????0??????0?192.168.122.1:53????????0.0.0.0:*???????????????LISTEN??????-??????????????????? tcp????????0??????0?0.0.0.0:22??????????????0.0.0.0:*???????????????LISTEN??????-??????????????????? tcp????????0??????0?127.0.0.1:631???????????0.0.0.0:*???????????????LISTEN??????-??????????????????? tcp????????0??????0?127.0.0.1:25????????????0.0.0.0:*???????????????LISTEN??????-??????????????????? tcp6???????0??????0?:::9200?????????????????:::*????????????????????LISTEN??????17523/java?????????? tcp6???????0??????0?:::9300?????????????????:::*????????????????????LISTEN??????17523/java?????????? tcp6???????0??????0?:::22???????????????????:::*????????????????????LISTEN??????-??????????????????? tcp6???????0??????0?::1:631?????????????????:::*????????????????????LISTEN??????-??????????????????? tcp6???????0??????0?::1:25??????????????????:::*????????????????????LISTEN??????-??????????????????? [elsearch@localhost?bin]$

好了，本篇就說這么多，如果你要更精細化的查詢，可以再搭建一個配套版本的kibana即可。

總結

以上是生活随笔為你收集整理的log4net直切ElasticSearch，小步快跑首选的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。

上一篇：好的自我介绍，面试成功一大半
下一篇：壹佰文章总结| 关于ASP.NETCor