0.文章系列鏈接

• SLS機器學(xué)習介紹（01）：時序統(tǒng)計建模
• SLS機器學(xué)習介紹（02）：時序聚類建模
• SLS機器學(xué)習介紹（03）：時序異常檢測建模
• SLS機器學(xué)習介紹（04）：規(guī)則模式挖掘
• SLS機器學(xué)習介紹（05）：時間序列預(yù)測

---

• 一眼看盡上億日志-SLS智能聚類(LogReduce)發(fā)布
• SLS機器學(xué)習最佳實戰(zhàn)：時序異常檢測和報警
• SLS機器學(xué)習最佳實戰(zhàn)：時序預(yù)測

---

1.手中的錘子都有啥？

圍繞日志，挖掘其中更大價值，一直是我們團隊所關(guān)注。在原有日志實時查詢基礎(chǔ)上，今年SLS在DevOps領(lǐng)域完善了如下功能：

• 上下文查詢
• 實時Tail和智能聚類，以提高問題調(diào)查效率
• 提供多種時序數(shù)據(jù)的異常檢測和預(yù)測函數(shù)，來做更智能的檢查和預(yù)測
• 數(shù)據(jù)分析的結(jié)果可視化
• 強大的告警設(shè)置和通知，通過調(diào)用webhook進行關(guān)聯(lián)行動

今天我們重點介紹下，日志只能聚類和異常告警如何配合，更好的進行異常發(fā)現(xiàn)和告警

2.平臺實驗

2.1 實驗數(shù)據(jù)

一份Sys Log的原始數(shù)據(jù)，，并且開啟了日志聚類服務(wù)，具體的狀態(tài)截圖如下：

通過調(diào)整下面截圖中紅色框1的大小，可以改變圖中紅色框2的結(jié)果，但是對于每個最細粒度的pattern并不會改變，也就是說：子Pattern的結(jié)果是穩(wěn)定且唯一的，我們可以通過子Pattern的Signature找到對應(yīng)的原始日志條目。

2.2 生成子模式的時序信息

假設(shè)，我們對這個子Pattern要進行監(jiān)控：

msg:vm-111932.tc su: pam_unix(*:session): session closed for user root
對應(yīng)的 signature_id : __log_signature__: 1814836459146662485

我們得到了上述pattern對應(yīng)的原始日志，可以看下具體的數(shù)量在時間軸上的直返圖：

上圖中，我們可以發(fā)現(xiàn)，這個模式的日志分布不是很均衡，其中還有一些是沒有的，如果直接按照時間窗口統(tǒng)計數(shù)量，得到的時序圖如下：

__log_signature__: 1814836459146662485 | select date_trunc('minute', __time__) as time, COUNT(*) as num from log GROUP BY time order by time ASC limit 10000

上述圖中我們發(fā)現(xiàn)時間上并不是連續(xù)的。因此，我們需要對這條時序進行補點操作。

__log_signature__: 1814836459146662485 | select time_series(time, '1m', '%Y-%m-%d %H:%i:%s', '0') as time, avg(num) as num from ( select __time__ - __time__ % 60 as time, COUNT(*) as num from log GROUP BY time order by time desc ) GROUP by time order by time ASC limit 10000

2.3 對時序進行異常檢測

使用時序異常檢測函數(shù)： ts_predicate_arma

__log_signature__: 1814836459146662485 | select ts_predicate_arma(to_unixtime(time), num, 5, 1, 1, 1, 'avg') from ( select time_series(time, '1m', '%Y-%m-%d %H:%i:%s', '0') as time, avg(num) as num from ( select __time__ - __time__ % 60 as time, COUNT(*) as num from log GROUP BY time order by time desc ) GROUP by time order by time ASC ) limit 10000

2.4 告警該如何設(shè)置

• 將機器學(xué)習函數(shù)的結(jié)果拆解開

__log_signature__: 1814836459146662485 | select t1[1] as unixtime, t1[2] as src, t1[3] as pred, t1[4] as up, t1[5] as lower, t1[6] as prob from ( select ts_predicate_arma(to_unixtime(time), num, 5, 1, 1, 1, 'avg') as res from ( select time_series(time, '1m', '%Y-%m-%d %H:%i:%s', '0') as time, avg(num) as num from ( select __time__ - __time__ % 60 as time, COUNT(*) as num from log GROUP BY time order by time desc ) GROUP by time order by time ASC )) , unnest(res) as t(t1)

• 針對最近兩分鐘的結(jié)果進行告警

__log_signature__: 1814836459146662485 | select unixtime, src, pred, up, lower, prob from ( select t1[1] as unixtime, t1[2] as src, t1[3] as pred, t1[4] as up, t1[5] as lower, t1[6] as prob from ( select ts_predicate_arma(to_unixtime(time), num, 5, 1, 1, 1, 'avg') as res from ( select time_series(time, '1m', '%Y-%m-%d %H:%i:%s', '0') as time, avg(num) as num from ( select __time__ - __time__ % 60 as time, COUNT(*) as num from log GROUP BY time order by time desc ) GROUP by time order by time ASC )) , unnest(res) as t(t1) ) where is_nan(src) = false order by unixtime desc limit 2

• 針對上升點進行告警，并設(shè)置兜底策略

__log_signature__: 1814836459146662485 | select sum(prob) as sumProb, max(src) as srcMax, max(up) as upMax from ( select unixtime, src, pred, up, lower, prob from ( select t1[1] as unixtime, t1[2] as src, t1[3] as pred, t1[4] as up, t1[5] as lower, t1[6] as prob from ( select ts_predicate_arma(to_unixtime(time), num, 5, 1, 1, 1, 'avg') as res from ( select time_series(time, '1m', '%Y-%m-%d %H:%i:%s', '0') as time, avg(num) as num from ( select __time__ - __time__ % 60 as time, COUNT(*) as num from log GROUP BY time order by time desc ) GROUP by time order by time ASC )) , unnest(res) as t(t1) ) where is_nan(src) = false order by unixtime desc limit 2 )

具體的告警設(shè)置如下：

---

3.硬廣時間

3.1 日志進階

這里是日志服務(wù)的各種功能的演示?日志服務(wù)整體介紹，各種Demo

原文鏈接
本文為云棲社區(qū)原創(chuàng)內(nèi)容，未經(jīng)允許不得轉(zhuǎn)載。

創(chuàng)作挑戰(zhàn)賽新人創(chuàng)作獎勵來咯，堅持創(chuàng)作打卡瓜分現(xiàn)金大獎

總結(jié)

以上是生活随笔為你收集整理的SLS机器学习最佳实战：日志聚类+异常告警的全部內(nèi)容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網(wǎng)站內(nèi)容還不錯，歡迎將生活随笔推薦給好友。