dash使用

In recent years, we have seen an explosion in the usage of Machine Learning (ML) algorithms for automating and supporting human decisions. From helping doctors in diagnosing potential diseases that could render diabetic patients blind, to supporting financial analysts in equity management, the impact of ML is irrefutable. However, as algorithms become more sophisticated, it becomes harder to understand how it makes decisions. As a result, various harmful biases could arise in critical situations, such as when ML is used to determine credit limit for credit cards or applied to facial recognition by law enforcement agencies. In order to identify and mitigate the effect of such biases before the model is deployed, it is important to leverage explainable AI (xAI) to better understand which features and factors have the most impact on the final output of the model. This information can then be utilized to help AI developers understand why a certain model might not be performing well in certain scenarios, or just in general. At the same time, such techniques could help us uncover more insights about a problem than simply using black-box models. Ultimately, such techniques can help both technical and non-technical stakeholders better understand and appreciate AI models.

近年來，我們看到了用于自動執行和支持人工決策的機器學習(ML)算法的爆炸式增長。 從幫助醫生診斷可能使糖尿病患者失明的潛在疾病 ，到在股權管理方面為財務分析師提供支持 ，ML的影響是無可辯駁的。 但是，隨著算法變得越來越復雜，變得越來越難以理解它如何做出決策。 結果，在緊急情況下可能會出現各種有害的偏見，例如當使用ML確定信用卡的信用額度或由執法機構將其應用于面部識別時 。 為了在部署模型之前識別并減輕此類偏差的影響，重要的是利用可解釋的AI(xAI)更好地了解哪些特征和因素對模型的最終輸出影響最大。 然后，可以利用此信息來幫助AI開發人員理解為什么某些模型在某些情況下或只是在一般情況下表現不佳。 同時，這些技術可以幫助我們發現更多關于問題的見解，而不僅僅是使用黑匣子模型。 最終，此類技術可以幫助技術和非技術利益相關者更好地理解和欣賞AI模型。

用Shapley值解釋黑匣子模型 (Explaining black-box models with Shapley Values)

Many researchers and software developers have been working on the subject of explainable AI systems for many years. One approach, which is called SHapley Additive exPlanations (SHAP), has been increasingly popular in the past few years. It has been used to identify chronic kidney disease, to model US mortality factors and to help anaesthesiologists during surgeries. By applying a game theory concept called Shapley values, you can interpret computer vision models, linear models, tree-based models, and more.

許多研究人員和軟件開發人員多年來一直在研究可解釋的AI系統。 在過去的幾年中，一種稱為SHapley Additive ExPlanations(SHAP)的方法越來越受歡迎。 它已被用于識別慢性腎臟疾病 ，模擬美國的死亡率因素并在手術過程中幫助麻醉師 。 通過應用稱為Shapley值的博弈論概念，您可以解釋計算機視覺模型，線性模型，基于樹的模型等。

使用Dash構建可投入生產的xAI應用 (Building a production-ready xAI app with Dash)

The dashboard was fully built in Python and runs SHAP and LightGBM in real-time. Try it out! 該儀表板完全使用Python構建，并 實時 運行 SHAP 和 LightGBM 。 試試看 ！

Let’s take, as an example, the task of predicting tips received by waiters based on features such as the total bill, the gender of the payer, the day and time, etc. A black-box model that ingests all those features just to predict the tips could be highly biased and might not be useful for business analysts trying to understand the behavior of the customers. However, by applying SHAP, we can gain more insights on the impact of each feature on the final value predicted by the model, which can be useful for understanding how the model perceives factors prone to biases, such as sex, age, race, etc.

讓我們以一個示例為例，該任務根據總賬單，付款人的性別，日期和時間等功能來預測服務員收到的小費 。一個黑盒模型會吸收所有這些功能，以便進行預測這些提示可能有很大的偏見，可能對試圖了解客戶行為的業務分析師沒有用。 但是，通過應用SHAP ，我們可以深入了解每個功能對模型預測的最終值的影響，這對于理解模型如何感知容易產生偏見的因素(例如性別，年齡，種族等)很有用。 。

In order to make such xAI methods more accessible and interactive, we used Dash to build an end-to-end application that runs an ensemble model called LightGBM on custom inputs. The dashboard and the full model are deployed on our Dash Enterprise Kubernetes servers, and both the black-box model and SHAP are running in real-time.

為了使此類xAI方法更易于訪問和交互，我們使用Dash構建了一個端到端應用程序，該應用程序在自定義輸入上運行名為LightGBM的集成模型。 儀表板和完整模型已部署在我們的Dash Enterprise Kubernetes服務器上，黑盒模型和SHAP均實時運行。

In the app, you will find controls that let you control the total bill, sex, day of the week, etc. Each one of those controls defines an input feature, and every time you update them, a new sample is given to the trained LightGBM model.

在應用程序中，您將找到控件，可用于控制總賬單，性別，星期幾等。這些控件中的每個控件都定義了一種輸入功能，每次更新時，都會向受訓人員提供一個新示例。 LightGBM模型。

However, since LightGBM is a type of gradient boosting method, it is hard to directly interpret it. Therefore, we simulated the controls to allow the app to compute the SHAP values and display them in a waterfall chart.

但是，由于LightGBM是一種梯度增強方法 ，因此很難直接對其進行解釋。 因此，我們模擬了控件，以允許該應用計算SHAP值并將其顯示在瀑布圖中 。

In addition to specifying custom model inputs, you can also select a random example from the training set. Whenever you do this, you will see the real label appear on the right side (as a scatter point). You can then tweak the feature values to see how the various SHAP values change.

除了指定自定義模型輸入之外，您還可以從訓練集中選擇一個隨機示例。 每當您執行此操作時，您都會看到實際標簽顯示在右側(作為分散點)。 然后，您可以調整要素值以查看各種SHAP值如何變化。

Moreover, you can also decide to make binary predictions (e.g. the sex of the customer) and interact with the graph using the Plotly modebar.

此外，您還可以決定進行二進制預測(例如，客戶的性別)，并使用Plotly模式欄與圖表進行交互 。

縮小Python和高級分析之間的差距 (Bridging the gap between Python and advanced analytics)

The current state-of-the-art ML algorithms (e.g. gradient boosting and neural networks) for modeling continuous and categorical features are usually written in optimized C/C++ codes, but they can be conveniently used through Python. As a result, powerful xAI libraries like SHAP are also interfaced in the same language, which lets us train and explain powerful models in just a few lines of Python code. However, although such models are popular in the ML community, considerable effort needs to be made to port them into traditional BI tools, including having to connect external servers and add third-party extensions. Furthermore, building UIs that lets you train these Python-based ML libraries can quickly become cumbersome if you are using those BI tools.

當前用于建模連續和分類特征的最新ML算法(例如，梯度增強和神經網絡)通常以優化的C / C ++代碼編寫，但是可以通過Python方便地使用它們。 結果，像SHAP這樣的功能強大的xAI庫也以相同的語言進行接口，這使我們僅用幾行Python代碼就可以訓練和解釋功能強大的模型。 但是，盡管這樣的模型在ML社區中很流行，但是需要付出很大的努力才能將它們移植到傳統的BI工具中，包括必須連接外部服務器并添加第三方擴展。 此外，如果使用那些BI工具，構建可讓您訓練這些基于Python的ML庫的UI很快就會變得很麻煩。

With Dash, you can seamlessly integrate popular and up-to-date ML libraries, which enable Dash app users to quickly answer “what if?” questions, and probe what the ML models have learned from the data. Most of the time, all you need to do is to install and freeze such libraries using pip, which is usually done in a few lines:

借助Dash，您可以無縫集成流行的和最新的ML庫，這使Dash應用程序用戶可以快速回答“如果...怎么辦”。 問題，并探討ML模型從數據中學到了什么。 大多數時候，您需要做的就是使用pip安裝和凍結此類庫，通常只需幾行即可完成：

pip install dash shap lightgbm
pip freeze > requirements.txt

When you are ready to share your dashboard, all the dependencies and deployment are handled by the Dash Enterprise App Manager.

當您準備共享儀表板時，所有依賴項和部署都由Dash Enterprise App Manager處理 。

獲得超越黑匣子預測的見解 (Gaining insights beyond black-box predictions)

A classical argument against the use of more advanced models is that, although they can improve the accuracy, the complexity makes them harder to interpret. With linear regression, you can use the coefficients to judge which features weigh more than others for making a prediction; in the case of decision trees, you can visualize how the tree splits and set thresholds for deciding the output. In the case of deep neural networks and ensemble models, you can’t visualize the tree structure nor the coefficients; however, with SHAP, it’s possible to not only explain how features generally affect the model, but also how each feature value influences the output in a specific example. For example, the model might believe that female customers tend to tip more when they are going out with a friend for dinner (on a Saturday) than when they are grabbing lunch alone (on a Thursday).

反對使用更高級的模型的經典論據是，盡管它們可以提高準確性，但其復雜性使其難以解釋。 通過線性回歸，您可以使用系數來判斷哪些特征比其他特征權重更大； 對于決策樹，您可以可視化樹的拆分方式并設置用于確定輸出的閾值 。 在深度神經網絡和集成模型的情況下，您無法可視化樹結構或系數。 但是，使用SHAP，不僅可以說明要素通常如何影響模型，還可以在特定示例中說明每個要素值如何影響輸出。 例如，該模型可能認為，女性顧客與朋友外出吃飯(星期六)時比他們單身午餐(星期四)時要多付小費。

Left: Thursday lunch alone. Right: Saturday Dinner with friends.左：僅星期四午餐。 右：與朋友共進晚餐。

Such insight could either lead to a stronger understanding of customer behavior if it is backed by additional studies, or it could reveal some degree of systematic bias that would not have been otherwise uncovered without SHAP. With Dash, we make it easier to build and deploy custom dashboards that let you interpret all sorts of ML models, whether they are trained on predicting tips, or other types of data.

如果有其他研究的支持，則這種洞察力可能導致對客戶行為的更深入了解，或者可能揭示某種程度的系統偏差，而如果沒有SHAP，這是無法發現的。 借助Dash，我們可以更輕松地構建和部署自定義儀表板，使您可以解釋各種ML模型，無論它們是根據預測技巧還是其他類型的數據進行訓練的 。

將Python的力量掌握在業務用戶手中 (Putting the power of Python in the hands of business users)

At Plotly, we are working on keeping Dash flexible, yet easy-to-use for building ML apps and dashboards. For this reason, we built our app with components from the Dash Enterprise Design Kit, which makes it easy to tailor apps to meet style guidelines without delving into HTML and CSS. For example, if you don’t like the color of the bars in the default waterfall charts, you can easily change it with the Design Kit Theme Editor.

在Plotly，我們致力于保持Dash的靈活性，但易于使用，用于構建ML應用程序和儀表板。 因此，我們使用Dash Enterprise Design Kit中的組件構建了應用程序，從而可以輕松定制應用程序以滿足樣式準則，而無需研究HTML和CSS。 例如，如果您不喜歡默認瀑布圖中的條形顏色，則可以使用設計工具包主題編輯器輕松更改它。

Furthermore, new features in Dash like pattern-matching callbacks let you simplify the process of creating callbacks. As a result, you can create very complex callbacks between components with very little effort. As an example, there are six controls in our app (one for each input feature), but we only need to specify one Input and one State to our callback to control all the components at the same time:

此外，Dash中的新功能(例如模式匹配回調)使您可以簡化創建回調的過程。 結果，您可以毫不費力地在組件之間創建非常復雜的回調。 例如，我們的應用程序中有六個控件(每個輸入功能一個)，但是我們只需為回調指定一個Input和一個State即可同時控制所有組件：

Then, in one line, we were able to construct a dictionary where the keys are the feature names and the values are what we will input to the model. From then on, it’s easy to process the dictionary in the input format most suitable to your model.

然后，我們可以在一行中構造一個字典，其中的鍵是要素名稱，而值是我們將輸入到模型的值。 從那時起，很容易以最適合您的模型的輸入格式處理字典。

Are you interested in creating similar apps using state-of-the-art models and xAI algorithms? Contact us to learn more about how Dash Enterprise can help you build, design, and deploy ML dashboards — with no compromise.

您是否對使用最新模型和xAI算法創建類似應用感興趣？ 請與我們聯系，以了解有關Dash Enterprise如何能夠幫助您構建，設計和部署ML儀表板的更多信息，而不會做出任何妥協。

翻譯自: https://medium.com/plotly/building-and-deploying-explainable-ai-dashboards-using-dash-and-shap-8e0a0a45beb6

dash使用

總結

以上是生活随笔為你收集整理的dash使用_使用Dash和SHAP构建和部署可解释的AI仪表盘的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。