mlflow

This is the first article in my MLflow tutorial series:

這是我的MLflow教程系列的第一篇文章：

Setup MLflow in Production (you are here!)

在生產(chǎn)環(huán)境中設(shè)置MLflow (您在這里！)

MLflow: Basic logging functions

MLflow：基本日志記錄功能

MLflow logging for TensorFlow

TensorFlow的MLflow日志記錄

MLflow Projects

MLflow項(xiàng)目

Retrieving the best model using Python API for MLflow

使用適用于MLflow的Python API檢索最佳模型

Serving a model using MLflow

使用MLflow服務(wù)模型

MLflow is an open-source platform for machine learning lifecycle management. Recently, I set up MLflow in production with a Postgres database as a Tracking Server and SFTP for the transfer of artifacts over the network. It took me about 2 weeks to get all the components right but this post would help you setup of MLflow in a production environment in about 10 minutes.

MLflow是用于機(jī)器學(xué)習(xí)生命周期管理的開源平臺(tái)。 最近，我在生產(chǎn)中設(shè)置了MLflow，并使用Postgres數(shù)據(jù)庫(kù)作為Tracking Server和SFTP來通過網(wǎng)絡(luò)傳輸工件。 我花了大約2周的時(shí)間才能正確安裝所有組件，但是這篇文章將幫助您在大約10分鐘的生產(chǎn)環(huán)境中設(shè)置MLflow。

要求 (Requirements)

• Anaconda

  水蟒

跟蹤服務(wù)器設(shè)置 (Tracking Server Setup)

Tracking Server stores the metadata that you see in the MLflow UI. First, let’s create a new Conda environment:

跟蹤服務(wù)器存儲(chǔ)您在MLflow UI中看到的元數(shù)據(jù)。 首先，讓我們創(chuàng)建一個(gè)新的Conda環(huán)境：

conda create -n mlflow_env
conda activate mlflow_env

Install the MLflow and PySFTP libraries:

安裝MLflow和PySFTP庫(kù)：

conda install python
pip install mlflow
pip install pysftp

Our Tracking Server uses a Postgres database as a backend for storing the metadata. So let’s install PostgreSQL:

我們的Tracking Server使用Postgres數(shù)據(jù)庫(kù)作為后端來存儲(chǔ)元數(shù)據(jù)。 因此，讓我們安裝PostgreSQL：

apt-get install postgresql postgresql-contrib postgresql-server-dev-all

Next, we will create the admin user and a database for the Tracking Server

接下來，我們將為跟蹤服務(wù)器創(chuàng)建管理員用戶和數(shù)據(jù)庫(kù)

sudo -u postgres psql

In the psql console:

在psql控制臺(tái)中：

CREATE DATABASE mlflow_db;
CREATE USER mlflow_user WITH ENCRYPTED PASSWORD 'mlflow';
GRANT ALL PRIVILEGES ON DATABASE mlflow_db TO mlflow_user;

As we’ll need to interact with Postgres from Python, it is needed to install the psycopg2 library. However, to ensure a successful installation we need to install the GCC Linux package before:

由于我們需要與Python中的Postgres進(jìn)行交互，因此需要安裝psycopg2庫(kù)。 但是，為了確保安裝成功，我們需要在安裝GCC Linux軟件包之前：

sudo apt install gcc
pip install psycopg2-binary

If you would like to connect to the PostgreSQL Server remotely or would like to give its access to the users. You can

如果您想遠(yuǎn)程連接到PostgreSQL服務(wù)器或希望將其訪問權(quán)限授予用戶。 您可以

cd /var/lib/pgsql/data

Then add the following line at the end of the postgresql.conf file.

然后在postgresql.conf文件的末尾添加以下行。

listen_addresses = '*'

You can then specify a remote IP from which you want to allow connection to the PostgreSQL Server, by adding the following line at the end of the pg_hba.conf file

然后，您可以通過在pg_hba.conf文件末尾添加以下行來指定要允許從其連接到PostgreSQL服務(wù)器的遠(yuǎn)程IP。

host all all 10.10.10.187/32 trust

where 10.10.10.187/32 is the remote IP. To allow connection from any IP, use 0.0.0.0/0 instead. Then restart the PostgreSQL Server to apply the changes.

其中10.10.10.187/32是遠(yuǎn)程IP。 要允許來自任何IP的連接，請(qǐng)改用0.0.0.0/0 。 然后重新啟動(dòng)PostgreSQL服務(wù)器以應(yīng)用更改。

service postgresql restart

The next step is creating a directory for our Tracking Server to log the Machine Learning models and other artifacts. Remember that the Postgres database is only used for storing metadata regarding those models. This directory is called artifact URI.

下一步是為Tracking Server創(chuàng)建一個(gè)目錄，以記錄機(jī)器學(xué)習(xí)模型和其他工件。 請(qǐng)記住，Postgres數(shù)據(jù)庫(kù)僅用于存儲(chǔ)有關(guān)那些模型的元數(shù)據(jù)。 此目錄稱為工件URI。

mkdir ~/mlflow/mlruns

Create a logging directory.

創(chuàng)建一個(gè)日志目錄。

mkdir ~/mlflow/mllogs

You can run the Tracking Server with the following command. But as soon as you do Ctrl-C or exit the terminal the server stops.

您可以使用以下命令運(yùn)行Tracking Server。 但是，一旦您執(zhí)行Ctrl-C或退出終端，服務(wù)器就會(huì)停止。

mlflow server --backend-store-uri postgresql://mlflow_user:mlflow@localhost/mlflow_db --default-artifact-root sftp://mlflow_user@<hostname_of_server>:~/mlflow/mlruns -h 0.0.0.0 -p 8000

If you want the Tracking server to be up and running after restarts and be resilient to failures, it is very useful to run it as a systemd service.

如果您希望跟蹤服務(wù)器在重新啟動(dòng)后能夠啟動(dòng)并運(yùn)行，并且能夠?qū)收线M(jìn)行恢復(fù)，那么將其作為systemd服務(wù)運(yùn)行非常有用。

You need to go into the /etc/systemd/system directory and create a new file called mlflow-tracking.service with the following content:

您需要進(jìn)入/ etc / systemd / system目錄，并創(chuàng)建一個(gè)名為mlflow-tracking.service的新文件，其內(nèi)容如下：

[Unit]
Description=MLflow Tracking Server
After=network.target[Service]
Restart=on-failure
RestartSec=30
StandardOutput=file:/path_to_your_logging_folder/stdout.log
StandardError=file:/path_to_your_logging_folder/stderr.log
User=root
ExecStart=/bin/bash -c 'PATH=/path_to_your_conda_installation/envs/mlflow_env/bin/:$PATH exec mlflow server --backend-store-uri postgresql://mlflow_user:mlflow@localhost/mlflow_db --default-artifact-root sftp://mlflow_user@<hostname_of_server>:~/mlflow/mlruns -h 0.0.0.0 -p 8000'[Install]
WantedBy=multi-user.target

Activate and enable the above service with the following commands:

使用以下命令激活并啟用上述服務(wù)：

sudo systemctl daemon-reload
sudo systemctl enable mlflow-tracking
sudo systemctl start mlflow-tracking

Check that everything worked as expected with the following command:

使用以下命令檢查一切是否按預(yù)期進(jìn)行：

sudo systemctl status mlflow-tracking

You should see an output similar to this:

您應(yīng)該看到類似于以下的輸出：

Systemd unit running系統(tǒng)單元運(yùn)行

Create user for the server named mlflow_user and make mlflow directory as the working directory for this user. Then create an ssh-key pair in the .ssh directory for the mlflow_user (/mlflow/.ssh in our case). Put the public key in the authorized_keys file and share the private key with the users.

為名為mlflow_user的服務(wù)器創(chuàng)建用戶，并將mlflow目錄作為該用戶的工作目錄。 然后創(chuàng)建.ssh目錄中的SSH密鑰對(duì)的mlflow_user(/mlflow/.ssh在我們的例子)。 將公鑰放入authorized_keys文件中，并與用戶共享私鑰。

Additionally, for the MLflow UI to be able to read the artifacts, copy the private key to /root/.ssh/ as well.

另外，為了使MLflow UI能夠讀取工件，請(qǐng)將私鑰也復(fù)制到/root/.ssh/ 。

Next, we need to create the Host Key for the server manually using this command:

接下來，我們需要使用以下命令為服務(wù)器手動(dòng)創(chuàng)建主機(jī)密鑰：

cd /root/.ssh
ssh-keyscan -H <hostname_of_server> >> known_hosts

You can now restart the machine and the MLflow Tracking Server will be up and running after this restart.

您現(xiàn)在可以重新啟動(dòng)計(jì)算機(jī)，并且重新啟動(dòng)后MLflow Tracking Server將啟動(dòng)并運(yùn)行。

在客戶端計(jì)算機(jī)上 (On the client machines)

In order to start tracking everything under the production Tracking Server, it is necessary to set the following environment variable in your .bashrc.

為了開始跟蹤生產(chǎn)跟蹤服務(wù)器下的所有內(nèi)容，必須在.bashrc中設(shè)置以下環(huán)境變量。

export MLFLOW_TRACKING_URI='http://<hostname_of_server>:8000'

Do not forget to source your .bashrc file!

不要忘記提供您的.bashrc文件！

. ~/.bashrc

Make sure you install pip packages for mlflow and pysftp in your environment (pysftp is required to facilitate the transfer of artifacts to the production server).

確保在您的環(huán)境中安裝了用于mlflow和pysftp的 pip軟件包(需要pysftp才能將工件傳輸?shù)缴a(chǎn)服務(wù)器)。

pip install mlflow
pip install pysftp

To be able to authenticate the pysftp transfers, put the private key generated on the Production Server in the .ssh directory of your local machine . Then do

為了能夠驗(yàn)證pysftp傳輸，請(qǐng)將生產(chǎn)服務(wù)器上生成的私鑰放在本地計(jì)算機(jī)的.ssh目錄中。 然后做

ssh <hostname_of_server>

When prompted to save <hostname_of_server> as a known host, answer yes.

當(dāng)提示您將<hostname_of_server>保存為已知主機(jī)時(shí)，請(qǐng)回答yes 。

You can access MLflow UI at http://<hostname_of_server>:8000

您可以通過以下 網(wǎng)址訪問MLflow UI ： http：// <hostname_of_server> ：8000

The Mlflow UIMlflow用戶界面

Run a sample machine learning model from the internet to check whether MLflow can track the runs.

從Internet運(yùn)行示例機(jī)器學(xué)習(xí)模型以檢查MLflow是否可以跟蹤運(yùn)行。

mlflow run git@github.com:databricks/mlflow-example.git -P alpha=0.5

In the next post, I’ll speak about basic MLflow logging functions

在下一篇文章中 ，我將介紹基本的MLflow日志記錄功能

翻譯自: https://towardsdatascience.com/setup-mlflow-in-production-d72aecde7fef

mlflow

總結(jié)

以上是生活随笔為你收集整理的mlflow_在生产中设置MLflow的全部?jī)?nèi)容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網(wǎng)站內(nèi)容還不錯(cuò)，歡迎將生活随笔推薦給好友。