Python梯度下降法實現(xiàn)二元邏輯回歸

二元邏輯回歸假設(shè)函數(shù)

定義當(dāng)函數(shù)值大于等于0.5時，結(jié)果為1，當(dāng)函數(shù)值小于0.5時，結(jié)果為0.函數(shù)的值域是(0, 1)。

二元邏輯回歸的損失函數(shù)

上圖為二元邏輯回歸的概率公式，則代價函數(shù)可以表示為

損失函數(shù)求偏倒數(shù)為

可以發(fā)現(xiàn)和線性回歸的結(jié)果是一樣的，只不過是假設(shè)函數(shù)h發(fā)生了變化。

正則化

為了避免過擬合，通常在代價函數(shù)后加一個正則化項，針對二元邏輯回歸，填加正則化項，
$\frac{1}{2}{\sum }_{j=1}^n{\theta }_j^2$
這樣，隨時函數(shù)后就應(yīng)該添加一項
$\frac{1}{m}{\sum }_{j=1}^n{\theta }_j$

Python代碼實現(xiàn)

import numpy as np import matplotlib.pyplot as plt# 特征數(shù)目 n = 2 # 構(gòu)造訓(xùn)練集 X1 = np.arange(-2., 2., 0.02) m = len(X1)X2 = X1 + np.random.randn(m) # print(X1, X2) one = np.full(m, 1.0) Y = 1 / (np.full(m, 1.0) + np.exp(-(0.1 * np.random.randn(m)-np.full(m, 0.6) + 5*X1 + 2 * X2))) # Y Y = np.array([np.int(round(i)) for i in Y])# 梯度下降法 theta = np.random.rand(n+1) print(theta) X = np.vstack([np.full(m, 1), X1, X2]).T # 前一次的theta pre = np.zeros(n + 1) diff = 1e-10 max_loop = 10000 alpha = 0.01 lamda = 1 now_diff = 0 while max_loop > 0:#sum = np.zeros(n + 1)sum = np.sum([(1 / (1. + np.exp(- np.dot(theta, X[i]))) - Y[i])*X[i] for i in range(m)], axis=0)# for i in range(m):# sum += (1 / (1. + np.exp(- np.dot(theta, X[i]))) - Y[i])*X[i]theta = theta - (alpha * sum + alpha * lamda * theta)print("還差 %d 次" % max_loop, "theta = ", theta)now_diff = np.linalg.norm(theta - pre)if(now_diff <= diff):breakpre = thetamax_loop -= 1# 打印 print("find theta : ", theta, "now_diff : ", now_diff)# 畫出平測試?yán)訄D X_1 = np.array([(X1[i], X2[i]) for i in range(m) if Y[i] == 1]) X_0 = np.array([(X1[i], X2[i]) for i in range(m) if Y[i] == 0]) plt.scatter(X_1[:, 0], X_1[:, 1], c='r', marker='o') plt.scatter(X_0[:, 0], X_0[:, 1], c='g', marker='v')# 畫出求得的模型圖 point1 = np.arange(-2, 2, 0.02) point2 = (theta[0] + theta[1] * point1)/(-theta[2])plt.plot(point1, point2) plt.xlabel('X1') plt.ylabel('X2') plt.show()

效果圖如下

總結(jié)

以上是生活随笔為你收集整理的Python梯度下降法实现二元逻辑回归的全部內(nèi)容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網(wǎng)站內(nèi)容還不錯，歡迎將生活随笔推薦給好友。