Lesson 12.4 邏輯回歸建模實(shí)驗(yàn)

??接下來(lái)進(jìn)行邏輯回歸的建模實(shí)驗(yàn)，首先需要導(dǎo)入相關(guān)庫(kù)和自定義的模塊。

# 隨機(jī)模塊 import random# 繪圖模塊 import matplotlib as mpl import matplotlib.pyplot as plt# numpy import numpy as np# pytorch import torch from torch import nn,optim import torch.nn.functional as F from torch.utils.data import Dataset,TensorDataset,DataLoader from torch.utils.tensorboard import SummaryWriter# 自定義模塊 from torchLearning import *# 導(dǎo)入以下包從而使得可以在jupyter中的一個(gè)cell輸出多個(gè)結(jié)果 from IPython.core.interactiveshell import InteractiveShell InteractiveShell.ast_node_interactivity = "all"

當(dāng)然，我們可以通過查看我們自定義的函數(shù)幫助文檔來(lái)驗(yàn)證是否導(dǎo)入成功

tensorGenCla? #Signature: #tensorGenCla( # num_examples=500, # num_inputs=2, # num_class=3, # deg_dispersion=[4, 2], # bias=False, #) #Docstring: #分類數(shù)據(jù)集創(chuàng)建函數(shù)。 # #:param num_examples: 每個(gè)類別的數(shù)據(jù)數(shù)量 #:param num_inputs: 數(shù)據(jù)集特征數(shù)量 #:param num_class：數(shù)據(jù)集標(biāo)簽類別總數(shù) #:param deg_dispersion：數(shù)據(jù)分布離散程度參數(shù)，需要輸入一個(gè)列表，其中第一個(gè)參數(shù)表示每個(gè)類別數(shù)組均值的參考、第二個(gè)參數(shù)表示隨機(jī)數(shù)組標(biāo)準(zhǔn)差。 #:param bias：建立模型邏輯回歸模型時(shí)是否帶入截距 #:return: 生成的特征張量和標(biāo)簽張量，其中特征張量是浮點(diǎn)型二維數(shù)組，標(biāo)簽張量是長(zhǎng)正型二維數(shù)組。 #File: f:\code file\pytorch實(shí)戰(zhàn)\torchlearning.py #Type: function

一、邏輯回歸手動(dòng)實(shí)現(xiàn)

??接下來(lái)，嘗試手動(dòng)實(shí)現(xiàn)二分類邏輯回歸，還是根據(jù)此前介紹的深度學(xué)習(xí)建模流程進(jìn)行手動(dòng)實(shí)現(xiàn)。

1.生成數(shù)據(jù)集

??利用此前創(chuàng)建的tensorGenCla進(jìn)行二分類數(shù)據(jù)集的創(chuàng)建

# 設(shè)置隨機(jī)數(shù)種子 torch.manual_seed(420) # 創(chuàng)建數(shù)據(jù)集 features, labels = tensorGenCla(num_class=2, bias=True)# 可視化展示 plt.scatter(features[:, 0], features[:, 1], c = labels)

features #tensor([[-2.0141, -0.9911, 1.0000], # [-0.6593, -2.7657, 1.0000], # [-1.9395, -1.2347, 1.0000], # ..., # [ 2.9623, 2.0861, 1.0000], # [ 0.4535, -0.2140, 1.0000], # [-2.6681, 3.3935, 1.0000]])

2.建模過程

• Stage 1.模型選擇

針對(duì)二分類問題（0-1問題），我們可以簡(jiǎn)單的輸出一個(gè)結(jié)果，作為標(biāo)簽取值為1的概率，因此模型結(jié)構(gòu)如下

對(duì)應(yīng)的可定義如下模型

1）激活函數(shù)

def sigmoid(z):return 1/(1+torch.exp(-z))

2）邏輯回歸模型

def logistic(X, w):return sigmoid(torch.mm(X, w))

3）輔助函數(shù)

??由于sigmoid輸出結(jié)果是連續(xù)值，而用于二分類判別時(shí)，我們需要將連續(xù)數(shù)值轉(zhuǎn)化為所判定的類別，可定義對(duì)應(yīng)分類函數(shù)如下：

def cal(sigma, p=0.5):return((sigma >= p).float()) a = torch.randint(10, (5, )) a #tensor([8, 0, 4, 4, 2]) a >= 5 #tensor([ True, False, False, False, False]) (a >= 5).float() #tensor([1., 0., 0., 0., 0.])

另外，對(duì)分類模型，我們往往會(huì)通過準(zhǔn)確率判別模型效果，因此還需要定義準(zhǔn)確率函數(shù)

def accuracy(sigma, y):acc_bool = cal(sigma).flatten() == y.flatten()acc = torch.mean(acc_bool.float())return(acc) p = torch.tensor([1, 1, 2]) == torch.tensor([1, 2, 2]) p #tensor([ True, False, True]) p.float() torch.mean(p.float()) #tensor([1., 0., 1.]) #tensor(0.6667)

• Stage 2.定義損失函數(shù)

def cross_entropy(sigma, y):return(-(1/y.numel())*torch.sum((1-y)*torch.log(1-sigma)+y*torch.log(sigma)))

• Stage 3.定義優(yōu)化方法

def sgd(params, lr):params.data -= lr * params.grad params.grad.zero_()

• Stage 4.訓(xùn)練模型

features #tensor([[-2.0141, -0.9911, 1.0000], # [-0.6593, -2.7657, 1.0000], # [-1.9395, -1.2347, 1.0000], # ..., # [ 2.9623, 2.0861, 1.0000], # [ 0.4535, -0.2140, 1.0000], # [-2.6681, 3.3935, 1.0000]]) # 設(shè)置隨機(jī)數(shù)種子 torch.manual_seed(420) # 初始化核心參數(shù) batch_size = 10 # 每一個(gè)小批的數(shù)量 lr = 0.03 # 學(xué)習(xí)率 num_epochs = 3 # 訓(xùn)練過程遍歷幾次數(shù)據(jù) w = torch.ones(3, 1, requires_grad = True) # 隨機(jī)設(shè)置初始權(quán)重# 參與訓(xùn)練的模型方程 net = logistic # 使用邏輯回歸方程 loss = cross_entropy # 交叉熵?fù)p失函數(shù)# 訓(xùn)練過程 for epoch in range(num_epochs):for X, y in data_iter(batch_size, features, labels):l = loss(net(X, w), y)l.backward()sgd(w, lr)train_acc = accuracy(net(features, w), labels)print('epoch %d, accuracy %f' % (epoch + 1, train_acc)) #<torch._C.Generator at 0x1d92af10cd0> #epoch 1, accuracy 0.904000 #epoch 2, accuracy 0.907000 #epoch 3, accuracy 0.914000 w #tensor([[1.0069], # [0.9753], # [0.5765]], requires_grad=True)

3.模型調(diào)試

根據(jù)上述迭代三輪返回的準(zhǔn)確率，能夠看出整體還在增加，讓我們?cè)俣嗟鷰纵啿榭唇Y(jié)果

# 設(shè)置隨機(jī)數(shù)種子 torch.manual_seed(420) # 迭代輪數(shù) num_epochs = 20# 設(shè)置初始權(quán)重 w = torch.ones(3, 1, requires_grad = True) # 設(shè)置列表容器 train_acc = []# 執(zhí)行迭代 for i in range(num_epochs):for epoch in range(i):for X, y in data_iter(batch_size, features, labels):l = loss(net(X, w), y)l.backward()sgd(w, lr)train_acc.append(accuracy(net(features, w), labels))# 繪制圖像查看準(zhǔn)確率變化情況 plt.plot(list(range(num_epochs)), train_acc)

train_acc #[tensor(0.8970), # tensor(0.9040), # tensor(0.9120), # tensor(0.9170), # tensor(0.9190), # tensor(0.9150), # tensor(0.9150), # tensor(0.9150), # tensor(0.9150), # tensor(0.9150), # tensor(0.9150), # tensor(0.9150), # tensor(0.9150), # tensor(0.9140), # tensor(0.9140), # tensor(0.9150), # tensor(0.9150), # tensor(0.9150), # tensor(0.9150), # tensor(0.9150)]

能夠看出，增加迭代次數(shù)之后，損失函數(shù)逼近最小值點(diǎn)，每次迭代梯度取值較小，整體準(zhǔn)確率趨于平穩(wěn)

當(dāng)然，如果我們將數(shù)據(jù)難度增加，也就是增加數(shù)據(jù)的離散程度，是否會(huì)對(duì)模型結(jié)果造成影響

tensorGenCla? #Signature: #tensorGenCla( # num_examples=500, # num_inputs=2, # num_class=3, # deg_dispersion=[4, 2], # bias=False, #) #Docstring: #分類數(shù)據(jù)集創(chuàng)建函數(shù)。 # #:param num_examples: 每個(gè)類別的數(shù)據(jù)數(shù)量 #:param num_inputs: 數(shù)據(jù)集特征數(shù)量 #:param num_class：數(shù)據(jù)集標(biāo)簽類別總數(shù) #:param deg_dispersion：數(shù)據(jù)分布離散程度參數(shù)，需要輸入一個(gè)列表，其中第一個(gè)參數(shù)表示每個(gè)類別數(shù)組均值的參考、第二個(gè)參數(shù)表示隨機(jī)數(shù)組標(biāo)準(zhǔn)差。 #:param bias：建立模型邏輯回歸模型時(shí)是否帶入截距 #:return: 生成的特征張量和標(biāo)簽張量，其中特征張量是浮點(diǎn)型二維數(shù)組，標(biāo)簽張量是長(zhǎng)正型二維數(shù)組。 #File: f:\code file\pytorch實(shí)戰(zhàn)\torchlearning.py #Type: function torch.manual_seed(420) features, labels = tensorGenCla(num_class=2, bias=True, deg_dispersion=[4, 4])# 可視化展示 plt.scatter(features[:, 0], features[:, 1], c = labels)

# 設(shè)置隨機(jī)數(shù)種子 torch.manual_seed(420) # 迭代輪數(shù) num_epochs = 20# 設(shè)置初始權(quán)重 w = torch.zeros(3, 1, requires_grad = True) # 設(shè)置列表容器 train_acc = []# 執(zhí)行迭代 for i in range(num_epochs):for epoch in range(i):for X, y in data_iter(batch_size, features, labels):l = loss(net(X, w), y)l.backward()sgd(w, lr)train_acc.append(accuracy(net(features, w), labels))# 繪制圖像查看準(zhǔn)確率變化情況 plt.plot(list(range(num_epochs)), train_acc)

能夠發(fā)現(xiàn)，隨著數(shù)據(jù)情況變復(fù)雜，相同模型的準(zhǔn)確率發(fā)生了很大的變化。后續(xù)我們將介紹模型優(yōu)化的相關(guān)方法，此處先熟悉代碼過程和基本結(jié)論。

train_acc #[tensor(0.5000), # tensor(0.7320), # tensor(0.7310), # tensor(0.7330), # tensor(0.7320), # tensor(0.7300), # tensor(0.7290), # tensor(0.7310), # tensor(0.7290), # tensor(0.7310), # tensor(0.7320), # tensor(0.7260), # tensor(0.7320), # tensor(0.7340), # tensor(0.7270), # tensor(0.7330), # tensor(0.7320), # tensor(0.7270), # tensor(0.7310), # tensor(0.7280)]

二、邏輯回歸的快速實(shí)現(xiàn)

1.構(gòu)建模型

??接下來(lái)，我們練習(xí)使用PyTorch中的函數(shù)和類，進(jìn)行邏輯回歸的快速構(gòu)建。

• 定義核心參數(shù)

batch_size = 10 # 每一個(gè)小批的數(shù)量 lr = 0.03 # 學(xué)習(xí)率 num_epochs = 3 # 訓(xùn)練過程遍歷幾次數(shù)據(jù)

• 數(shù)據(jù)準(zhǔn)備

# 設(shè)置隨機(jī)數(shù)種子 torch.manual_seed(420) # 創(chuàng)建數(shù)據(jù)集 features, labels = tensorGenCla(num_class=2) labels = labels.float() # 損失函數(shù)要求標(biāo)簽也必須是浮點(diǎn)型 data = TensorDataset(features, labels) batchData = DataLoader(data, batch_size = batch_size, shuffle = True) #<torch._C.Generator at 0x1d92af10cd0> features #tensor([[-2.0141, -0.9911], # [-0.6593, -2.7657], # [-1.9395, -1.2347], # ..., # [ 2.9623, 2.0861], # [ 0.4535, -0.2140], # [-2.6681, 3.3935]])

• Stage 1.定義模型

class logisticR(nn.Module):def __init__(self, in_features=2, out_features=1): # 定義模型的點(diǎn)線結(jié)構(gòu)super(logisticR, self).__init__()self.linear = nn.Linear(in_features, out_features)def forward(self, x): # 定義模型的正向傳播規(guī)則out = self.linear(x) return out# 實(shí)例化模型和 logic_model = logisticR()

• Stage 2.定義損失函數(shù)

criterion = nn.BCEWithLogitsLoss()

• Stage 3.定義優(yōu)化方法

optimizer = optim.SGD(logic_model.parameters(), lr = lr)

• Stage 4.模型訓(xùn)練

def fit(net, criterion, optimizer, batchdata, epochs):for epoch in range(epochs):for X, y in batchdata:zhat = net.forward(X)loss = criterion(zhat, y)optimizer.zero_grad()loss.backward()optimizer.step()

和線性回歸相同，由于上述模型只有一層，因此也可以通過nn.Linear(2, 1)函數(shù)直接建模。由于我們所采用的BCEWithLogitsLoss類進(jìn)行的損失函數(shù)求解，該類會(huì)自動(dòng)對(duì)輸入對(duì)象進(jìn)行sigmoid轉(zhuǎn)化，因此上述過程和線性回歸過程沒有區(qū)別。

接下來(lái)，即可執(zhí)行模型訓(xùn)練

# 設(shè)置隨機(jī)數(shù)種子 torch.manual_seed(420) fit(net = logic_model, criterion = criterion, optimizer = optimizer, batchdata = batchData, epochs = num_epochs) #<torch._C.Generator at 0x1d92af10cd0>

查看模型訓(xùn)練結(jié)果

logic_model #logisticR( # (linear): Linear(in_features=2, out_features=1, bias=True) #) # 查看模型參數(shù) list(logic_model.parameters()) #[Parameter containing: # tensor([[0.8394, 0.8016]], requires_grad=True), # Parameter containing: # tensor([-0.2617], requires_grad=True)] # 計(jì)算交叉熵?fù)p失 criterion(logic_model(features), labels) #tensor(0.2293, grad_fn=<BinaryCrossEntropyWithLogitsBackward>) def acc_zhat(zhat, y):"""輸入為線性方程計(jì)算結(jié)果，輸出為邏輯回歸準(zhǔn)確率的函數(shù):param zhat：線性方程輸出結(jié)果 :param y: 數(shù)據(jù)集標(biāo)簽張量:return：準(zhǔn)確率 """sigma = sigmoid(zhat)return accuracy(sigma, y) acc_zhat(logic_model(features), labels) #tensor(0.9130)

2.模型調(diào)試

同樣，我們首先嘗試多迭代幾次，看下準(zhǔn)確率如何發(fā)生變化

#創(chuàng)建數(shù)據(jù) torch.manual_seed(420) features, labels = tensorGenCla(num_class=2) labels = labels.float() data = TensorDataset(features, labels) batchData = DataLoader(data, batch_size = batch_size, shuffle = True) #<torch._C.Generator at 0x1d92af10cd0> # 設(shè)置隨機(jī)數(shù)種子 torch.manual_seed(420) # 初始化核心參數(shù) num_epochs = 20 LR1 = logisticR() cr1 = nn.BCEWithLogitsLoss() op1 = optim.SGD(LR1.parameters(), lr = lr)# 創(chuàng)建列表容器 train_acc = []# 執(zhí)行建模 for epochs in range(num_epochs):fit(net = LR1, criterion = cr1, optimizer = op1, batchdata = batchData, epochs = epochs)epoch_acc = acc_zhat(LR1(features), labels)train_acc.append(epoch_acc)# 繪制圖像查看準(zhǔn)確率變化情況 plt.plot(list(range(num_epochs)), train_acc)

train_acc #[tensor(0.5170), # tensor(0.9040), # tensor(0.9180), # tensor(0.9180), # tensor(0.9150), # tensor(0.9150), # tensor(0.9150), # tensor(0.9150), # tensor(0.9150), # tensor(0.9150), # tensor(0.9150), # tensor(0.9150), # tensor(0.9150), # tensor(0.9150), # tensor(0.9140), # tensor(0.9150), # tensor(0.9150), # tensor(0.9150), # tensor(0.9150), # tensor(0.9150)]

接下來(lái)，和此前一樣，接下來(lái)嘗試增加數(shù)據(jù)難度來(lái)測(cè)試模型分類性能

#創(chuàng)建數(shù)據(jù) torch.manual_seed(420) features, labels = tensorGenCla(num_class=2, deg_dispersion=[4, 4]) labels = labels.float() data = TensorDataset(features, labels) batchData = DataLoader(data, batch_size = batch_size, shuffle = True)plt.scatter(features[:, 0], features[:, 1], c = labels)

#創(chuàng)建數(shù)據(jù) torch.manual_seed(420) # 數(shù)據(jù)封裝與加載 data = TensorDataset(features, labels) batchData = DataLoader(data, batch_size = batch_size, shuffle = True)# 初始化核心參數(shù) num_epochs = 20 LR1 = logisticR() cr1 = nn.BCEWithLogitsLoss() op1 = optim.SGD(LR1.parameters(), lr = lr)# 創(chuàng)建列表容器 train_acc = []# 執(zhí)行建模 for epochs in range(num_epochs):fit(net = LR1, criterion = cr1, optimizer = op1, batchdata = batchData, epochs = epochs)epoch_acc = acc_zhat(LR1(features), labels)train_acc.append(epoch_acc)# 繪制圖像查看準(zhǔn)確率變化情況 plt.plot(list(range(num_epochs)), train_acc)

train_acc #[tensor(0.4970), # tensor(0.7250), # tensor(0.7260), # tensor(0.7290), # tensor(0.7300), # tensor(0.7260), # tensor(0.7260), # tensor(0.7300), # tensor(0.7260), # tensor(0.7250), # tensor(0.7290), # tensor(0.7310), # tensor(0.7320), # tensor(0.7340), # tensor(0.7300), # tensor(0.7300), # tensor(0.7320), # tensor(0.7290), # tensor(0.7290), # tensor(0.7330)]

和此前一樣，準(zhǔn)確率在0.7-0.75之間徘徊。

【補(bǔ)充】實(shí)例化模型時(shí)參數(shù)的隨機(jī)取值

在手動(dòng)創(chuàng)建模型類之后，每次實(shí)例化都會(huì)隨機(jī)生成一組參數(shù)值

class logisticR(nn.Module):def __init__(self, in_features=2, out_features=1): # 定義模型的點(diǎn)線結(jié)構(gòu)super(logisticR, self).__init__()self.linear = nn.Linear(in_features, out_features)def forward(self, x): # 定義模型的正向傳播規(guī)則out = self.linear(x) return outlist(logisticR().parameters()) #[Parameter containing: # tensor([[-0.2511, -0.1878]], requires_grad=True), # Parameter containing: # tensor([-0.6789], requires_grad=True)] list(logisticR().parameters()) #[Parameter containing: # tensor([[0.0188, 0.0345]], requires_grad=True), # Parameter containing: # tensor([-0.1411], requires_grad=True)]

若需要完全復(fù)現(xiàn)模型訓(xùn)練過程，則需要在實(shí)例化之前設(shè)置隨機(jī)數(shù)種子，或者在上一個(gè)隨機(jī)數(shù)種子之前規(guī)定有限次的隨機(jī)次數(shù)。

• 隨機(jī)數(shù)種子可以在全域發(fā)揮作用

torch.manual_seed(420) #<torch._C.Generator at 0x1d92af10cd0> list(logisticR().parameters()) #[Parameter containing: # tensor([[ 0.4318, -0.4256]], requires_grad=True), # Parameter containing: # tensor([0.6730], requires_grad=True)] list(logisticR().parameters()) #[Parameter containing: # tensor([[-0.5617, -0.2157]], requires_grad=True), # Parameter containing: # tensor([-0.4873], requires_grad=True)] torch.manual_seed(420) #<torch._C.Generator at 0x1d92af10cd0> list(logisticR().parameters()) #[Parameter containing: # tensor([[ 0.4318, -0.4256]], requires_grad=True), # Parameter containing: # tensor([0.6730], requires_grad=True)] list(logisticR().parameters()) #[Parameter containing: # tensor([[-0.5617, -0.2157]], requires_grad=True), # Parameter containing: # tensor([-0.4873], requires_grad=True)]

• torch.manual_seed不會(huì)對(duì)random中的隨機(jī)過程造成影響

l = list(range(5)) l random.shuffle(l) l #[0, 1, 2, 3, 4] #[1, 4, 2, 0, 3] torch.manual_seed(420) l = list(range(5)) l random.shuffle(l) l #<torch._C.Generator at 0x1d92af10cd0> #[0, 1, 2, 3, 4] #[4, 3, 1, 0, 2] torch.manual_seed(420) l = list(range(5)) l random.shuffle(l) l #<torch._C.Generator at 0x1d92af10cd0> #[0, 1, 2, 3, 4] #[2, 3, 0, 4, 1]

• random中可以通過設(shè)置random.seed來(lái)控制隨機(jī)過程

random.seed(420) l = list(range(5)) l random.shuffle(l) l #[0, 1, 2, 3, 4] #[4, 3, 1, 2, 0] random.seed(420) l = list(range(5)) l random.shuffle(l) l #[0, 1, 2, 3, 4] #[4, 3, 1, 2, 0]

總結(jié)

以上是生活随笔為你收集整理的Lesson 12.4 逻辑回归建模实验的全部?jī)?nèi)容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網(wǎng)站內(nèi)容還不錯(cuò)，歡迎將生活随笔推薦給好友。