前言

學以致用，以學促用，通過筆記總結，鞏固學習成果，復習新學的概念。

目錄

文章目錄

• 前言
• 目錄
• 正文
• • 問題引入
  • 正則化
  • 正則化邏輯回歸
• 作業答案

正文

本節主要探討過擬合以及如何使用l2正則化抑制過擬合

問題引入

在使用面積預測房價這個問題上，如何選擇模型的階數？
過擬合的結局方案。
直觀展示解決方法對模型的影響

正則化

通過添加參數的正則化項，從而抑制了過擬合的現象。
正則化詳解。
正則化的超參數是$\lambda$，如果它太大了會造成負面影響。

#正則化線性回歸
正則化線性回歸時梯度下降算法的形態，公式做細微的調整以適應新的誤差函數。
正則化后，對應的正規方程求解公式。

正則化邏輯回歸

正則化邏輯回歸的公式。
正則化邏輯回歸梯度下降算法的表達式如上所示

作業答案

ex2_reg,m

%% Machine Learning Online Class - Exercise 2: Logistic Regression % % Instructions % ------------ % % This file contains code that helps you get started on the second part % of the exercise which covers regularization with logistic regression. % % You will need to complete the following functions in this exericse: % % sigmoid.m % costFunction.m % predict.m % costFunctionReg.m % % For this exercise, you will not need to change any code in this file, % or any other files other than those mentioned above. %%% Initialization clear ; close all; clc%% Load Data % The first two columns contains the X values and the third column % contains the label (y).data = load('ex2data2.txt'); X = data(:, [1, 2]); y = data(:, 3);plotData(X, y);% Put some labels hold on;% Labels and Legend xlabel('Microchip Test 1') ylabel('Microchip Test 2')% Specified in plot order legend('y = 1', 'y = 0') hold off;%% =========== Part 1: Regularized Logistic Regression ============ % In this part, you are given a dataset with data points that are not % linearly separable. However, you would still like to use logistic % regression to classify the data points. % % To do so, you introduce more features to use -- in particular, you add % polynomial features to our data matrix (similar to polynomial % regression). %% Add Polynomial Features% Note that mapFeature also adds a column of ones for us, so the intercept % term is handled X = mapFeature(X(:,1), X(:,2));% Initialize fitting parameters initial_theta = zeros(size(X, 2), 1);% Set regularization parameter lambda to 1 lambda = 1;% Compute and display initial cost and gradient for regularized logistic % regression [cost, grad] = costFunctionReg(initial_theta, X, y, lambda);fprintf('Cost at initial theta (zeros): %f\n', cost); fprintf('Expected cost (approx): 0.693\n'); fprintf('Gradient at initial theta (zeros) - first five values only:\n'); fprintf(' %f \n', grad(1:5)); fprintf('Expected gradients (approx) - first five values only:\n'); fprintf(' 0.0085\n 0.0188\n 0.0001\n 0.0503\n 0.0115\n');fprintf('\nProgram paused. Press enter to continue.\n'); pause;% Compute and display cost and gradient % with all-ones theta and lambda = 10 test_theta = ones(size(X,2),1); [cost, grad] = costFunctionReg(test_theta, X, y, 10);fprintf('\nCost at test theta (with lambda = 10): %f\n', cost); fprintf('Expected cost (approx): 3.16\n'); fprintf('Gradient at test theta - first five values only:\n'); fprintf(' %f \n', grad(1:5)); fprintf('Expected gradients (approx) - first five values only:\n'); fprintf(' 0.3460\n 0.1614\n 0.1948\n 0.2269\n 0.0922\n');fprintf('\nProgram paused. Press enter to continue.\n'); pause;%% ============= Part 2: Regularization and Accuracies ============= % Optional Exercise: % In this part, you will get to try different values of lambda and % see how regularization affects the decision coundart % % Try the following values of lambda (0, 1, 10, 100). % % How does the decision boundary change when you vary lambda? How does % the training set accuracy vary? %% Initialize fitting parameters initial_theta = zeros(size(X, 2), 1);% Set regularization parameter lambda to 1 (you should vary this) lambda = 1;% Set Options options = optimset('GradObj', 'on', 'MaxIter', 400);% Optimize [theta, J, exit_flag] = ...fminunc(@(t)(costFunctionReg(t, X, y, lambda)), initial_theta, options);% Plot Boundary plotDecisionBoundary(theta, X, y); hold on; title(sprintf('lambda = %g', lambda))% Labels and Legend xlabel('Microchip Test 1') ylabel('Microchip Test 2')legend('y = 1', 'y = 0', 'Decision boundary') hold off;% Compute accuracy on our training set p = predict(theta, X);fprintf('Train Accuracy: %f\n', mean(double(p == y)) * 100); fprintf('Expected accuracy (with lambda = 1): 83.1 (approx)\n');

costFunction_reg.m

function [J, grad] = costFunctionReg(theta, X, y, lambda) %COSTFUNCTIONREG Compute cost and gradient for logistic regression with regularization % J = COSTFUNCTIONREG(theta, X, y, lambda) computes the cost of using % theta as the parameter for regularized logistic regression and the % gradient of the cost w.r.t. to the parameters. % Initialize some useful values m = length(y); % number of training examples% You need to return the following variables correctly J = 0; grad = zeros(size(theta));% ====================== YOUR CODE HERE ====================== % Instructions: Compute the cost of a particular choice of theta. % You should set J to the cost. % Compute the partial derivatives and set grad to the partial % derivatives of the cost w.r.t. each parameter in thetaerror=0; for i=1:m error=error-y(i)*log(sigmoid(X(i,:)*theta))-(1-y(i))*log(1-sigmoid(X(i,:)*theta)); end l2=0; for j=2:length(theta) l2=l2+theta(j).^2; end l2=l2*lambda/(2*m); J=error/m+l2; for j=1:length(theta)factor=0;for i=1:mfactor=factor+(sigmoid(X(i,:)*theta)-y(i))*X(i,j);endgrad(j)=factor/m;if j>1grad(j)=grad(j)+lambda*theta(j)/m;end end% =============================================================end

總結

以上是生活随笔為你收集整理的吴恩达 coursera ML 第六课总结+作业答案的全部內容，希望文章能夠幫你解決所遇到的問題。

如果覺得生活随笔網站內容還不錯，歡迎將生活随笔推薦給好友。