Exercise 1: Linear Regression

需要用到的库

from mpl_toolkits.mplot3d import Axes3D
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt

(1) Axes3D用来画3D Figure；

(2) pandas用来读取数据；

(3) numpy——python基础库

(4) matplotlib——python画图库

1. warmUpExercise

输出1个5x5的单位矩阵。

使用numpy的eye函数生成单位矩阵。

def warmUpExercise():
    A = np.eye(5)
    print(A)


## ==================== Part 1: Basic Function ====================
print('Running warmUpExercise ...')
print('5x5 Identity Matrix:')
warmUpExercise()
print('Program paused. Press enter to continue.')
input()

输出：

2. Plotting the Data

读取数据并画图

def read_file(file):
    data = pd.read_csv(file, header=None)
    data = np.array(data)
    return data


def plotData(X, y):
    plt.figure(figsize=(5, 4), dpi=200)
    plt.plot(X, y, 'rx')
    plt.xlabel('Population of city in 10,000s')
    plt.ylabel('Profit in $10,000s')
    plt.show()


## ======================= Part 2: Plotting =======================
print('Plotting Data ...')
data=read_file('ex1data1.txt')
X = data[:, 0]
y = data[:, 1]
m = np.size(y)
X = X.reshape((m, 1))
y = y.reshape((m, 1))
plotData(X, y)
print('Program paused. Press enter to continue.')
input()

画图结果

3. Gradient Descent

梯度下降的目标是最小化损失函数：

$J\left ( \Theta \right )=\frac{1}{2m}\sum_{i=1}^{m}\left ( h_{\Theta } \left ( x^{\left ( i \right )} \right )-y^{\left ( i \right )}\right )^{2}$

$h_{\Theta }\left ( x \right )$ 函数为：

$h_{\Theta }\left ( x \right )=\Theta ^{T}x=\Theta _{0}+\Theta _{1}x$

梯度下降时 $\Theta$ 根据以下公式更新：

$\Theta _{j}:=\Theta _{j}-\alpha \frac{1}{m}\sum_{i=1}^{m}\left ( h_{\Theta } \left ( x^{\left ( i \right )} \right )-y^{\left ( i \right )}\right )x_{j}^{i}$

3.1 Computing the cost

数据中X为 $97\times 2$ 的数组，y为 $97\times1$ 的数组，theta为 $2\times 1$ 的数组。x与theta相乘得到 $97\times1$ 的数组，差的平方的均值即为损失函数结果。

def computeCost(X, y, theta):
    m = np.size(y)
    J = 0
    J = np.sum(np.square(X.dot(theta) - y)) / (2 * m)
    return J


## =================== Part 3: Cost and Gradient descent ===================
#X = np.array([[np.ones(m)], [X]], dtype='float64')  # Add a column of ones to x
X = np.c_[np.ones(m), X]
theta = np.zeros((2,1))  # initialize fitting parameters

# Some gradient descent settings
iterations = 1500
alpha = 0.01

print('Testing the cost function ...')
J = computeCost(X, y, theta)
print('With theta = [0 ; 0]\nCost computed = %f' % J)
print('Expected cost value (approx) 32.07')

theta1 = np.array([[-1], [2]], dtype='float64')
J = computeCost(X, y, theta1)
print('With theta = [-1 ; 2]\nCost computed = %f' % J)
print('Expected cost value (approx) 54.24')
print('Program paused. Press enter to continue.')
input()

输出结果：

3.2 Gradient descent

梯度下降时每次更新的大小为导数乘以学习率。

def gradientDescent(X, y, theta, alpha, iterations):
    m = np.size(y)
    theta = 0
    J_history = np.zeros((iterations, 1))
    for i in range(iterations):
        r = np.sum((X.dot(theta) - y) * X, 0).reshape((2, 1)) / m * alpha
        theta = theta - r
        J_history[i] = computeCost(X, y, theta)
    return theta, J_history


print('Running Gradient Descent ...')
# run gradient descent
theta, _ = gradientDescent(X, y, theta, alpha, iterations)

# print theta to screen
print('Theta found by gradient descent:')
print('%f %f' % (theta[0], theta[1]))
print('Expected theta values (approx)')
print(' -3.6303 1.1664')

predict1 = np.array([1, 3.5], dtype='float64').dot(theta)
print('For population = 35,000, we predict a profit of %f' % (predict1*10000))
predict2 = np.array([1, 7], dtype='float64').dot(theta)
print('For population = 70,000, we predict a profit of %f' % (predict2*10000))

print('Program paused. Press enter to continue.')
input()

输出结果：

线性回归结果：

4. Visualizing J

## ============= Part 4: Visualizing J(theta_0, theta_1) =============
print('Visualizing J(theta_0, theta_1) ...')

theta0_vals = np.linspace(-10, 10, 100)
theta1_vals = np.linspace(-1, 4, 100)

J_vals = np.zeros((np.size(theta0_vals), np.size(theta1_vals)))

for i in range(np.size(theta0_vals)):
    for j in range(np.size(theta1_vals)):
        t = np.array([[theta0_vals[i]], [theta1_vals[j]]], dtype='float64')
        J_vals[i][j] = computeCost(X, y, t)

J_vals =J_vals.T

fig = plt.figure(figsize=(5, 4), dpi=200)
ax = Axes3D(fig)
ax.plot_surface(theta0_vals, theta1_vals, J_vals, rstride=1, cstride=1, cmap=plt.cm.coolwarm)
ax.set_xlim(-10, 10)
ax.set_ylim(-1, 4)
ax.set_zlim3d(0, 1000)
ax.set_xlabel('theta_0')
ax.set_ylabel('theta_1')
ax.set_zlabel('J')
plt.show()


plt.figure(figsize=(5, 4), dpi=200)
plt.contour(theta0_vals, theta1_vals, J_vals, np.logspace(-2, 3, 20))
plt.plot(theta[0], theta[1], 'rx', linewidth=2, markersize=10)
plt.xlim(-10, 10)
plt.ylim(-1, 4)
plt.xlabel('theta_0')
plt.ylabel('theta_1')
plt.show()

可视化损失函数

画损失函数等高线及theta位置，可以看出theta在等高线图的最低位置。

芦花似雪

发布了20 篇原创文章 · 获赞 6 · 访问量 2185

私信关注

机器学习——Andrew Ng machine-learning-ex1 python实现