Implementation of handwritten gradient descent for linear regression of y=kx+b
Algorithm steps:
(1) Construct data, y=3*x+5;
(2) Random initialization sum
, any value, such as
=9,
=10;
(3) Calculate ,
, and calculate
(4) Calculate the derivatives of and respectively , ,
in
Stop after repeating the loop n times
Construct a linear function:
Code:
X=[i for i in range(0,15)]
k=3
b=5
Y=[k*i+b for i in X]
import matplotlib.pyplot as plt
fig, ax = plt.subplots()
# 在同一个图形中绘制散点图和折线图
ax.scatter(X, Y, color='blue', label='scatter')
ax.plot(X, Y, color='red', label='line')
# 添加图例
ax.legend()
# 显示图形
plt.show()
MSE loss function:
loss.append((Y[i]-y_[i])**2) #公式对应代码
The results of deriving the derivatives of k and b respectively are as shown in the figure:
#公式对应代码
delta_K_sum.append((Y[i]-y_[i])*(-2)*X[i])
delta_B_sum.append((Y[i]-y_[i])*(-2))
All code:
X=[i for i in range(0,15)]
X
k=3
b=5
Y=[k*i+b for i in X]
Y
import matplotlib.pyplot as plt
# 创建散点图
plt.scatter(X, Y)
# 显示图形
plt.show()
#随机初始化要求的k和b
K=8
B=10
#k和b是正确答案,根据数据和随机初始化的K和B去拟合函数,找到最优的k和b
#y=Kx+B
loss=[]
#计算预测值
for i in range(1000):
y_=[K*i+B for i in X]
loss=[]
for i in range(len(X)):
loss.append((Y[i]-y_[i])**2)
print(sum(loss)/len(loss))
# cha=loss.sum()/len(loss)
#计算loss
#根据最小二乘法 对y_求导,等我用纸写一下,利用loss对K求梯度,去更新K的值,对B求梯度,求更新B的值
#直到K和B基本拟合图像
delta_K_sum=[]
delta_B_sum=[]
for i in range(len(X)):
delta_K_sum.append((Y[i]-y_[i])*(-2)*X[i])
delta_B_sum.append((Y[i]-y_[i])*(-2))
delta_K=sum(delta_K_sum)/len(delta_K_sum)
delta_B=sum(delta_B_sum)/len(delta_B_sum)
#0.01是学习率,保证稳定收敛
K=K-0.01*delta_K
B=B-0.01*delta_B
print(K,B)
print(K,B)
Result image:
X=[i for i in range(0,15)]
Y=[K*i+B for i in X]
import matplotlib.pyplot as plt
fig, ax = plt.subplots()
# 在同一个图形中绘制散点图和折线图
ax.scatter(X, Y, color='blue', label='scatter')
ax.plot(X, Y, color='red', label='line')
# 添加图例
ax.legend()
# 显示图形
plt.show()