SGD 简单实现

梯度下降法，梯度下降法是机器学习中最常用的优化算法，就是利用负梯度方向来决定每次迭代的新的搜索方向，使得每次迭代（步长）能使待优化的目标函数逐步减小。
一个多元函数的梯度方向是该函数值增大最陡的方向，在一元函数中，梯度方向是沿着切线方向；而在多元函数中，梯度向量是函数值f对每个变量的导数，向量的方向就是梯度方向。

# TODO:梯度下降，代码实现
XOld = 0
XNew = 6
# 误差
eps = 0.00002
# 步长
alpha = 0.01
# 原函数
def f(x):
    return x ** 4 - 3 * x ** 3 + 2
# 导函数
def f_prime(x):
    return 4 * x ** 3 - 9 * x ** 2
if __name__ == "__main__":
    while abs(f(XOld) - f(XNew)) > eps:
        XOld = XNew
        XNew = XOld - alpha * f_prime(XOld)
        print XNew, f(XNew)
print "Final value is:", XNew, f(XNew)

# 0.6 1.4816
# 0.62376 1.42330940883
# 0.64906927314 1.35714506412
# 0.676047576377 1.28194284602
# 0.704821965499 1.19637272239
# 0.735526136604 1.09892354491
# 0.768299272111 0.987891154918
# 0.803284227869 0.86137271188
# 0.840624861848 0.717271371077
# 0.88046226843 0.553316926426
# 0.922929650731 0.367109726374
# 0.968145546031 0.15619685785
# 1.01620513052 -0.081809146844
# 1.06716938994 -0.349058724923
# 1.12105207953 -0.647232637901
# 1.17780464215 -0.977246003799
# 1.23729963782 -1.3388979935
# 1.29931377823 -1.73049381913
# 1.36351233705 -2.14848907758
# 1.42943744216 -2.5872319521
# 1.4965033789 -3.03889793566
# 1.56400228023 -3.49371209591
# 1.63112312708 -3.9405221914
# 1.69698555495 -4.36771572513
# 1.7606875095 -4.76437501458
# 1.82136265967 -5.12146725974
# 1.8782404676 -5.43281543495
# 1.9307000092 -5.69562610266
# 1.97830894728 -5.91046358554
# 2.02084170657 -6.08071511144
# 2.05827518314 -6.21172370623
# 2.09076484553 -6.30982324943
# 2.11860741572 -6.38148276877
# 2.14219762654 -6.43268436378
# 2.16198587623 -6.46856637243
# 2.17844164082 -6.49329499682
# 2.19202515764 -6.51009564369
# 2.20316786273 -6.52137357996
# 2.21226069427 -6.52886864004
# 2.21964868776 -6.53380848774
# 2.22563013049 -6.53704209379
# 2.23045870769 -6.53914702284
# 2.23434738269 -6.5405110299
# 2.23747309029 -6.54139166987
# 2.23998162192 -6.54195854809
# 2.24199231748 -6.54232258158
# 2.24360235159 -6.54255590417
# 2.24489051849 -6.54270521831
# 2.2459204946 -6.54280065314
# 2.24674360154 -6.54286159025
# 2.24740111486 -6.54290046896
# 2.24792617405 -6.54292525836
# 2.24834535002 -6.54294105628
# Final value is: 2.24834535002 -6.54294105628

喜欢就点赞评论+关注吧

在这里插入图片描述

感谢阅读，希望能帮助到大家，谢谢大家的支持！

SGD 简单实现

SGD 简单实现

猜你喜欢