1 Introduction
torch.nn.SmoothL1Loss
is a loss function in PyTorch, usually used for regression problems. It is a combination of L1 loss and L2 loss, aiming to reduce sensitivity to outliers.
loss_function = nn.SmoothL1Loss(reduction='mean', beta=1.0)
2. Parameters
size_average (deprecated) : Previously used to determine whether the loss should be averaged per element. If set to
False
, losses are summed. This option is now deprecated andreduction
parameters should be used instead.reduce (deprecated) : This is also an old parameter that specifies whether to apply reduction. Now also
reduction
replaced by parameters.reduction : Specifies the reduction method to be applied to the output. Optional values are:
'none'
: No reduction is applied.'mean'
: Calculates the average of losses.'sum'
: Calculates the sum of losses.beta : Used to determine the smooth transition point. For errors less
beta
than , the loss function becomes L2 loss, and forbeta
errors greater than , it becomes L1 loss.
3. Image
In the image of the Smooth L1 loss function, when the difference between the predicted value and the true value is small (less than beta
, which defaults to 1.0 here), it is calculated similarly to the L2 loss (squared error). When the difference is large, it is calculated similar to L1 loss (absolute error). This hybrid property makes the Smooth L1 loss less sensitive to outliers and more stable in optimization.
4. Examples
Suppose we have the following situation: we are training a model to predict some continuous value, such as house prices. We have the following target values (true values) and predicted values:
- Target (real value):
[1.5, 2.0, 3.0]
- predict:
[1.4, 2.1, 2.9]
As the loss function we use SmoothL1Loss
:
import torch
import torch.nn as nn
# 定义目标和预测值
targets = torch.tensor([1.5, 2.0, 3.0])
predictions = torch.tensor([1.4, 2.1, 2.9])
# 创建 SmoothL1Loss 实例
loss_function = nn.SmoothL1Loss(reduction='mean', beta=1.0)
# 计算损失
loss = loss_function(predictions, targets)
print(loss)
In this example, the loss function will calculate the Smooth L1 loss between the target and prediction and return its average. If the difference between prediction and target is less than beta
(1.0 in this case), then it applies the squared form of the L2 loss; if the difference is greater beta
, it applies the absolute form of the L1 loss. This blending makes the Smooth L1 loss less sensitive to outliers, especially when the predicted values differ greatly from the true values.
5, reference
【pytorch】nn.SmoothL1Loss function usage_nn.smoothl1loss()-CSDN Blog
PyTorch study notes: nn.SmoothL1Loss——Smooth L1 loss_nn.smoothl1loss()-CSDN Blog