And FIG autograd calculation is very powerful tool that can define complex operation and automatically derivative; however, for large-scale networks, the underlying autograd too.
When building the neural network, we often consider calculating arranged layers , some of which parameters can be learned , they will be optimized in the learning process.
TensorFlow, there is a similar Keras , TensorFlow-Slim and TFLearn this highly abstract interface to encapsulate the underlying calculation map, which is very convenient that build the network.
In PyTorch, the package nnis completed the same function. nnThe package defines a set of substantially equivalent to the layer modules . A module accepts input tesnor, tensor calculation output, but also preserves some parameters such as the internal state of the tensor to learn the like. nnPacket loss also defines a set of functions (loss functions), used to train the neural network. Nn package while only some of the outer layer activation function and operation, further comprising a common loss function.
code show as below:
Torch Import
Device torch.device = ( 'CUDA' IF torch.cuda.is_available () the else 'CPU')
N , D_in , H , D_OUT = 64 , 1000 , 100 , 10
# randomly generated inputs and outputs
x = torch.randn (N , D_in , Device = Device)
Y = torch.randn (N , D_OUT , Device = Device)
# use nn package our model is defined as a series of layers.
# Nn.Sequential comprising modules other modules, these modules and sequentially applied to produce its output.
# Each module uses a linear function to calculate the linear input and output from and to save weight and weight variation inside tensor.
# After constructing the model, we used .to () method to move it to the desired device.
= torch.nn.Sequential Model (
torch.nn.Linear (D_in , H) ,
torch.nn.ReLU () ,
torch.nn.Linear (H , D_OUT) ,
) .to (Device)
'' '
NN package there define commonly loss function
MSELoss () parameter reducetion initially 'mean', as mean, we use the 'sum' to and
in practice, by setting reduction = 'elementwise_mean' is used as a mean square error loss is more common
'' '
loss_fn = torch.nn.MSELoss ( Reduction = ' elementwise_mean ')
learning_rate = 1E-. 4
for T in Range ( 500):
' ''This operation is the forward propagation, by passing to model X , and thus to obtain an output
y while the module has __call__ properties can call calling functions as they we will enter the tensor X , to obtain the output tensor y_pred '' ' y_pred = Model (X) Loss = loss_fn (y_pred , y) Print (T , Loss. Item ()) # until clear gradient calculation model.zero_grad () '' ' back-propagation: calculated loss values of the model parameters of the model can be trained gradient whether each parameter may be dependent on the training require_grad Boolean value so this operation can All training parameters calculated gradient '' ' loss.backward () # gradient descent update # use for loop extraction model of parameters () # Out of param.data operate with torch.no_grad (): for param
in model.parameters():
param.data -= learning_rate * param.grad