Simple Recurrent Network
(
Simple Recurrent Network
,
SRN
)
is a very simple recurrent neural network, a
neural network with only one hidden layer
.
In
a two-layer
feed-forward neural network
, connections exist between adjacent layers , and there is no connection between nodes in the hidden layer . The simple recurrent network increases the feedback connection from hidden layer to hidden layer.
If we regard the state at each moment as a layer of the feedforward neural network, the
recurrent neural network can be regarded as a neural network with weight sharing in the time dimension.
Figure
6.2
shows the recurrent neural network unrolled in time.
Computational Power of Recurrent Neural Networks
General Approximation Theorem for Recurrent Neural Networks
The fitting ability of cyclic neural network is also very powerful.
A fully connected recurrent network is an approximater for any nonlinear dynamical system.
