Recurrent Neural Network
Follow a recurrent hidden state through time and backpropagate the real sequence loss.
A focused simple RNN page teaches temporal state and parameter sharing before gated recurrent cells.
Step by step
- Read one sequence element.
- Combine it with the previous hidden state.
- Apply tanh.
- Predict from the final hidden state.
- Backpropagate through time.
- Update shared recurrent parameters.
Core formulas
RNN state
hₜ = tanh(Wₓxₜ + Wₕhₜ₋₁ + b)The hidden state carries information from earlier sequence positions.
When to use Recurrent Neural Network
- Learning temporal parameter sharing and BPTT.