Long Short-Term Memory
See exactly what gated cell memory adds beyond a plain RNN.
LSTM is kept on its own page so students can inspect every gate and cell-state update without mixing it into the RNN lesson.
Step by step
- Calculate forget gate.
- Calculate input gate and candidate content.
- Update cell state.
- Calculate output gate and hidden state.
- Predict from the final hidden state.
- Backpropagate through gates and cell memory.
Core formulas
Cell state
cₜ = fₜcₜ₋₁ + iₜgₜGates control what memory is retained and written.
Hidden state
hₜ = oₜ tanh(cₜ)The output gate controls exposed memory.
When to use Long Short-Term Memory
- Learning gated recurrent memory.