Gradient Flossing: Improving Gradient Descent through Dynamic Control of Jacobians

Training recurrent neural networks (RNNs) remains a challenge due to the instability of gradients across long time horizons, which can lead to exploding and vanishing gradients.