Temporally smoothed incremental model-based heuristic dynamic programming for command-filtered cascaded online learning flight control

Approximate Dynamic Programming (ADP) enables online adaptation of control laws through adaptive critic methods. However, its application to online learning Angle-of-Attack (AoA) tracking control remains challenging due to oscillatory control actions under nonlinear dynamics, which may induce system oscillations, degrade tracking performance, and increase actuator effort. To address this challenge, this paper proposes two innovations for online learning flight control: (1) incorporating temporal-scale policy smoothness regularization into the Incremental Model-based Heuristic Dynamic Programming (IHDP) framework; and (2) employing a low-pass filter to attenuate high-frequency pitch-rate commands. Furthermore, a primal-dual approach is developed to adaptively adjust the smoothness penalty weight in the policy objective according to a prescribed smoothness criterion. Tracking control simulations demonstrate that the proposed methods reduce control system oscillations, improve tracking performance, and exhibit post-training policy robustness to model uncertainties.