MTSSL: Meta-Thresholding Semi-Supervised Learning

A large body of Semi-supervised Learning (SSL) algorithms encounter the threshold τ to select pseudo-labels. The value of τ across different SSL algorithms can vary depending on the learning perspective, yet they may achieve similar performance. It motivates us to establish a unified theoretical framework to explain the role of τ in SSL. We statistically explained that the unsupervised loss is affected independently by correct and incorrect pseudo-labels, while τ adjusts their numbers to balance the corresponding error term. This inherent trade-off indicates that SSL can reach the same loss with varying τ, precise optimal values of τ during training may be unnecessary. With this, we treat τ as an updatable parameter and optimize it via differentiation; the new policy is named Meta-Thresholding Semi-Supervised Learning (MTSSL). Extensive experiments demonstrate the superior performance of MTSSL. We observe that the accuracy curves of SSL algorithms can overlap completely even when the values of τ differ significantly, which supports our theoretical framework and indicates that the selection of τ can be relaxed in the future design of SSL algorithms.