RankFormer: A Propose-then-Select Transformer for Multi-Agent Multimodal Trajectory Prediction

Predicting vehicle trajectories plays an important role in autonomous driving, transportation safety analysis, traffic operations, etc. Although many deep learning algorithms are devised to predict future vehicle trajectories, the vehicle trajectory prediction problem is still challenging due to the complexity of decision-making process, interactions with surrounding vehicles, and the existence of multiple possible intentions for the traveling agents even under similar scenarios. For modeling interactions between vehicles, previous methods are either limited by specific graph structure (e.g., Graph Neural Network) or limited by fixed labeled intentions. In this study, we propose a pure Transformer-based network considering both temporal dependencies and spatial interactions without specific graph structures or labeled samples for intentions. By employing a cross-modal attention module, the model can learn a group of trajectories with ordered intentions. Also, we enhance the spatial encoding module to consider ego-centric velocity and acceleration of neighboring vehicles. Two tracks of decoders are employed to learn the ordered group of trajectories with ordered intentions and their probabilities. The probability decoder also provides by-product of spatial attentions among traveling vehicles. In short, the proposed model provides an efficient and effective way to predict agent trajectories under aerial scenes.