Towards Human Motion World Models via Executable Behaviour Representations

Human motion world models should capture motion's intentionality by being executable: adaptable to different actions and capable of assessing motion quality. To achieve this, we introduce a domain-specific language ExAct that represents human motions as underspecified programs that can be compiled to a reward model for zero-shot policy inference. By leveraging the compositional nature of ExAct programs, we combine individual policies into executable behaviour representations. We evaluate the utility of the proposed approach by analysing human motion capture for the tasks of human action segmentation and human action anomaly detection. Our results suggest that the improvement in data efficiency and the capture of intuitive relationships between human actions are better than those of task-specific models.