Demo: Real-time Generative Multicasting with On-Device Intent-aware Semantic Decomposition

We present a demonstration for generative multicasting with on-device, intent-aware semantic decomposition. At the transmitter, DNN-based segmentation extracts a semantic map from the source video, decomposing it into multiple sub-signal classes based on multi-user receiver intents. The transmitter broadcasts the semantic map to all users over shared wireless/network resources, thereby utilizing orthogonal resources only to transmit the sub-signal classes intended for each user. Users partially reconstruct and partially synthesize the signal by combining the received intended classes with non-intended classes locally synthesized by a generative model from the semantic map. We derive the rate-distortion/perception curves for reconstruction/synthesis with the generative model, to adaptively set compression rates for the semantic map and intended classes. Generative multicasting significantly reduces the wireless/network resources required for existing/emerging multimedia multicasting applications. The system is real-time on a Google Coral Edge TPU with 4 TOPS (int8). This is the first demonstration of generative multicasting representing a substantial advancement in on-device generative SemCom.