MoE-Prism: Disentangling Monolithic Experts for Elastic MoE Services via Model-System Co-Designs

Mixture-of-Experts (MoE) scales model capacity through sparse activation, and is becoming an important architecture for large language models (LLMs). However, existing MoE serving systems typically execute all requests under a fixed routing configuration, limiting their ability to exploit heterogeneous computation requirements across requests. Routing top-k, which determines the number of routed experts activated per token, directly controls routed-expert computation and provides a natural mechanism for request-level compute elasticity. Realizing this capability, however, requires finer-grained routing units and efficient runtime execution for heterogeneous routing budgets. We present MoE-Prism, a model and system support framework for request-level compute elasticity in MoE serving. MoE-Prismdecomposes monolithic experts into fine-grained sub-experts to expose denser routing operating points and provides a k-aware serving runtime that effectively serves heterogeneous routing budgets under both throughput-oriented and latency-sensitive workloads. We implement MoE-Prism on top of vLLM and evaluate it on three representative MoE models. MoE-Prism expands the number of available routing operating points by 4\times, improves offline inference throughput by up to 33.9%, and reduces online serving TTFT under heterogeneous workloads. These results demonstrate practical elastic MoE serving with request-level routing targets.