Exploiting the Alternatives: Coordinated Learning via Hierarchical RL for Dynamic VNEAP

Virtual Network Embedding (VNE) is a key enabler of network slicing, yet most formulations assume that each Virtual Network Request (VNR) has a fixed topology. Recently, VNE with Alternatives (VNEAP) was introduced to capture malleable VNRs, where each request can be instantiated using one of several functionally equivalent topologies that trade resources differently. This flexibility can improve embedding feasibility, but only if the orchestrator can jointly select suitable alternatives and embed them under dynamic arrivals. This paper proposes HRL-VNEAP, a hierarchical reinforcement learning approach for dynamic VNEAP. A high-level policy selects the most suitable alternative topology (or rejects the request), and a low-level policy embeds the chosen topology onto the substrate network. Experiments on realistic substrate topologies under varying arrival rates show that naive exploitation strategies provide only modest gains, whereas HRL-VNEAP outperforms state of the art approaches, improving acceptance ratio by up to 22%, and net profit by up to 20%. An offline MILP upper bound is also used on tractable instances to quantify the remaining optimality gap.