Optimizing Service Function Chains in Edge Computing: A Two-Stage Reinforcement Learning Framework

With the rapid proliferation of the Internet of Things (IoT), the number of smart devices connected to the internet has surged, leading to the continuous generation of massive volumes of data. This development poses significant challenges to traditional centralized cloud computing models, particularly with respect to bandwidth limitations, high latency, and security vulnerabilities. In response, edge computing has emerged as a compelling paradigm that facilitates low-latency, bandwidth-efficient, and secure data processing by relocating computational tasks closer to end-users. Unlike cloud infrastructures, edge networks are geographically distributed and located in proximity to users. This architectural advantage allows edge networks to perform a subset of computational tasks locally, thereby alleviating the burden on centralized data centers and improving service responsiveness. However, edge environments are inherently resource-constrained in terms of computational power, storage capacity, and communication bandwidth. User service demands in such environments frequently manifest as Service Function Chains sequential workflows consisting of Virtual Network Functions (VNFs) that must be executed in a specified order. The efficient deployment and scheduling of SFCs in edge networks is a complex, NP-hard problem, further complicated by dynamic request patterns and heterogeneous resource availability. Suboptimal scheduling can lead to increased latency, resource underutilization, and high service rejection rates. To address these challenges, a variety of heuristic and metaheuristic algorithms have been proposed. Recently, Reinforcement Learning (RL) has gained traction as a viable alternative due to its capacity to adapt to dynamic environments and learn optimal policies through interaction. This study introduces a novel two-stage RL-based scheduling framework for SFC deployment in edge computing. Experimental evaluations reveal that the proposed method significantly enhances system performance, improving user Quality of Experience (QoE) by up to 10×, reducing service request rejection rates by 43%, lowering average service completion times by 58%, and increasing resource utilization efficiency by 23%. These outcomes underscore the potential of deep RL approaches in enabling intelligent, scalable, and adaptive service orchestration in edge networks.