Deep Q-Learning for Edge Caching in Cell Free Massive MIMO Networks: A Multi-Layered Approach

Abstract

Cell-Free Massive MIMO (CF-mMIMO) networks have emerged as a groundbreaking advancement in wireless communication. This study proposes a novel hierarchical multi-layered approach for the optimization of edge caching in CF-mMIMO networks using the Deep Q-Learning (DQL) Framework. The objective of the proposed framework is to balance service quality and minimize response times, backhaul traffic, and power consumption. A substantial dataset encompassing user queries, response times, content sizes, and caching choices is utilized to evaluate the proposed approach. The proposed framework achieves an impressive 95% Cache Hit Ratio (CHR), indicating efficient cache usage and reduced reliance on remote services. By leveraging the DQL models, the efficiency of edge caching in CF-mMIMO networks is substantially enhanced. This enhancement leads to consistently high cache hit ratios, expedited content delivery, and improved network efficiency. Furthermore, the model’s energy efficiency contributes to the network’s long-term sustainability. Rigorous testing across diverse scenarios confirms the model’s versatility across rural, suburban, urban, and stadium settings. By comparing different caching eviction policies, we establish LFU as optimal for cache efficiency, considering the cache hit ratio and processing time. Overall, this research underscores the substantial advantages of the proposed deep Q-learning model for edge caching in cell-free Massive MIMO networks, significantly impacting cache hit ratio, processing time, and energy efficiency across a range of practical scenarios. Keywords— Deep Reinforcement Learning (DRL), Network optimization, Content caching, Intelligent decision-making Wireless networks.