Browse > Article
http://dx.doi.org/10.3837/tiis.2016.07.017

A Semi-Markov Decision Process (SMDP) for Active State Control of A Heterogeneous Network  

Yang, Janghoon (Department of New Media Contents, Seoul Media Institute of Technology)
Publication Information
KSII Transactions on Internet and Information Systems (TIIS) / v.10, no.7, 2016 , pp. 3171-3191 More about this Journal
Abstract
Due to growing demand on wireless data traffic, a large number of different types of base stations (BSs) have been installed. However, space-time dependent wireless data traffic densities can result in a significant number of idle BSs, which implies the waste of power resources. To deal with this problem, we propose an active state control algorithm based on semi-Markov decision process (SMDP) for a heterogeneous network. A MDP in discrete time domain is formulated from continuous domain with some approximation. Suboptimal on-line learning algorithm with a random policy is proposed to solve the problem. We explicitly include coverage constraint so that active cells can provide the same signal to noise ratio (SNR) coverage with a targeted outage rate. Simulation results verify that the proposed algorithm properly controls the active state depending on traffic densities without increasing the number of handovers excessively while providing average user perceived rate (UPR) in a more power efficient way than a conventional algorithm.
Keywords
MDP; active state control; load balancing; heterogeneous networks; handover;
Citations & Related Records
연도 인용수 순위
  • Reference
1 E. Oh, X. Liu, Z. Niu, "Toward dynamic energy-efficient operation of cellular network infrastructure," IEEE Communications Magazine, Vol.49 , No.6 , pp. 56 - 61, June, 2011. Article (CrossRef Link)   DOI
2 L.M. Correia, D. Zeller, O. Blume, D. Ferling, Y. Jading, I. Gódor, G. Auer, L. Van der Perre, "Challenges and Enabling Technologies for energy aware mobile radio networks," IEEE Communications Magazine, Vol.48, No.11, pp. 66-72, Nov. 2011. Article (CrossRef Link)   DOI
3 G. Auer, V. Giannini, C. Desset, I. Godor, P. Skillermark, M. Olsson, M. Imran, D. Sabella, M. Gonzalez, O. Blume and A. Fehske, "How much energy is needed to run a wireless network?," IEEE Wireless Commun., Vol. 18, No. 5, pp. 40-49, Oct. 2011. Article (CrossRef Link)   DOI
4 K. Son, H. Kim, Y. Yi and B. Krishnamachari, "Base Station Operation and User Association Mechanisms for Energy-Delay Tradeoffs in Green Cellular Networks," IEEE J. Select. Areas Commun., Vol. 29, No. 8, pp. 1525-1536, Aug. 2011. Article (CrossRef Link)   DOI
5 Z. Yang and Z. Niu, “Energy Saving in Cellular Networks by Dynamic RS-BS Association and BS Switching,” IEEE Trans. Veh. Technol., Vol.62, No.9, pp. 4602–4614, Nov. 2013. Article (CrossRef Link)   DOI
6 M. A. Marsan, L. Chiaraviglio, D. Ciullo, and M. Meo, "Optimal Energy Savings in Cellular Access Networks," in Proc. of IEEE International Conference on Communications Workshops, Dresden, June 2009. Article (CrossRef Link)
7 R. Gupta and E. C. Strinati, "Base-Station Duty-Cycling and Traffic Buffering as a Means to Achieve Green Communications," in Proc. of IEEE Vehicular Technology Conference (VTC Fall), Quebec City, Sept. 2012. Article (CrossRef Link)
8 L. Chiaraviglio, D. Ciullo, G. Koutitas, M. Meo, and L. Tassiulas, "Energy-efficient planning and management of cellular networks," in Proc. of 9th Annual Conference on Wireless On-Demand Network Systems and Services (WONS), 2012. Article (CrossRef Link)
9 W.-C. Liao, M. Hong, and Z.-Q. Luo, "Base station activation and linear transceiver design for utility maximization in Heterogeneous networks," in Proc. of 2013 IEEE International Conference on Acoustics, Speech and Signal Processing. Vancouver, BC, May 2013. Article (CrossRef Link)
10 M. F. Hossain, K. S. Munasinghe, and A. Jamalipour, “Distributed Inter-BS Cooperation Aided Energy Efficient Load Balancing for Cellular Networks,” IEEE Trans. Wireless Commun., Vol.12, No.11, pp. 5929–5939., Nov. 2013. Article (CrossRef Link)   DOI
11 A. Bousia, E. Kartsakli, L. Alonso, and C. Verikoukis, "Dynamic energy efficient distance-aware Base Station switch on/off scheme for LTE-advanced," in Proc. of IEEE Global Communications Conference (GLOBECOM), Anaheim, CA, Dec. 2012. Article (CrossRef Link)
12 G. H. Carvalho, I. Woungang, A. Anpalagan, R. W. Coutinho, and J. C. Costa, “A semi-Markov decision process-based joint call admission control for inter-RAT cell re-selection in next generation wireless networks,” Computer Networks, Vol. 57, No.17, pp. 3545–3562., Dec. 2013. Article (CrossRef Link)   DOI
13 M. Coupechoux, J.-M. Kelif, and P. Godlewski, "SMDP approach for JRRM analysis in hetrogeneous networks," in Proc. of 14th European Wireless Conference, Prague, June 2008. Article (CrossRef Link)
14 E. Stevens-Navarro, Y. Lin, and V. Wong, “An MDP-Based Vertical Handoff Decision Algorithm for Heterogeneous Wireless Networks,” IEEE Trans. Veh. Technol., pp. 1243–1254, Mar. 2008. Article (CrossRef Link)   DOI
15 D. Cao, S. Zhou, and Z. Niu, “Optimal Combination of Base Station Densities for Energy-Efficient Two-Tie Heterogeneous Cellular Networks,” IEEE Transactions on Wireless Communications IEEE Trans. Wireless Commun., Vol.57, No.2, pp. 4350–4362, Mar. 2008. Article (CrossRef Link)
16 K. Son, S. Nagaraj, M. Sarkar, and S. Dey, "QoS-aware dynamic cell reconfiguration for energy conservation in cellular networks," in Proc. of 2013 IEEE Wireless Communications and Networking Conference (WCNC), Shanghai, Apr. 2013. Article (CrossRef Link)
17 L. Saker, S. Elayoubi, R. Combes, and T. Chahed, “Optimal Control of Wake Up Mechanisms of Femtocells in Heterogeneous Networks,” IEEE J. Select. Areas Commun. IEEE Journal on Selected Areas in Communications, Vol.30, No.3, pp. 664–672. Mar. 2012. Article (CrossRef Link)   DOI
18 S. Kim, S. Choi, and B. G. Lee, “A Joint Algorithm for Base Station Operation and User Association in Heterogeneous Networks,” IEEE Commun. Lett., Vol. 17, No.8, pp. 1552–1555, Aug 2013. Article (CrossRef Link)   DOI
19 Y. Cui, V. K. N. Lau, and Y. Wu, “Delay-Aware BS Discontinuous Transmission Control and User Scheduling for Energy Harvesting Downlink Coordinated MIMO Systems,” IEEE Trans. Signal Process., Vol.60, No.7, pp. 3786–3795., July 2011. Article (CrossRef Link)   DOI
20 D. P. Bertsekas, Dynamic programming and optimal control, 3rd ed. Belmont, Mass.: Athena Scientific, 2005.
21 S. Singh, T. Jaakkola, M. L. Littman,and C. Szepesvari, "Convergence Results for Single-Step On-Policy Reinforcement-Learning Algorithms," Machine Learning, Vol. 39, pp. 287-308, 2000. Article (CrossRef Link)   DOI
22 R. S. Sutton, and A. G. Barto, Reinforcement Learning, Cambridge, Mass. The MIT Press, 1998.
23 C. Liu, B. Natarajan, and H. Xia, "Small Cell Base Station Sleep Strategies for Energy Efficiecy," IEEE Trans. Vehi. Tech. Vol. 65, No. 3, pp. 1652-1661, Mar. 2016. Article (CrossRef Link)   DOI
24 E. Chavarria-Reyes, I. F. Akyildiz, and E. Fadel, "Energy Consumption Analysis and Minimization in Multi-Layer Heterogeneous Wireless Systems," IEEE Trans. Mobile Comp. Vol. 14, No. 12, pp.2474-2487, Dec. 2015. Article (CrossRef Link)   DOI
25 Y. S. Soh, T. Q. S. Quek, M. Kountouris, and H. Shin, "Energy Efficient Heterogeneous Cellular Networks," IEEE Journal on Select. in Commun. Vol. 31, No. 5, pp. 840-850, May 2013. Article (CrossRef Link)   DOI
26 A. Bousia, E. Kartsakli, A. Antonopoulos, L. Alonso, and C. Verikoukis, "Multiobjective Auction-based Switching Off Scheme in Heterogeneous Networks," IEEE Trans. on Vehi. Tech. Article (CrossRef Link)
27 J. Pang, J. Wang, D. Wang, G. Shen, Q. Jiang, and J. Liu, "Optimized time-domain resource partitioning for enhanced inter-cell interference coordination in heterogeneous networks," in Proc. of WCNC, Shanghai, China, 2012. Article (CrossRef Link)
28 M. Wiering, and J. Schmidhuber, "Fast Online Q(λ)," Machine Learning, 33, pp.105-115, 1998. Article (CrossRef Link)   DOI
29 G. P. Koudouridis, G.Hui, and P. Legg, "A Centralized Approach to Power On-Off Optimization for Heterogeneous Networks," in Proc. of IEEE VTC Fall, pp.1-5, Quebec City, Sept. 2012. Article (CrossRef Link)