1. Introduction
With the rapid increase in multimedia/internet traffic and the requirement of many other broadband services in wireless, the resource management in Long Term Evolution (LTE) and LTE-Advanced (LTE-A) need to be carried out intelligently not only to share spectrum but also network resources optimally. The ITU-R report [1] identified the vital features related to the use of cognitive radio system (CRS), and these systems employ a technique which agrees to obtain the knowledge of its environment and dynamically adjust its operational parameters and learn from the results obtained.
A suitable machine learning technique in CRS can be adopted to learn and analyze various traffic patterns on different channels over time and then predict the preeminent idle channels [2-3]. If a carrier provider is in need of extra resources in a given sector, an intelligent sharing mechanism based on predefined policy will not only help in overall better resource utilization but also help in achieving better quality of service (QoS) requirement of different services. Game theory in CRS could be an appropriate method to ensure coexistence of different carrier providers while optimally sharing the resources either in a collaborative or non-collaborative mode.
The dynamic spectrum access (DSA) by the SCPs is modeled [4] in three groups – shared use, commons, and the exclusive use model. In the shared use, the SCPs can make use of the spectrum owned by PCPs without any price when not used by them. In the commons model, the spectrum is open for everyone to access (e.g. ISM bands). These two methods have some specific drawbacks. In the exclusive use model, the PCPs lease their vacant spectrum to SCPs and gain some revenue while the SCPs could have the assured access to the spectrum for a shorter or longer period of time as per the agreement made. The ITU report [5] explains the different prices and various techniques involved in sharing and evaluating the spectrum.
In a multi operator radio access network (MO-RAN) [6] apart from spectrum sharing, RAN can also be shared which leads to unified trading policy. The benefits of including RAN along with the spectrum are that a carrier provider-A can handoff some surplus users to another carrier provider-B subjected to the agreement in trading policy. The billing and charging could be still with the home carrier provider based on its usual tariff rate. Now the sharing approach can be viewed in three dimensions (Fig. 1). In one dimension decision making methods are various trading mechanism is represented while in the other two dimensions the scenario of single or multiple carrier providers and trading mechanism is considered.
Fig. 1.Inter-Carrier provider spectrum sharing
The spectrum sharing mechanism need to provide financial incentives to the parties involved and market driven spectrum trading could be a better approach[7,8] In MO-RAN the challenges increases by many fold as the trading mechanism need to consider not only DSA but also the cost involved in implementation. Additionally, fairness remains as an issue in any multiple access systems. Unlike previous work [9], the goal of our work presented here is to devise an adaptive sharing mechanism of resource block (RB)for a future LTE based network (e.g., LTE-A) while considering major issue in trading, fairness, and implementation. For this reason we include the cost of RAN sharing apart from other goal in problem formulation. Furthermore, the focus of our proposed algorithms is to incorporate utility based incentive to each party while increasing the over all resources utilization and throughput.
Our earlier work [10] was focused on an intelligent mechanism of resource utilization approach using reinforcement learning with game theory [11] which identifies the best free resource blocks (RB) and allocated using different modes among secondary users. This work is logical extension of previous work [10] but sharing is based on trading policy. The proposed system is formulated on agreement model based on the utility maximization in leasing the resources. The policy includes role changes i.e., any carrier provider can act as a buyer (seller), when the resources at a particular time period are deficient (surplus).
In practice sometimes LTE-SCP runs out of radio resource to serve additional user equipments (UEs) but this can be availed dynamically with LTE-PCP by suitable trading mechanism (Fig. 2). There could be unified spectrum trading policy in CRS which includes the cost of RAN sharing by LTE-PCP. In order to facilitate spectrum trading, in this paper we have proposed and evaluated three different approaches in short term leasing by considering real world cellular mobile market scenario.
Fig. 2.Spectrum sharing scenario between LTE- PCP and LTE-SCP
In the first approach, we consider a utility function based resource sharing (UFRS) where the PCP shares its resources with the SCP upon considering various factors in order to maximize its utility. In the second approach a non-cooperative resource sharing (NCRS) is formulated using Nash Equilibrium where each SCPs shares the resources with the PCPs based on reinforcement learning. The third policy defines a recommendation entity based resource sharing (RERS) which is the variation of second policy based on Nash bargaining method where the PCP and SCP leases and acquires the resources depending on the price declared by a recommender. We formulate a mathematical model for the above three policies and simulate in LTE environment considering several system parameters.
The rest of the paper is organized as follows: Section 2 describes a review of related works on spectrum trading in CR networks. The system model and optimization frame work are explained in Section 3. The problem formulation for our three approaches along with the heuristic algorithm is presented in Section 4. Section 5 presents the simulation results under various simulation parameters and conditions. The conclusion and future work are stated in Section 6.
2. Related Work
The challenges in spectrum trading has been analyzed in different ways to solve the pricing issue in CR networks which includes bargaining game, auction, noncooperative, classical optimization and micro economic approach [12]. In bargaining game, a Nash solution is obtained as the players can negotiate and bargain with each other ensuring fairness and efficiency. In auction approach, the bidding decision is carried out at a certain interval or at a fixed time and price of the spectrum varies largely with the bidders. Optimization approach maximizes the revenue of a seller and maximizes the throughput of a bidder under some constraints. Non cooperative game is involved when the multiple entities share the spectrum while applying game theory to find to the solution. Market equilibrium is another method of spectrum trading where the competition among players determine the market dynamics.
In [9], authors discussed two models of spectrum sharing where the utility based profit maximization problem is analyzed for one primary user and used Nash equilibrium for multiple primary user scenarios. The short term and long term spectrum trading with two different approaches have been analyzed in [8] where the agreement based spectrumsharing is discussed for future market. The secondary users can compete with each other in a spot market based on symmetry and asymmetry method. The spectrum leasing scenario with multiple primary users and secondary users are discussed in [13] is based on two classes of solutions including generalized Nash equilibrium and solutions where the secondary users compete for a spectrum only with its satisfied QoS.
The multiple primary and secondary users employing market equilibrium price compete for a spectrum based on a Bertrand game model [14] and Nash equilibrium. An oligopoly optimal auction mechanism [15] based on graph theory was proposed for dynamic spectrum sharing. A random leader based short term and long term incentive aware spectrum sharing is discussed in [16] and analyzed using special mobility management model. In [17] the authors have proposed a demand based optimization problem and formulated using Nash equilibrium for spectrum sharing among single and multiple agents with the help of a spectrum broker. In a typical cellular based cognitive radio network, the performance and in particular throughput for the UEs present near to eNodeB can be maintained easily but at cell edge it become difficult as inter-cell interference becomes dominant. In [18], a dynamic spectrum allocation scheme based on game theory through distributed pricing calculation and exchange has been proposedto take care of UEs throughput at cell edge.
In [19], the spectrum leasing has been proposed in two different cases using interference temperature constraint. The resources allocated among secondary users using second price auction mechanism has been discussed in [20]. The non cooperative way of spectrum sharing scenario can be formulated with multiple users using sub gradient and Q- learning algorithm. An evolutionary game with multiple buyers and sellers and Nash equilibrium can be used to analyze the competition [7][21]. The two level dynamic spectrum sharing game provide further flexibility where the secondary users adopt strategies based on quality and price [22]. A Nash Bargaining game using two different cases and market equilibrium price has been formulated in [23] for spectrum trading where the sharing among primary user and secondary user is analyzed and random matching based spectrum sharing among secondary users is discussed.
3. System Model &Optimization Framework
3.1System Model
We consider an LTE based cognitive radio systems (LTE-CRS) model with Np PCPs and Ns SCPs. The total bandwidth W is distributed into Mresource blocks (RBs). Let the available free RBs for trading by the PCP at time t be and randomly distributed in time-resource grid (Table 1). Based on the role played by the LTE carrier provider with CRS capability, we name them as primary (LTE-PCP) or secondary (LTE-SCP). The LTE-SCP request the resources from LTE-PCP for a contract period τ and these resources are reused by a PCP after a contract period. The contract starts upon satisfying minimum utility of both primary and secondary carrier providers. The PCP also offers its RAN to facilitate the traded spectrum by the SCP and the cost of maintaining RAN for PCP is treated as overhead in the utility function. The optimization framework in framing the utility expression is modeled below.
Table 1.List of Symbols
3.2 Optimization Framework for LTE-PCP & LTE-SCP
The valuation price ( VRBi ) for the resources owned by the PCPi (∀ i ∈ NP) at a particular time interval t is dictated by the quality of the channel, which in turn mainly depends on the signal to interference noise ratio (SINR) γRBi and hence the achievable data rate. The SINR outlines a boundary how far the SCPj can use the requested resources of PCPi that can vary mainly due to noise level, fading and hardware capability of SCPj. Let the total resource of a PCP be MRBi and maximum amount of RBs leased to the SCPjbe . The utility model for PCP and SCP needs to be formulated separately.
3.2.1 Utility function of PCP
The utility of a PCPi is the function of it’s own valuation price (VRBi) of retained RBs, offered price (Pj) by the SCPj and loss it suffers in leasing its resources to the SCPj. The objective function (Upcp) for utility of PCP is formulated as
Where is loss factor for the m RBs allocated to the SCPj for a contract period τ and is linked with availability at PCPi ; is the loss function due to RAN sharing.
The objective function defined in (1), includes three purposes that need to be optimized. The first term indicates the valuation of the total RBs left with the PCPi in a given time. This represent the non-trading component of utility function of a PCPi . The second term is the result of direct return from the short-term trading with SCPj which depends on the prevailing market condition. The third term denotes total loss for a PCPi in the process of trading. This includes overhead in maintaining the RAN to support the trading process with SCPj .
3.2.2 Utility function of SCP
The utility of a SCPj is a function of payoff (, VRBi, τ) attained in using the resources and price pjpaid to the PCPi. The overhead represents the cost of signaling (So) in handing over surplus UEs to PCP under spectrum trading policy. Now the objective function (Uscp) for utility of SCP is defined as
Subjected to,
Where are the requested and available contract period agreed by SCPj and PCPi. The quality of resources owned by PCP is measured in terms of SINRRBi which should be greater than a predefined level. Each carrier provider defines its own threshold limit for its utility before entering into transaction.
The objective function for a SCPj defined in (5) has three major component. The first term represents gain in terms of resource and hence data rate dictated by the channel quality on prevailing conditions. The second terms is loss due to payment at the end of trading process. The SCPj need additional signaling mechanism to utilize the traded spectrum. Although the surplus resource demand at SCPj is met through RAN of PCPi , a suitable signaling method need to be maintained at SCPj for seamless utilization of traded resources.
4. Development of Heuristic Algorithm
Based on the problem formulation discussed in Section3 for agreement in spectrum trading, we develop three heuristic algorithms based on the transaction model for trading policy. All the three approaches discussed here assume that in a given time SCP is in need of additional resource which can be served in the same cell covered by PCP. It is also assumed that both SCP and PCP are willing to participate and role change can happen i.e., the SCP may become PCP and vice versa in a given circumstance.
4.1 Utility Function Based Resource Sharing (UFRS)
In this approach, the PCP advertises its price, maximum available RBs and the contract period. Based on requirements, each SCP calculate its utility and if it satisfies minimum value it sends its requisition for the required resources and valuation price to the PCP. The PCPi upon receiving requisition estimates its utility based on the valuation price, requirement and its system parameters. Now PCP evaluates offers from different SCP and then starts short listing SCPs for which it’s utility (UPCP) is maximum.
Where is the utility of PCP corresponding to Nth SCP.If the requisition of resources by the SCPs is less than the maximum available unused resource then allocate all the resources else allocate the resources based on priority which is expressed as
When the requisition of resources of all the SCPs are equal then allocation to the SCPs is carried out based on the Jain’s fairness [24] given by
Now the complete UFRS algorithm is listed in Table 2.
Table 2.UFRS Algorithm
4.2 Non Cooperative Resource Sharing (NCRS)
The NCRS algorithm does not require any cooperation among SCPs. After receiving advertisement from PCP each SCP sets an optimal strategy in obtaining the resources in using reinforcement learning. A non cooperative game consisting of “Players, Action and Reward” is formulated to deal with the contention among SCPs. The players here are the set of SCPs (∀ j, j', j" ∈ Ns) .The strategies based on game theory are the valuation price VRBi , requisition of resources (RRBi) and contract period τ; and reward is the utility obtained using (5). The strategy in bidding the resources is based on reinforcement learning where the strategy made at the current time perioddepends on the previous trading experience. At time period t the set of all strategies selected by the SCPs are represented by the vector and the strategy performed at time period t+1 is represented as
Where 0 ≤ β ≤ 1 is the learning rate. When β is zero, weight is assigned to the current strategy only, when the learning rate is one the action depends the previous trading experience. Now the NCRS algorithm is summarized in Table 3.
Table 3.Non Cooperative Resource Sharing (NCRS)
4.3 Recommendation Entity based Resource Sharing(RERS)
In this approach a recommending entity is proposed which recommends a market price to all carrier providers. The carrier providers are not to exceed recommender’s price in trading. We formulate using Nash bargaining game and the players here are the PCP (Np) and the SCP (Ns). The Nash Bargaining confines the concept of efficiency and fairness [25.26]. The solution is to maximize the product of all the players utility over the minimum utility. The bargaining game personifies detailed bargain procedure, where a PCP starts the game by offering a price p and the SCP accepts or rejects. When the SCP accepts, both parties (PCP and SCP) obtain the utility using (1) and (5) respectively. The solution for bargaining game is expressed as
Where Umin represent the minimum utility obtained by the PCP and SCP in bargaining of resources. The SCP upon rejecting re-offers a new at time period t and the game proceeds until the time deadline t+1. If disagreement is reached at time t+1 both PCPi and SCPj walk out and the bargaining game continues with other PCP and SCP. The algorithm is now shown in Table 4.
Table 4.RERS Algorithm
5. SIMULATION RESULTS & DISCUSSION
The three proposed policies:UFRS, NCRS, and RERS were simulated and analyzed using LTE System level simulator [27]. The simulator set up at link level employ link-adaptation and resource allocation and pre generate many needed parameters mainly to minimize run time execution.The macroscopic path loss between an eNodeB sector and UE includes distance based propagation loss and gain of the antenna.The shadow fading here is represented by a log-normal distribution of mean 0 dB and standard deviation 10 dB.The multiple input multiple output (MIMO) transmission modes enabling transmission diversity and open loop spatial multiplexing (OLSM)) were used for physical channel model which is based on simple zero forcing (ZF) receiver. The spatial layer which multiplexes different data stream basically maps symbol onto ports of transmit antenna.The adaptive modulation and coding (AMC) were incorporated with coding rates between 1/13 and 1 along with 4-QAM, 16-QAM and 64-QAM modulation techniques.
The spectrum sharing scenario was created with multiple primary carrier provider and secondary carrier providers each having a bandwidth of 5 MHz. To start with, the total number of resources available with each carrier providers was fixed at a minimum of 25 RBs. At a particular time period the PCP has free resources (say 8 RBs) leases its RBs to SCPs after serving its own UEs. The SCPs based on the trading policy acquires the RBs from a PCP and then utilize this resource to its own UEs through RAN of PCP. Table 5 lists the main simulation parameters.
Table 5.Simulation Parameters
5.1 UE throughput
Mapping between the UE wideband SINR and the throughput achieved by each UE moving at 5 km/h for three policies is plotted in Fig. 3. The SINR values over the dynamic range of -15 dB to 30 dB.The UE throughput(Fig. 4) as empirical cumulative distribution function (ECDF) for each proposed algorithm, were observed in a typical LTE environment. The mean value is marked in the CDF as a black dot. The ECDF of UFRS and RERS follows closely and rate of change is relatively higher for UFRS which is attributed to the simplified trading procedure in UFRS.
Fig. 3.SINR to throughput mapping
Fig. 4.The ECDF of UE throughput
The mean and peak UE throughput for the UFRS, NCRS, and RERSare shown in Fig. 5 and values are listed in Table6 along with fairness index. The reason for higher peak and average throughput in NCRS could be the adoption of the reinforcement learning method with Nash equilibrium.Another advantage with NCRS is higher level of fairness.Furthermore,the NRCS policy does not require any co-operation among the SCPs. It is interesting to note that the ECDF of throughput observed during simulationfor UFRS follows a steeper curve than NCRS (Fig. 4). The RERS policy could be an intermedia choice but it requires a third party as a recomender entity.
Fig. 5.Comparison of mean and peak throughput
Table 6.Comparison of Fairness, Peak and Average Throughput
5.2 Spectral Efficiency
The ECDF of the spectral efficiency of three approaches were observed during simulation. All the three policies show the similar pattern of spectral efficiency (Fig. 6). The adaptive modulation and coding techniqe were adopted by the LTE system simulator based on the prevailing scenario.To implement AMC,although the coding rates were varried between 1/13 and 1,the order of modulationwere chosen among 4-QAM, 16-QAM and 64-QAM based on the channel conditions. The small variation in spectral efficinecy for the UFRS, NCRS, and RERS policy could be attributed to their data stream handling process. As can be seen in Fig. 6, the ECDF of spectral efficiency follows an usual pattern for all three proposed policies, thereby justifying the effectiveness of these techniques for dynamic spectrum access.
Fig. 6.The ECDF of spectral efficiency
6. Conclusion
To facilitate short term spectrum trading at system level three policies were formulated, simulated and analyzed for the future LTE based cognitive radio systems. The incentives to different trading policies were based on estimation of utility of each player. When there is a co-operation among PCPs and SCPs (UFRS algorithm), it results in higher throughput. The third policy (RERS algorithm) was formulated on Nash Bargaining game. A further analysis and formulation is needed that will guarantee Nash equilibrium among players. All the three proposed policies were targeted in achieving higher overall throughput and resource utilization thereby benefiting each carrier provider. This could be highly desirable in future wireless networks as demand of radio resources by high end user equipment and its applications varies in a given time and location. Another future work could be the use of higher degree of learning to evaluate and adapt in a given scenario.
References
- ITU-R SM.2152 Y.2009, "Definitions of Software Defined Radio (SDR) and Cognitive Radio System (CRS)," Tech. Rep., Year 2009.
- ITU-R 241-2/5, "Cognitive Radio System (CRS) applications in the land mobile service," Annex 26 Doc.5A/306-E, Working Party 5A Chairman's Report, pp. 1-56, 3 June 2012.
- ITU- R 241-2/5, "Cognitive Radio System applications in the land mobile service," Annex 22, Doc. 5A/198-E, Working Party 5A Chairman's Report, pp.1-55, 20 November 2012.
- Ekram Hossain, Dusit Niyato, Zhu Han, "Dynamic Spectrum Access and Management in Cognitive Radio Networks," Cambridge University Press, 2009.
- Adrian Foster, "Spectrum Sharing and Tarrifs- Impact of Sharing on Prices," a Seminar on economic and financial aspects of telecommunications Study Group 3(SG3RG-LAC), Y.2011.
- Network sharing MORAN and MOCN for 3G, Nokia Siemens Networks, May 2013.
- DusitNiyato, Ekram Hossain and Zhu Han, "Dynamics of Multiple- Seller and Multiple-Buyer Spectrum Trading in Cognitive Radio Networks: A Game- Theoretic Modeling Approach," IEEE Transactions on Mobile Computing, Vol. 8, No. 8, pp. 1009-1022, August 2009. https://doi.org/10.1109/TMC.2008.157
- Lin Gao, Jianwei Huang, Ying-Ju Chen and BiyingShou, "An Integrated Contract and Auction design for Secondary Spectrum Trading," IEEE Journal on selected Areas in Communications, Vol. 31, No.3, pp. 581-592, March 2013. https://doi.org/10.1109/JSAC.2013.130322
- Lei Yang,Hongseok Kim, Junshun Zhang, Mung Chiang, Chee Wei Tan, " Pricing- Based Decentralized Spectrum Access Control in Cognitive Radio Networks," IEEE/ ACM Transactions on Networking, Vol. 21, No. 2, pp. 522-535, April 2013. https://doi.org/10.1109/TNET.2012.2203827
- Dhananjay Kumar, Kanagaraj. N.N, R.Srilakshmi, " Harmonized Q Learning for Radio Resource Management in LTE based Networks," in the proceedings of Building Sustainable communities, ITU- T Kaleidoscope, pp. 95-102, April 2013.
- Sanjay Kumar Suman, Dhananjay Kumar and L Bhagyalakshmi, "SINR Pricing in Non Cooperative Power Control Game for Wireless Ad Hoc Networks," KSII TransactionsonInternet and Information Systems, Vol. 8, No. 7, pp. 2281-2301, July 2014.
- DusitNiyato, Ekram Hossain, "Spectrum Trading in Cognitive Radio Networks: A Market Equilibrium Based Approach," IEEE Wireless Communications, Vol. 15, No. 6, pp. 71-80, Dec 2008. https://doi.org/10.1109/MWC.2008.4749750
- TariqElkourdi, Osvaldo Simeone, " Spectrum Leasing Via Cooperation With Multiple Primary Users," IEEE Transactions on Vehicular Technology, Vol. 61, No. 2, pp. 820-825, February 2012. https://doi.org/10.1109/TVT.2011.2181967
- DusitNiyato, Ekram Hossain, "Competitive Pricing for Spectrum Sharing in Cognitive Radio Networks: Dynamic Game, Inefficiency of Nash Equilibrium and Collusion," IEEE Journal on Selected Areas in Communications, Vol. 26, No.1,pp. 192-202, January 2008. https://doi.org/10.1109/JSAC.2008.080117
- Liang Qian, Feng Ye, Lin Gao, XiaoyingGan, Tian Chu, XiaohuaTian, Xinbing Wang and Mohsen Guizani, " Spectrum Trading in Cognitive Radio Networks: An Agent- Based Model under Demand Uncertainty," IEEE Transactions on Communications, Vol. 59,No. 11, pp. 3192-3203, November 2011. https://doi.org/10.1109/TCOMM.2011.100411.100446
- Sungjin Jang, Jongbae Kim, JungwonByun, and Yongtae Shin "Game Theory based Dynamic Spectrum Allocation for Secondary Users in the Cell Edge of Cognitive Radio Networks," KSII TransactionsonInternet and Information Systems Vol. 8, No. 7, July 2014,pp. 2231-2245.
- Gonzalo Vazquez- Vilar, Carlos Mosquera and Sdharman K. Jayaweera, "Primary User Enters the Game: Performance of Dynamic Spectrum Leasing in Cognitive Radio Networks," IEEE Transactions on Wireless Communications, Vol. 9,No. 12, pp. 3625-2629, December 2010.
- Hyoung- Jin Lim, Moon- Gun Song and Gi-Hong Im, "Cooperation- Based Dynamic Spectrum Leasing via Multi- Winner Auction of Multiple Bands," IEEE Transactions on communications, Vol. 61, No. 4, pp. 1254-1263, April 2013. https://doi.org/10.1109/TCOMM.2013.012913.120133
- Yong Xiao, Guoan Bi and DusitNiyato, "Game Theoretic Analysis for Spectrum Sharing with Multi- Hop Relaying," IEEE Transactions on Wireless Communications, Vol. 10, No. 5, pp. 1527-1537, May 2011. https://doi.org/10.1109/TWC.2011.032411.100753
- Kun Zhu, DusitNiyato, Ping Wang, Zhu Han, " Dynamic Spectrum Leasing and Service Selection in Spectrum Secondary Market of Cognitive Radio Networks," IEEE Transactions on Wireless Communications, Vol. 11, No. 3, pp. 1136-1145 ,March 2012. https://doi.org/10.1109/TWC.2012.010312.110732
- Dan Xu, Xin Liu and Zhu Han, "A Two- Tier Market for Decentralized Dynamic Spectrum Access in Cognitive Radio Networks," in Proc. of 7th Annual IEEE Communications Society Conference on Sensor Mesh and Ad Hoc Communications and Networks (SECON), 21-25 June 2010.
- Jain, R, Chiu, D.M, and Hawe, W., "A Quantitative Measure of Fairness and Discrimination for Resource Allocation in Shared Computer Systems," DEC Research Report TR-301,
- M. Osborne and A. Rubinstein, "Bargaining and Markets," Academic Press, 1990.
- ShiranRachmilevitch, "Fairness, Efficiency, and the Nash Bargaining Solution," Journal of Economic Literature, JEL Codes: D63; D71, September 21, 2011.
- J.C. Ikuno, M. Wrulich and M. Ruppo, "System Level Simulation of LTE Networks," in Proc. of 2010 IEEE 71st Vehicular Technology Conference, Taipei, Taiwan, May 2010.