Conflicts in Overlay Environments: Inefficient Equilibrium and Incentive Mechanism

Abstract

Overlay networks have been widely deployed upon the Internet by Service Providers (SPs) to provide improved network services. However, the interaction between each overlay and traffic engineering (TE) as well as the interaction among co-existing overlays may occur. In this paper, we adopt both non-cooperative and cooperative game theory to analyze these interactions, which are collectively called hybrid interaction. Firstly, we model a situation of the hybrid interaction as an n+1-player non-cooperative game, in which overlays and TE are of equal status, and prove the existence of Nash equilibrium (NE) for this game. Secondly, we model another situation of the hybrid interaction as a 1-leader-n-follower Stackelberg-Nash game, in which TE is the leader and co-existing overlays are followers, and prove that the cost at Stackelberg-Nash equilibrium (SNE) is at least as good as that at NE for TE. Thirdly, we propose a cooperative coalition mechanism based on Shapley value to overcome the inherent inefficiency of NE and SNE, in which players can improve their performance and form stable coalitions. Finally, we apply distinct genetic algorithms (GA) to calculate the values for NE, SNE and the assigned cost for each player in each coalition, respectively. Analytical results are confirmed by the simulation on complex network topologies.

1. Introduction

Overlays are logical networks built above the physical network, which can improve the network performance without modifying the underlay network. Over the past few years, a wide variety of overlay networks have been deployed upon the Internet by Service Providers (SPs) to provide different kinds of services, such as content delivery network (CDN), peer-to-peer network (P2P) and resilient overlay network (RON) [1]. Although these overlay applications improve the performance of traditional IP layer routing, the interaction between each overlay and underlay network, as well as the interaction among multiple co-existing overlay networks may occur.

The overlay usually optimizes its performance in terms of specific performance metrics, e.g., minimizing the total delay cost. However, Internet Service Provider (ISP) adopts traffic engineering (TE) to optimize the global cost of the network, such as balancing the network load [2]. As the emerging overlays allocate traffic in the logical layer according to their own objectives, the established TE routing strategy may lead to sub-optimization for the underlay network. Then, TE is triggered to readjust the routes and the new physical routes may turn back to affect the performance of overlays [3]-[7]. The misalignment of objectives between overlay routing and TE makes them interact with each other, e.g., overlay routing’s action affects the demand matrix of TE and TE’s action affects the overlay traffic’s routes. Therefore, the interaction between each overlay and TE affects the stability and optimality of the global network.

On the other hand, when multiple co-existing overlays are deployed upon the same physical network, their overlay routes may overlap each other since a physical link may belong to several overlay routes at the same time. These overlays compete for physical resources to optimize their own performance regardless of the impact on others, and they can interact with each other by adjusting the traffic on the overlapping routes [8]-[12]. Therefore, the interaction among co-existing overlays also affects the stability and optimality of overlay networks. For simplicity, we use the term hybrid interaction to represent the interaction between each overlay and TE and the interaction among co-existing overlays, which are shown in Fig. 1.

Fig. 1.Hybrid interaction in multiple overlay environments

This paper focuses on studying the hybrid interaction of a scenario where multiple co-existing overlays are built upon the physical network of ISP and figure out how the hybrid interaction affects the performance of both overlays and TE. Since ISP provides the physical network for SPs in reality, its status should be equal or higher than SPs. Thus, we adopt two non-cooperative game models to analyze the hybrid interaction. We assume that the overlay’s objective is to minimize its own total delay cost and TE’s objective is to minimize the total congestion cost of the underlay physical network. In this paper, we make the following main contributions:

First, we model a situation of the hybrid interaction as an n+1-player non-cooperative game, where overlays and TE have equal status and the hybrid interaction between players ends up with a stable state that is Nash equilibrium (NE). We prove the existence of NE, which can be achieved through dynamic best response.

Second, we model another situation of the hybrid interaction as a 1-leader-n-follower Stackelberg-Nash non-cooperative game, where TE is the leader and overlays are followers. In this game, TE has higher status than overlays and plays its routing strategy first, and then all the overlays react optimally. We prove that the cost at Stackelberg-Nash equilibrium (SNE) is at least as good as that at NE for TE. Third, in order to improve the performance of NE and SNE, we adopt a coalition game to explore a cooperative approach for co-existing overlays and TE. Our cooperative approach considers Pareto efficiency and fairness, where the players in the coalition cooperate to optimize the performance of the coalition and share costs based on Shapley value.

Last, we apply genetic algorithms (GA) to calculate NE, SNE and the assigned cost for each player in the coalition game. Our work is different from the previous studies in the following aspects. Firstly, we focus on the hybrid interaction among co-existing overlays and TE, which is the combination of [3]-[7] and [8]-[12]. Researchers in [3]-[7] focused on the interaction between overlay routing and TE in the single overlay scenario without studying the scenario of multiple overlays, and researchers in [8]-[12] focused on the interaction among co-existing overlays without considering the effect of TE. Secondly, we are the first to propose a cooperative coalition approach based on Shapley value to solve the efficiency loss in the hybrid interaction, which generalizes the Nash bargaining solution in [6] and [10].

The rest of the paper is organized as follows. Related work is given in Section 2. Section 3 formulates the optimization problem of multiple co-existing overlays and TE. In Section 4, we model the hybrid interaction as an n+1-player non-cooperative game. Section 5 models the hybrid interaction as a 1-leader-n-follower Stackelberg-Nash game. In Section 6, the coalition game based on Shapley value is introduced. Section 7 provides the simulations on real networks and Section 8 concludes the paper.

 

2. Related Work

Game theory [13] has been extensively used in networking research. In the area of networking, a user equilibrium modeling the interaction between users as the standard network optimization problem was proposed by Roughgarden [14]. Liu et al. [3] studied the interaction between one single overlay and TE by using best-reply dynamics and demonstrated the impact of overlay routing on the underlay network. Wang et al. [4] studied the non-cooperative interaction between the P2P overlay and TE and pointed out the non-optimal performance of the network. Jiang et al. [8] studied the interaction between multiple co-existing overlays on top of a physical network and proved that the interaction may cause the efficiency loss and fairness paradox in multiple overlay routing. Keralapura et al. [9] studied the interaction among co-exisiting overlays competing for limited network resources. Wu et al. [15] adopted a dynamic auction game to study conflicts among co-existing streaming overlays. Ma and Misra [16] studied the role of congestion in network equilibrium. Xiao et al. [7] modeled the interaction between overlays and underlay networks in multi-domain networks as a congestion game and provided some operational guidelines to ensure system stability.

Researchers have also explored some ways to solve the conflicts in the interaction. Seetharaman et al. [17] proposed preemptive strategies to improve the routing performance of overlays and underlay networks. Kwon et al. [18] introduced overlay agents to explore cooperation between heterogeneous co-existing overlays. Jiang et al. [8] proposed a pricing scheme to improve the performance of overlays. Gong et al. [5] adopted a repeated game to reduce the oscillations between overlay and TE. Seetharaman et al. [19] developed three strategies to increase the underlay awareness at the overlay layer. Cohen et al. [20] studied the optimization problem of deploying overlay nodes. Wang et al. [11] studied the collaborations of multiple selfish overlays by using multi-path resources. Yang et al. [12] studied the interaction among multiple co-existing P2P systems and proposed an ISP-friendly inter-overlay coordination framework to control P2P traffic.

Cooperative game theory can be applied as an alternative way to overcome the inefficiency of NE. Jiang et al. [6] and Cui et al. [10] adopted a Nash bargaining theory to improve the inefficiency of NE. However, Nash bargaining can only be applied to the game with two players. When there are more players in the game, the problem becomes more complex. Ma et al. [21] applied Shapley value to network environments for ISP settlement. Niyato et al. [22] considered a mobile cloud computing environment in which cooperative SPs can form a coalition to create a resource pool to support the mobile applications and share the revenue obtained by the resource pool. Misra et al. [23] proposed a ideal incentive structure based on Shapley cooperative theory so that each content provider can receive a fair price for the usage of its resources. In our work, we also apply Shapley value in mulitple overlay environments to implement cost allocation.

 

3. Model and Problem Statement

In this section, we model the overlay and the underlay network, and present the objectives of overlays and TE.

3.1 Physical and Overlay Network Models

Let G = (V, E) represent an underlay physical network, where V is the set of physical nodes, E is the set of physical links and the number of physical links is |E|. Then we define a capacity vector C = (c1,c2,⋯c|E|), where ce is the capacity for each link e ∈ E. In addition, we define a routing set R, where each route r ∈ R denotes a possible route of the underlay network and |R| is the total number of routes. We also define a |E| × |R| physical indicator matrix A where aer = 1 if route r traverses link e, and aer = 0 otherwise.

An overlay s in the logical level is represented by graph G(s) = (V(s), E(s)), where V(s) is the set of overlay nodes and E(s) is the set of overlay links. Each overlay node maps to a physical node, and each overlay link maps to a set of physical routes, which is denoted by r → e(s). We define R(s) as the set of all possible overlay paths in overlay s, where r(s) ∈ R(s) denotes an overlay path and |R(s)| is the total number of overlay paths. Each overlay may have several demands, each of which is a source-sink pair associated with a flow f with traffic demand wf. The traffic of each flow is split and allocated onto several paths. Let denote the set of available paths that can be used by flow f to transfer data. Consider that there are n co-existing overlays on top of an underlay network and let N denote the full set of these overlays. Let Fs denote all demands of overlay s. Also, we consider background demands from underlay users that directly use the underlay network to transfer data. Let Fb represent all background demands. We use set F = ∪s∈N Fs ∪Fb to denote all flows. Let b(i, f), i ∈ N, b denote a logical indicator matrix, where if flow f traverses overlay link otherwise. Here, the source and destination nodes of each background demand are connected with a logical link, so the logical network generated by all background demands can be viewed as an overlay network. Then we rewrite the logical indicator matrix B as: B = (b(1),⋯,b(n),b(b))T, b(i) = (b(i, f1),⋯,b(i, f|Fi|)), i ∈ N, b.

The overlay determines the routing of all demands for its overlay users. For each flow f ∈ Fs, the overlay needs to decide how to assign its traffic wf to possible routes. Thus, we define an allocation vector for flow f, where is the amount of traffic assigned to path r(s) for flow f in overlay s. We have which indicates that the summation of the amount of traffic assigned to all possible paths serving flow f is equal to the traffic demand wf. Similarly, the allocation vector y(b, f) is defined for background flow f ∈ Fb. As no routing policy is applied for those background flows, we have y(b, f) = wf. Then we can write the amount allocation matrix Y as:

The overlay and background users pass their demands to the underlay network, and then TE decides how to allocate these demands onto different physical paths. The traffic on an overlay link is interpreted by TE as a demand between two neighbor overlay nodes. Denote X as a matrix and its element xre(i) is the fraction of the flow in overlay link e(i) that TE allocates to route r, and we have Here, TE does not differentiate demands of overlay users and underlay users, and performs the same fraction allocations for demands with the same source-sink. Then, the amount of traffic on each physical link e can be represented by a vector

where le is the amount of traffic allocated to physical link e.

We say allocation decisions X, Y of overlays and TE are feasible if they satisfy the conditions Y ≥ 0, X ≥ 0, LT ≤ C, i.e., the traffic on overlay links and the fraction of flows on physical routes are non-negative and the total amount of traffic allocated to link e is no more than its capacity ce.

3.2 Traffic Engineering’s Objective

Let Oe(le) denote the congestion function for the physical link e ∈ E, where le is the amount of traffic that traverses link e. The total congestion cost for the physical network is the summation of congestions of all physical links such that In our paper, the objective of TE is to minimize the total congestion cost of the physical network. Thus, the optimization problem of TE can be rewritten as:

where δ = (1,1,⋯,1), |δ| = |E| and denotes the congestion function vector for all physical links. In this optimization problem, the allocation decisions Y from overlays are considered as parameters, and the fraction allocation X is the variable. The optimization operation of TE is to compute the optimal value of X for the physical network.

3.3 Overlay’s Objective

In general, the delay of a physical link is closely related to the total traffic on the physical link. We define de(le) as the delay function of the physical link e ∈ E, which can be approximately represented by the sum of M/M/1 queuing delay and propagation delay [2]. Then the delay for each physical route is the sum of the delay for each physical link that comprises this route, that is, We also define dce(s) as the delay cost for each overlay link e(s) in overlay s. Since each overlay link maps to a set of physical routes and overlay traffic on the overlay link is split and allocated onto mapped physical routes, the delay cost for each overlay link is the weighted summation of the delays of mapped physical routes. Thus, dce(s) is calculated as in which xre(s) is the weight of physical route r for overlay link e(s). The value of weight xre(s) is computed by the previous optimization operation of TE and passed to overlay s as a parameter.

The total delay cost for overlay s can be interpreted as the summation of delay costs of all overlay links for transferring overlay traffic in overlay s, which is calculated as

where le(s) denotes the total amount of overlay traffic on overlay link le(s) such that

Here, we expand the size of b(s) and y(s) to B and Y, respectively, by filling the vacant elements with zero. And we define a vector to denote the delay function for all physical links. In our paper, the objective of overlay routing is to minimize the total delay cost of the overlay network. Thus, the optimization problem of overlay routing can be rewritten as:

where the fraction allocation X of TE and routing decisions y(−s) from other overlays are considered as parameters, and routing decision y(s) is the variable. The optimization operation of overlay routing is to compute the optimal value of y(s) for overlay s.

 

4. N+1-player Non-cooperative Game

In this section, we consider a situation of the hybrid interaction where co-existing overlays and TE have equal status. We model this situation of the hybrid interaction as an n+1-player non-cooperative game, and then propose an algorithm to compute its NE.

4.1 Non-cooperative Game

We define a finite set of players N + 1 = {1,2,⋯,n+1}. The first n players are overlays and the last player is TE. The strategy of overlay s is an amount allocation matrix of its demands, and the strategy of TE is a fraction allocation matrix of all flows. Thus, the set of strategies for each player is described as follows:

where R+ denotes the nonnegative set. Furthermore, let Ui denote the payoff function of overlay i with Ui = -g(i) (y(i)), i = 1,2,⋯,n. Let Un+1 denote the payoff function of TE such that Un+1 = -f (x). Thus, we define the n+1-player non-cooperative game as G, where, Γ = Γ1 × ... × Γn × Γn+1 denotes the set of strategy profiles and U = U1 × ... × Un × Un+1 denotes the set of corresponding utility profiles. We have the following definition of NE:

Definition 1 A feasible strategy profile (Y*, X*) ∈ Γ, (Y*, X*) = (y*(1), y*(2), ⋯, Y*(n), X*) is NE if for each overlay i ∈ N and TE:

Namely, NE describes a situation where no player can improve its own objective by altering its routing strategy unilaterally. NE is a stable state, since all plays do not have inducements to change their strategies.

Theorem 1 In G

Proof: If NE exists, the game should meet the following two conditions [24]: (1) each player’s strategy space Γi is a nonempty compact convex subnet of a Euclidean space; and (2) the preference relation between U is continuous and quasi-concave on Γi. Firstly, as the strategy spaces in G

In order to compute the NE allocation for overlays and TE, we first define the notation of best response. NE is the status where each player adopts its best response.

Definition 2 In G , each player’s best response to the strategies of other players is the strategy that minimizes its objective function, that is,

4.2 Dynamic Best Response

The NE for the above non-cooperative game can be computed by the static or dynamic best response. The static (simultaneous) best response is based on Karush-Kuhn-Tucker (KKT) conditions, in which all routes with non-zero traffic serving the same demand must have the same end-to-end first derivative length [8][14]. The strategies of all players at NE should satisfy the KKT conditions. However, when using the static best response to compute NE, we should assume that each player has the complete knowledge of others. This assumption is too strict because in reality players generally achieve NE through adjusting their strategies continuously. Moreover, when the network topology is complex with many routes, it is very difficult to solve the equation set. Therefore, we use the dynamic best response to compute NE in our work.

When using the dynamic best response to compute NE, players do not know each other at the beginning of the game, and they make their dynamic best responses (i.e., current optimal strategies) based on the current situation. Since players repeatedly interact with each other, they will gradually obtain the knowledge of others and finally have the complete knowledge of others, which then leads to the convergence of NE. Note that the computing of dynamic best response for all players is NP-hard. As genetic algorithm (GA) [25] has demonstrated considerable success in providing good solutions to many complex optimization problems [26], here we apply it to compute the dynamic best response for all players. GA has five key operations: initialization, evaluation, selection, crossover and mutation. To compute the dynamic best response for TE (or overlay s), a chromosome in GA can be denoted as X (or y(s)). We use the following algorithm 1 to compute the dynamic best response for TE:

Similarly, we can use algorithm 1 to compute the dynamic best response for each overlay s by replacing

In order to compute NE, we first give TE and overlay networks an initial allocation, which is an arbitrary feasible allocation of X,Y. Then, TE and overlays take turns to use algorithm 1 to compute their dynamic best responses until they reach NE. We stipulate that a player alters its strategy only when its performance can be improved, otherwise, it keeps its strategy unchanged. The algorithm 2 of computing TE is presented as follows:

4.3 Example

To illustrate how to apply the above algorithms, we use an example network in Fig. 2 (a). The physical network is a 9-node directed graph. There are two co-existing overlays deployed upon the underlay network. We assume each overlay has one single source-sink pair and the overlay flow has 1Mbps traffic demand. In overlay 1, the flow is from A(1) to D(1), which has two overlay paths A(1) - C(1) - D(1) and A(1) - F(1) - D(1). The routing strategy for overlay 1 is such that which means the sum of traffic rates over all paths is equal to the demand of the flow. In overlay 2, the flow is from G(2) to I(2), which has three overlay paths G(2) - E(2) - I(2) , G(2) - F(2) - I(2) and G(2) - H(2) - I(2). The routing strategy for overlay 2 is such that

Fig. 2.(a) An physical network with two co-existing overlays, in which overlay 1’s routing strategy is and overlay 2’s routing strategy (b) The set of physical routes to which each overlay link maps

Fig. 2 (b) lists the set of physical routes to which each overlay link maps. Traffic on each overlay link is interpreted by TE as a flow between two neighbor overlay nodes, e.g., traffic on overlay link A(1) - C(1) is interpreted by TE as the flow from A to C. Besides, there is 1Mbps background demand between every neighbor overlay nodes. Thus, there are ten flows in the physical network. The routing strategy for TE is denoted by matrix X = (xij)14×10 such that x22 +x23 = 1, x34 +x35 = 1, x68 +x69 = 1 and x8,11 +x8,12 = 1 since the sum of fractions allocated to all possible paths is equal to 1. x11,x46,x57,x7,10,x9,13 and x10,14 are equal to 1, because there is only one path for these flows. Here, we set both ce(le) and de(le) to be equal to the amount of traffic that traverses the physical link, i.e., ce(le) = le and de(le) = le.

At each iteration, TE and co-existing overlays use algorithm 1 to compute their dynamic best responses. For TE and two overlays, the chromosomes are X, y(1) and y(2) respectively and 50 feasible chromosomes are initialized at the beginning. Algorithm 1 is a local heuristic search. Its search curve first drops very fast, then drops slowly and finally becomes stationary, where x-axis represents the number of iterations and y-axis represents the cost for TE or overlays. After several iterations, TE and two overlays reach the NE, in which we have y*(1) = (0.356, 0.644), y*(2) = (0.385, 0.033, 0.582) and X* : x22 = 0.753, x23 = 0.247, x34 = 0.874, x35 = 0.124, x68 = 0.253, x69 = 0.747, x8,11 = 0, x8,12 = 1.

 

5. 1-leader-n-follower Stackelberg-Nash Game

In this section, we consider another situation of the hybrid interaction where TE has higher status than all overlays. We adopt a 1-leader-n-follower Stackelberg-Nash game [25] to model this situation, where TE is the leader and all overlays are followers. TE has the complete knowledge of all overlays, such as objective functions, topologies and demands.

5.1 Stackelberg-Nash Equilibrium

In the Stackelberg-Nash game, TE plays its optimal routing strategy first, then all overlays react optimally. All overlays’ strategies depend on TE’s strategy. Let Y(X) denote the strategy set of all overlays dependent on strategy X ∈ Γn+1 of TE, which can be represented by Y(X) = {Y|Y ∈ Γ1 × Γ2 × ⋯ × Γn}. Then, the optimization problem of the 1-leader-n-follower Stackelberg-Nash game can be described as:

This optimization problem is classified as Bilevel Programming (BP) problem [27]. In this Stackelberg-Nash game, all overlays are of equal status. For all overlays, the best solution is the NE among them, which is defined by Y*(X) = (y*(1), y*(2), ⋯, y*(n)) ∈ Y(X) with respect to X. Then, we have the following definition of SNE for the 1-leader-n-follower Stackelberg-Nash game.

Definition 2 A feasible strategy profile (Y*(X*), X*) ∈ Γ, (Y*(X*), X*) = (y*(1), y*(2), ⋯, y*(n), X*) is SNE if and only if

According to this definition, SNE prescribes an optimal strategy for TE, if TE plays first and then all overlays react optimally. We can easily prove that the cost at SNE is at least as good as that at NE for TE.

Theorem 2 In G

Proof: Let be the NE and (Y*(X*), X*) be the SNE. If TE first choose as the leader, holds for all overlays. Thus, we have By the definition of SNE, we have

5.2 Algorithm for Solving Stackelberg-Nash Equilibrium

For the BP problem describled in (8), we also use GA to search the optimal strategy for TE. The following algorithm 3 is proposed to compute SNE:

 

6. Coalition Game

It has been pointed out that NE and SNE in the non-cooperative game are usually inefficient [14]. Thus, in order to improve the performance of overlays and TE at NE and SNE, we propose a cooperative coalition mechanism and a cost allocation scheme by applying the concepts of core and Shapley value to determine the share of cost taken by each player.

6.1 Coalition Game

A coalition game [13] is denoted by , where C ⊆ N + 1 is the coalition representing the set of cooperative players in the same group and v(⋅) is the coalitional function, i.e., v(C) is the objective cost for the coalition C. C = N + 1 is called the grand coalition. There may be multiple coalitions and the set of all coalitions can be denoted as the coalitional structure Φ = {C1,C2,⋯,C|Φ|} such that Ci ∩ Cj = ∅ and Ci ⊆ Φ. For each coalition C, the strategy is the union set of strategies of the players in this coalition and the objective function is to minimize the cost of this coalition, which can be written as:

where λ is the equivalent weight, i.e., 1 unit congestion equals to λ units delay cost. The value of λ is dependent on the negotiation results of TE and overlays or other external factors. Thus, the interaction between coalitions is a non-cooperative game and will end up with NE after several iterations, in which each coalition is regarded as a player. Likewise, we use algorithm 2 to compute NE for the interaction between coalitions.

6.1.1 Core Solution

We define the core among cooperative players in coalition C as:

where z = (zi, i ∈ S), and zi is the assigned cost for player i in coalition C. The core is a set of cost shares, which guarantees that no player leaves the coalition C to form subcoalition S ⊆ C. Namely, the summation of assigned costs by coalition C is always less than or equal to that of assigned costs by any subcoalition Thus, the core solution can make the coalition stabilized.

6.1.2 Shapley Value

We now apply the concept of Shapley value to assign fair cost shares to each player in coalition C. The Shapley value of player i in coalition C can be obtained as follows:

The Shapley value φi (v) determines the cost to be shared by player i, and it is obtained by evaluating the contribution of each player i in reducing the cost of the coalition. The Shapley value enables the cooperative players in coalition C to share cost because of the following properties.

Efficiency: Since the summation of costs of all cooperative players is minimized.

Symmetry: For two arbitrary player i, j ∈ C, if v(S ∪ {i}) = v(S ∪ {j}) holds for all the subcoalition S ⊆ C without these two players, then φi (v) = φij (v). That is, when players i and j have the same contribution to the coalition, the cost shares of the players i and j are equal.

Dummy: For a player i, if v(S) = v(S ∪ {i}) holds for all the subcoalition S ⊆ C without player i, then φi (v) = 0. That is, if player i does not contribute to the total cost of the coalition (e.g., overlay i has no traffic in the network), then cost share of this player is zero.

Additivity: If v and v′ are the coalitional functions, then φ(v + v′) = φ(v′ + v) = φ(v) + φ(v′).

By using the Shapley value, the individual efficiency and fairness can be achieved. Specifically, the cost shared by the cooperative player is less than or equal to the cost of the non-cooperative player (φi (v) ≤ v({i}). Moreover, the Shapley value is unique.

6.2 Coalition Formation

We assume that all players are rational and self-interested to minimize their own costs by forming a coalition. The coalition formation process can be described as a non-cooperative game. The set of players consisting all overlays and TE is {N + 1}. In the game, each player has to decide whether or not to form a coalition with other players. The cooperation between player i and player j can be denoted by a binary variable qij, where qij = 1 if they cooperate and qij = 0 otherwise. The strategy of player i is qi = (qi1,qi2,⋯,qi,n+1). Thus, we rewrite the strategies of all players Q as Q = (q1,q2,⋯,qn+1), qij = qji, i, j = 1,2,⋯,n+1). The feasible set of strategies for each player is described as follows:

The NE for the coalition formation game can be defined as:

The NE of the coalition formation game can be obtained by the dynamic best response. The player makes a decision on how to form coalition with others, who are willing to form coalition, iteratively. In each iteration, the player evaluates the new strategy, and then switches to the new strategy in order to achieve the least cost. The algorithm of computing NE for the coalition formation game is presented as follows:

 

7. Performance Evaluation

In this section, we conduct simulations to evaluate the performance of two situations of the hybrid interaction, and compare them with the performance in coalition game.

7.1 Simulation Setup

Our simulation adopts a 50-node physical network of a single ISP, which comprises one central region and three marginal regions. There are three overlays deployed by the SPs above this physical network. The physical and overlay network structures are shown in Fig. 3. In each of the three overlays, there is one source-sink pair and the traffic demand is equal to 1Mbps. The source and destination nodes for overlay 1 are 48 and 22, for overlay 2 are 5 and 48, and for overlay 3 are 5 and 25. Moreover, there is 1Mbps background demand between every neighbor overlay nodes. Traffic split over multiple paths for TE and overlay can be implemented by means of MPLS [28]. We set the capacity of central links as 3Mbps and the capacity of marginal links as 7Mbps, so that we can simulate the situation where overlays compete for the limited common link bandwidth.

Fig. 3.Physical network with three co-existing overlay networks

The delay function de(le) and congestion function oe(le) for a physical link are chosen as follows. First, the link delay function is set as de(le) = 1/(ce-le)+p, where the queuing delay is approximated by the M/M/1 model 1/(ce-le) and the propagation delay is equal to a constant value p. We set the value p as one in our simulation. Here, the link delay function is continuous, increasing and convex. Second, the link congestion oe(le) is modeled as a piecewise linear, increasing and convex function, which is described as follows [6]:

Besides, we set the parameters in the algorithm 1 as follows, which is used to compute the optimal strategy for players in each iteration. The number of chromosomes m is set as 50, the tolerable error σ is set as 0.001, the parameter α is set as 0.5, the parameter M0 is set as 0.1, the probability of crossover pc is set as 0.8 and the probability of mutation pm is set as 0.5. We conduct simulations based on different parameters and fix the parameters with the best performance in term of computational speed. Note that the setting of parameters will not affect the optimal results but only the calculation time of the algorithm.

7.2 Simulation Results

Our simulation results demonstrate the efficiency loss caused by the hybrid interaction in the network, as well as the variation of routing decisions during the interaction process.

7.2.1 Nash Equilibrium

We start to evaluate the performance of NE, where overlays and TE have equal status. In the simulation, the sequence of interactions executed is TE-overlay1-overlay2-overlay3. In the beginning, there are only background demands in the underlay network for TE and then overlay 1, 2, 3 start to transfer their data in turn. At each iteration, each player applies algorithm 1 to optimize its strategy. In order to reduce the iteration time of the algorithm, we set some initial chromosomes such that TE adopts the shortest route, and set others as arbitrary feasible allocation for TE.

Fig. 4 and Fig. 5 shows the congestion cost for TE and delay cost for three overlays in the iteration process. We observe that a gradual increase of congestion cost and delay cost at the beginning, since overlay 1, 2, 3 start to transfer their data and the traffic in the network increases. Then, we observe some oscillations in the middle of figures, which are caused by the interactions among overlays and TE. Finally, the oscillations subside and come to a stable state, which is NE and the results of NE are: f = 126.2054, g(1) = 11.6089, g(2) = 11.2442, g(3) = 11.2983. The simulation results demonstrate how the hybrid interaction among overlays and TE converges to a stable NE. Note that, in general, the results may vary due to the multiplicity of NE. Nevertheless, the convergence can be attained after certain iterations, which leads to a stable NE. Assume that TE and all overlays are aware of each other’s information, such as topology, demand and the objective function, so they can compute their strategies at NE off-line before the game start, and then directly execute their strategies at NE at the beginning of the game to avoid the network oscillation in the interaction process.

Fig. 4.Congestion cost for TE in the iteration process

Fig. 5.Delay cost for three overlays in the iteration process

7.2.2 Stackelberg-Nash Equilibrium

We then conduct the simulation to compare the performance of SNE with NE. In this simulation, TE executes its optimal strategy first and then overlays react optimally. In order to reduce the search time of algorithm 2, we use the results of NE from the previous simulation as initial chromosomes to search SNE for TE. We choose the strategy with minimum congestion cost to be found as the optimal SNE strategy for TE, since TE will be sure to choose this this strategy to obtain the optimal performance. Finally, we obtain the results of SNE that are: f = 126.1532, g(1) = 11.8404, g(2) = 11.2394, g(3) = 11.0585. The simulation results demonstrate that TE can obtain less congestion cost at SNE than at NE when playing first. Fig. 6 shows the interaction process of overlays after TE executes the SNE strategy. We observe that the interaction among overlays converge to a stable NE state after several iterations.

Fig. 6.Delay cost for three overlays when TE plays SNE strategy

7.2.3 Coalition Cooperation

In this simulation, we evaluate the performance of three overlays (denoted by OR1, OR2 and OR3) and TE with λ = 1 by coalition cooperation. Table 1 shows the Shapley value obtained by each player with different coalition formations. There are totally 15 coalition structures. We can apply Algorithm 4 to reach the stable coalition. We observe that the stable coalition structure is where TE, overlay 1 and overlay 3 cooperate, and overlay 2 is separate. is stable since all players have no better choice than staying in their current coalitions. Note that Φ1 is the situation of n+1-player non-cooperative game. Φ15 is the situation of global optimal routing, which achieves the least total cost of all players. However, if TE is concerned about its own cost, it will leave Φ15 and go to Φ14. And then, overlays take turns to reconsider their strategies. The convergence path of coalition formation is Therefore, is the stable coalition structure for three overlays and TE.

Table 1.Cost Allocation based on the Shapley value

7.2.4 Performance Comparison

In the end, we compare the performance of NE, SNE, friendly preemptive strategy [17], Nash bargaining solution (NBS) [6], and coalition cooperation. Fig. 7 shows the cost obtained by TE and three overlays under five different schemes. We can see that TE and all overlays can obtain better performance by NBS and coalition cooperation than the other three schemes. However, the performance of NBS is not stable, since it does not take into account that the players may form a coalition. The performance of our coalition cooperation scheme is stable. In addition, friendly preemptive strategy is effective to increase the stability of the network, but its performance is always not Pareto optimal. Thus, in multiple overlay environments, our proposed coalition cooperation scheme based on Shapley value will provide a stable, efficient and fair solution for both ISP and SPs to obtain better performance.

Fig. 7.Performance comparison in normal traffic network environments

In the previous simulation, we assume that all users are legitimate users, and the traffic they generate is normal. However, malicious traffic attacks are very common in real networks. Thus, we conduct a simulation to observe the effect of malicious traffic on the performance of TE and overlay networks. We use the same network structures as the previous simulation, from which we choose two source-sink pairs to simulate the malicious traffic. We set the demand of malicious traffic to be equal to 3Mbps. Fig. 8 shows the cost obtained by TE and three overlays under the different schemes. We can see that the costs of TE and three overlays under all schemes are increased by the malicious traffic, since the malicious traffic takes up the network resources, which damages the performance of TE and overlays. Although our coalition cooperation scheme still outperforms other schemes, it is also difficult for our scheme to improve the performance of TE and overlays when the network is heavily burdened by malicious traffic, since the network resources will not be able to satisfy the traffic demands of users. Thus, network security caused by malicious traffic attack is a serious issue, some recent studies have focused on the direction of malicious traffic filtration, e.g., researchers in [29] and [30] proposed a distributed edge-to-edge filtration model, which can precisely detect and filters malicious traffic. With the help of these related works, the impact of malicious traffic on the performance of TE and overlays may be eliminated.

Fig. 8.Performance comparison in malicious traffic network environments

 

8. Conclusion

This paper focuses on the scenario where multiple co-existing overlays are deployed above a physical network. We consider two situations of the hybrid interaction, and model them as an n+1-player non-cooperative game and a 1-leader-n-follower Stackelberg-Nash game, respectively. However, the results for overlays and TE at NE and SNE are inefficient. In order to improve the performance of NE and SNE, we propose a cooperative coalition game based on Shapley value. We observe that the performance of all overlays and TE is significantly improved by the stabilized coalition game. Thus, for ISP and SPs, they can apply proposed coalition cooperation scheme to obtain better performance when operating the networks. However, the delay and congestion cost is inherent non-transferable, the Shapley value needs external coordination, therefore, the approach to reduce negotiation costs and to specify equivalent weight λ is worthy of future study.