Xin Zhang,Bo Qian,Xiaohan Qin,Ting Ma,Jiachen Chen,Haibo Zhou,Xuemin(Sherman)Shen
Abstract—Ultra-dense low earth orbit(LEO)integrated satellite-terrestrial network(ULISTN)has become an emerging paradigm to support massive access of Internet of things(IoT)in beyond fifth generation mobile networks(B5G).In ULISTN,there are two communication modes:cellular mode and satellite mode,where IoT users assessing terrestrial small base stations(TSBSs)and terrestrial-satellite terminals(TSTs)respectively.However,how to optimize the network performance and guarantee self-interests of the operator and IoT users in ULISTN is a challenging issue.In this paper,we propose a cybertwin-assisted joint mode selection and dynamic pricing(JMSDP)scheme for effective network management in ULISTN,where cybertwin serves as the intelligent agent.In JMSDP,the operator determines optimal access prices of TSBSs and TSTs,while each user selects the access mode according to access prices.Specifically,the operator conducts the Stackelberg game aiming at maximizing average throughput depending on the mode selection results of IoT users.Meanwhile,IoT users as followers adopt the evolutionary game to choose an access mode based on the access prices provided by the operator.Simulation results show that the proposed JMSDP can improve the average throughput and reduce the delay effectively,comparing with random access(RA)and maximum rate access.
Keywords—ULISTN,massive IoT,cybertwin,mode selection,dynamic pricing,game theory
With the accelerated development of massive Internet of things(IoT),the amount of potential IoT users,and traffic demands for massive access are ever-increasing[1,2].Compared with the traditional terrestrial network,satellite network has various advantages including global coverage,long-distance communication[3]which allows users to connect to the network at anywhere in anytime[4].Companies and organizations like SpaceX plan to deploy more than 10 000 LEO satellites for broadband communication in the near future[5].Meanwhile,the third generation partnership project(3GPP)has studied on the integration of fifth generation mobile networks(5G)satellite-based access components in the 5G system since Release 15(e.g.,3GPP TR 28.821[6]).Ultra-dense low earth orbit(LEO)integrated satellite-terrestrial network(ULISTN)not only provides global and seamless coverage[7],but also meets the demand of massive access for massive IoT in the future beyond 5G[8].
There are two access modes for IoT users in the ULISTN:cellular mode and satellite mode,i.e.IoT users access terrestrial small base stations(TSBSs)in terrestrial small cells(TSCs)or access terrestrial-satellite terminals(TSTs)in LEObased small cells(LSCs).IoT users who choose their communication schemes between cellular mode and satellite mode are potential users in ULISTN.In LSCs,TSTs work as relays for communications between IoT users and LEOs in satellite mode[9],supporting data offloading of IoT users through Cband and connecting to ultra-dense LEOs through Ka-band for satellite backhaul[10].Due to the inherent characteristics of high user density of massive IoT in ULISTN,it will cause significant interference between IoT users accessing TSCs and LSCs with inappropriate mode selection scheme.In order to reduce the interference in ULISTN and fully utilize the network performance,it is essential for the operator to maintain a balance between the percentage of IoT users accessing TSCs and LSCs when designing a mode selection scheme.Meanwhile,new challenges arise when designing an effective mode selection scheme in such an integrated architecture considering the heterogeneity of ULISTN.
In this paper,we investigate a heterogeneous network architecture where we integrate the terrestrial 5G network with an ultra-dense LEO constellation satellite network to provide connectivity for massive IoT users.Recently,cybertwin is used to represent physical entities for synergic resource management in heterogeneous networks[11].With the assistance of the cybertwin deployed at the edge cloud,the mode selection of IoT users in heterogeneous ULISTN could be managed effectively.We propose a cybertwin-assisted two-stage game of mode selection and dynamic pricing for network management in the ULISTN.The operator establishes the objective function and uses optimization to derive the best pricing approach in order to maximize the average network throughput of ULISTN.IoT users in ULISTN analyze the cost and performance different access before making a non-cooperative decision based on replicator dynamics.The contributions are highlighted as below:
•Cybertwin-assisted joint network management scheme:
We propose a cybertwin-assisted joint mode selection and dynamic pricing(JMSDP)scheme in ULISTN.Cybertwin ensures the global information exchange and promotes the game between heterogeneous entities including terrestrial base stations(TBSs),TSTs and IoT users.
•Closed form expressions of network performance:We analyze the network performance including outage probability and average rate of IoT devices accessing both C-band and Ka-band communications using the tool of stochastic geometry.Closed-form expressions of the relationship between mode selection of IoT users and network throughput of ULISTN are derived.
•Stackelberg game-based mode selection:We formulate the Stackelberg game in ULISTN where the operator works as the leader and IoT users as the followers.The operator makes the optimal pricing strategy through optimization while IoT users independently determine their access modes based on the replicator dynamics of evolutionary game.
The rest of this paper is organized as follows.In section II,we review the related works.In section III,we propose a cybertwin-assisted JMSDP scheme.In section IV,we analyze the outage probability and average rate.In section V,an evolutionary game is proposed for user access mode selection.And dynamic pricing strategy is proposed in section VI.We simulate the algorithms and analyze the results in section VII.Finally,we draw the main conclusions in section VIII.
Cybertwin serving at the edge cloud plays an important role in holistic network virtualization in the development of 6G promoting the implementation of network intelligence of 6G[12].And cybertwin is widely used to orchestrate multiple network resources in heterogeneous networks including cellular-Wi-Fi network,and space-air-ground integrated network[11].Yu et al.[13]propose a cybertwin-based network architecture for heterogeneous networks.Under this architecture,cybertwin supports real-time heterogeneous resource orchestration based on the trading platform.Yin et al.[14]consider physical layer security in cybertwin-based integrated satellite-terrestrial vehicle networks.This paper designs a beamforming optimization scheme based on the information gathered at each cybertwin.Meanwhile,cybertwin is also widely applied in IoT.Minerav et al.conducts a comprehensive survey of cybertwin for IoT and sums up the architectural models of cybertwins in IoT networks[15].Kumar et al.propose algorithms guiding the operation of cybertwins for offloading of IoT in mobile multi access edge computing[16].
Most of the existing works focusing on the integrated satellite and terrestrial networks use methods including optimization theory,bipartite graph matching techniques[9]and machine learning algorithms[17,18].Deng et al.[19]propose a pricing mechanism based on the Stackelberg game to motivate terrestrial operator and satellite operator for efficient data offloading in ULISTN.Objective optimization is used to achieve the equilibrium using the golden-section search and subgradient method.Di et al.[7]investigate the backhaul scenario and has maximized the sum data rate and the number of accessed users in the backhaul scenario of satellite-terrestrial integrated network using objective optimization and swap matching algorithm.Du et al.[20]present a mechanism for traffic offloading and spectrum sharing,which is based on secondprice auctions to achieve collaboration and competition between satellite-terrestrial beam groups and cellular base stations.The self-interests of IoT users and operators are not jointly considered in these optimization methods where the optimization objective is the function of the interests of the operator or user only.
Recently,game theory is widely utilized as an effective tool in the design of algorithms for mode selection considering multi-party interests in the heterogeneous network scenarios[21,22].In the previous literature,Qian et al.[23]propose a dynamic Stackelberg game architecture for VAHN which enables three-mode selection and spectrum sharing in which vehicle users choose access mode following evolutionary game independently and dynamically.Yan et al.[24]provide a hierarchical Stackelberg game framework for the challenge of unmanned aerial vehicle(UAV)mode selection and bandwidth resource allocation in a UAV assisted IoT communication network.Liang et al.[25]design a joint access selection and bandwidth allocation scheme in heterogeneous wireless networks,where users choose the optimal access mode with the supervised learning method along with the neural network.Du et al.[26]propose a Stackelberg differential game for resource trading and an evolutionary game for user service selection in 5G wireless heterogeneous networks.
Motivated by the above works,in this paper we jointly consider the interests of the operator and IoT users where the average throughput of the ULISTN and the payoff of IoT users are taken into account.We propose a two-layer Stackelberg game for effective access selection and dynamic pricing with the assistance of cybertwin which serves as the intelligent agents in ULISTN.
In this section,we consider the communication mode selection for massive access in the ULISTN as shown in Fig.1,where IoT users can offload the data to the core network using one of the two modes:cellular mode and satellite mode.Major notations are summarized and described in Tab.1.
Fig.1 Cybertwin-assisted massive access for IoT in ULISTN
Tab.1 Variables list
A.ULISTN Architecture
We investigate a cybertwin-assisted ULISTN architecture shown in Fig.1 which consists of two spaces including physical space and cyber space.In physical space,there are massive IoT users with the need for data offloading,multiple LEO satellites together with base stations(BSs)including TSBSs and TSTs.IoT users can also be mobile users and the proposed JMSDP scheme is also suitable for mobile users to access ULISTN.
Cyber space consists of cybertwins reflecting the entities in physical space that are shown as below[14]:
•IoT-CT:represents the cybertwin of IoT.IoT-CT is deployed at the edge cloud which supervises the network status and serves as the intelligent communication agent.
•LEO-CT:represents the cybertwin of LEOs.LEO-CT is deployed at a terrestrial satellite gateway holding information about the orbit data and status of LEOs.
•TSBS-CT:represents the cybertwin of TSBS and is deployed at the edge cloud managing the communications between TSBS and IoT.
•TST-CT:represents the cybertwin of TST deployed at the edge cloud which has a similar function as TSBS-CT.
•O-CT:represents the cybertwin of the operator.It can monitor the operation status of the network and manage the resources in ULISTN.
Ultra-dense LEO satellite networks are considered where LEOs are deployed atnvarious orbit heightsH1,H2,···,Hn(H1 1.Cellular mode:IoT users can connect to the network via 5G TSC over C-band.In this case,user access is provided via orthogonal frequency division multiplexing(OFDM)with frequency reuse. 2.Satellite mode:IoT users access the ultra-dense LEOs via LSCs.For LSC with TST work as a relay,IoT users first offload the data to the TST and TST then uploads the users’data to LEO satellites.The transmission process in this mode can be divided into two parts:terrestrial part and ground to satellite(G2S)part.In G2S,the spot beam coverage scheme alone with multi-frequency time-division multiple access(MF-TDMA)[27]is used in satellite mode for data transmission in LEO satellite.In this scheme,TST gets access to the LEOs using different time slot and frequency where interference is not considered between TSTs. 3GPP Release 16[28]states that a distinct spectrum is used by the ground and G2S,where the G2S part of satellite mode shares the spectrum with cellular mode.Each mode uses a round-robin(RR)scheduling method where various subchannels are repeated with an equal likelihood.The average amount of bandwidth allocated to each cellular mode user and satellite mode user are whereλTSBSandλTSTdenote the density of TSBSs and TSTs,Bcis the subchannel bandwidth of C-band communications,lcis the number of subchannels in C-band communications,xcandxsare the proportion of the cellular mode population and the satellite mode population.And the average bandwidth allocated to each TST via Ka-band can be calculated as whereλTSTis the density of TSTs,Nis the number of accessible LEO,Bsis the subchannel bandwidth in Ka-band communication andlsis the number of subchannels in Ka-band communication. B.Channel Model The propagation channel is affected by both large-scale attenuation and small-scale fast fading.Terrestrial links over C-band are modeled as Rayleigh fading channels in this paper with power gain|hc|2,where|hc|2follows exponential distribution of parameterµ The signal received at BSs including TSBSs and TSTs sent by the userkcan be expressed as wherePcis the transmission power of C-band communications,hcandgiare channel gain,ηcis the path loss coefficient,dcis the distance between the BS and userk,liis the distance between userkand usericonsidering co-frequency interference,N0cis white noise following Gaussian distribution. Data transmission between TST and LEO satellite is typical line of sight(LOS)communication that can transmit and receive data only where the transmitter and receiver are in view of each other without any sort of obstacle between them.In this paper,the LOS terrestrial-satellite links are modeled as shadowed Rician fading channels[29]with power gain|hs|2,wherehsfollows Rician distribution whereKis the Rician fading factor which denotes the ratio of the power in the dominant component to the average power in the diffuse component andI0(·)is the modified Bessel function of the first kind and zero order. LEO satellite constellation uses a hybrid spot beam for data transmission between satellite and TSTs.Co-frequency interference between the TSTs is not considered in this mode.The data received at the satellite can be expressed as wherePkais the transmission power of G2S part in satellite mode via Ka-band,hsis channel gain,ηsis the path loss coefficient,dsis the distance between the satellite and the TST,N0sis white noise following Gaussian distribution. C.Cybertwin-Assisted Two-Stage Scheme Cybertwin serves as an intelligent assistant between IoT users and the operator[13].Cybertwins in the same cyberspace can exchange information with each other.A cybertwinassisted JMSDP scheme is proposed in this paper which can be divided into two stages.In cybertwin-assisted JMSDP,OCT serves as the intelligent regent of the operator acts as the game leader and IoT-CTs representing IoT users act as the follower as shown in Fig.2.The exact process of dynamic pricing and mode selection is described below. Fig.2 Cybertwin-assisted two-stage JMSDP scheme •Dynamic Pricing for the O-CT:The operator which acts as the leader at the terrestrial base station decides the access price for both cellular mode and satellite mode at the beginning of each time slot.The prices of accessing cellular mode and satellite mode at timetare denoted aspc(t)andps(t)respectively.At the beginning of each time slot wheret=nT(n=0,1,2,···)andTis the time length of every slot,the operator broadcasts the current pricepc(t)andps(t)to every potential user in coverage.Meanwhile,the average transmit rate of both modesRc(t)andRs(t)are broadcast to all potential users at the same time.Then the operator receives the mode selection choice and current transmission rate returned by each user. •Mode Selection for IoT-CTs:Potential users in this twostage pricing game act as followers who receive the dynamic pricespc(t)andps(t)of the two modes broadcast by the operator at the beginning of each time slott.Each IoT-CT would independently choose its communication mode with its own different game strategy based on the current access price and average rate of each mode.After IoT-CTs determine their access modes,the selected results are delivered to their corresponding IoT users and returned to the O-CT together with their current transmission rates. •Two-stage dynamic pricing game:A two-stage dynamic pricing game is proposed in this paper as shown in Fig.2 which consists of pricing optimization operated at the operator and evolutionary game operated at each IoT user terminal.The operator exchanges information with the IoT users at the beginning of each time slot.The pricing optimization scheme and evolutionary game are operated during each time slot. The outage probability of cellular mode and satellite mode is analyzed respectively based on stochastic geometry to evaluate the impact of mode selection on the network throughput of the ULISTN.Due to the high mobility of satellite,the instantaneous data transmission rate is time-varying.Average rates of both modes are derived from outage probability to further depict the network performance.In this section,we will analyze the outage probability and average rate for cellular mode and satellite mode by using stochastic geometry. A.Outage Probability and Average Rate Definition 1The probability that the signal to interference plus noise ratio(SINR)is less than some numbervis defined as the outage probability[30](Pr(v)),as shown below. whereγdenotes the SINR andvis SINR outage threshold.This equation indicates that outage probability also represents the cumulative distribution function(CDF)of SINR. Definition 2The average rate is defined as the average transmission rate in statistics taking into account all possible channel states.Average rate can be calculated as the product of average bandwidth and spectral efficiency as below Average rate evaluating the rate of transmission mode in statistics is significantly impacted by outage probability.By denoting log(1+γ)asx,the relationship between the outage probability and average rate can be derived where Pr(v)is the outage probability andis the average bandwidth allocated to the user. Based on the aforementioned equations,the outage probabilities and average rates which are used as the utility functions for user access mode selection game of both modes are analyzed in the following subsections. B.Outage Probability and Average Rate for TSC and LSC In ULISTN,we assumed that IoT users,TSBSs and TSTs are randomly distributed in a two-dimensional plane of circular region following PPP random distributions.IoT users of cellular mode and satellite mode are assumed to connect to their most nearby TSBS and TST respectively.The probability density function of the distance between the user and the BSs including TSBSs and TSTs is derived as whereR0is the coverage radius of TSBS or TST. The SINR can be derived from(4)wherepcis the transmission power of cellular mode,hcandgiare channel gain,ηcis the path loss coefficient,dcis the distance between the BS and userk,liis the distance between userkand usericonsidering co-frequency interference,n0is white noise following Gaussian distribution. The outage probability of TSBSs and TSTs is According to the definition of the Laplace transform and the probability generating function[31]of PPP,we can get the results of special cases forη=3 andη=4. Special caseη=3: In special caseη=3,the Laplace transform included in the integral term in(12)can be simplified as And the outage probability and average rate can be simplified respectively as Special caseη=4: In the special case ofη=4,the outage probability and average rate are derived and simplified respectively as below. C.Outage Probability and Average Rate for LEO Network TSTs are all equipped with multiple antennas allowing them to connect with multiple LEOs at the same time slot.In satellite mode,TSTs work as a relay which uploads IoT users’data to the LEOs.LEO satellites use multi spot beams of four-color frequency multiplexing[32],where the total frequencies are divided into four pools with each border across a different frequency pool.With the help of frequency and spatial multiplexing[27]among beams,the co-frequency interference between the TSTs is orders of magnitude less than the transmit signal from TST to LEO satellite and can be ignored. Currently the Walker-Delta constellation with a circular orbit geometry is a popular satellite constellation widely applied in LEO satellites including Starlink,Oneweb[27].In this paper,ultra-dense LEO satellites are modeled as a combination ofnWalker-Delta constellation[33]where both orbit planes and satellites are evenly spaced and the orbit height of LEO satellites inside each Walker constellation is the same.Forith Walker constellation,misatellites are distributed at the sphere{Sk}with the altitude ofHi.The total number of satellites in the corresponding ultra-dense LEO constellation is wheremiis the number of satellites deployed at theith sphere plane{Si}. We assume that TST is in the LEO’s communication coverage if an LEO satellite is in TST’s line of sight.Assuming LEO satellites are distributed at the orbit height ofHi,TST can communicate with the LEO if the LEO satellite is in TST’s line of sight and the evaluation angle between TST and LEO is larger than certainθ.The CDF of the distance between TST and LEO is determined by spatial solid geometry illustrated in Fig.3 as Fig.3 Geometric position of LEO satellites and TST Lemma 1The CDF of the distancedsbetween TST and LEO follows where the complementary cumulative distribution function(CCDF)ofdiis whereDi=−Rsinθis the maximum connection distance between TST and the LEOideployed at altitudeHiwith the maximum evaluation angle ofθ,miis the number of LEOs which deployed at theith sphere with heightHi. The probability density function(PDF)of the distancedsbetween the TST and LEO can be deduced as where(x)is the derived function ofFdi(x).The expression of SINR in satellite mode can be derived from(6)as wherepsis the power of transmission from TST to LEO,hsdenotes channel gain between TST and LEO,ηsis the path loss coefficient between TST and LEO,dsis the distance between TST and LEO,andn0is Gaussian white noise. The outage probability of the TST to LEO part of satellite mode is wheref|hs|2(y)is the PDF of|hs|2andfdsis the PDF of the distance between TST and LEO satellite. The average rate of Ka-band communications between TST and LEOs can be calculated as whereE[log(1+γs)]can be expressed as whereθdenotes the maximum elevation of LEO satellite,Hrepresents the orbital height of LEO satellite,f|hs|2(y)is the PDF of|hs|2. For potential mode selection IoT users,we first propose an evolutionary game in this section.The payoff function of each IoT user is set up according to the average transmission rate deduced in the previous section and the price charged by the operator.Replicator dynamics is then used to describe the adaptation rate of each population.Finally,we prove the convergence of evolutionary game played by IoT users. A.Evolutionary Game Formulation The evolutionary game for user access mode selection is defined as follows: •Players:Players in this user access mode selection game are IoT users who choose their communication schemes between cellular mode and satellite mode. •Strategy:Players in this game choose strategies including cellular mode and satellite mode represented byc,srespectively.The set of strategies isS={c,s}. •Population:In this user access mode selection game,the set of users which have the same strategy set constitutes the cellular mode population and the satellite mode population. •Population Share:The number of IoT users choose strategyc,andsis denoted asnc(t)andns(t)respectively at timet.Then the cellular mode population and the satellite mode population of IoT users can be represented asxc(t)=nc(t)/Nandxs(t)=ns(t)/Nrespectively. •Payoff:The payoff function measures satisfaction of user adopting different strategies.In this paper,we assumed the satisfaction level of each user is influenced by data transmission rate and service price.Thus the payoff function of an access mode selection game user is described as the linear combination of the access price and average rate for each mode whereαis the product constant for average rate,βis the product constant for price,rc(t)andrs(t)represent the average transmission rates that have been normalized for cellular mode and satellite mode •Average Payoff:The average payoff of all IoT users is wherexcandxsare the proportion of the cellular mode population and the satellite mode population respectively,ucandusdenote the payoff in cellular mode and satellite mode. B.Replicator Dynamics for Evolutionary Game IoT users compare their payoff with the average in each iteration of the evolutionary game to determine whether to change their access mode.The replicator dynamic is satisfied by the mutation rate of the population share in the manner shown[23]as whereσ>0 is the learning rate that is used to regulate IoT users’observation and adaptation speed while choosing an access mode in the evolutionary game. Lemma 2∀t∈[0,∞),˙xc(t)+˙xs(t)=0 andxc(t)+xs(t)=1. ProofSubstituting formula(29)into formula(30),we can obtain that C.Evolutionary Equilibrium Analysis In this subsection,we find the solution to equilibrium of the proposed evolutionary game.The stability of evolutionary equilibrium in the evolutionary game for user access mode selection is analyzed. The group converges to stable state where the population shares of different modes stay unchanged and the payoff function of each individual stays unchanged through the evolutionary game[35].This indicates that the rate of access mode adaptation is zero at the evolutionary equilibrium point with Definition 3x⋆is an evolutionary stable strategy(ESS),if for∀x̸=x⋆,∃0<ε<1 satisfying Theorem 1There exist an ESS for the IoT users access evolutionary game.For∀x(0)∈(0,1)there existsT0>0,∀t>T0satisfiesx(t)=0. ProofIn order to prove the stability of the evolutionary game in fixed pointx∗,all eigenvalues of the Jacobi matrix corresponding to the replicator dynamics should have a negative real part[36].Thus we can covert the proof of existence of ESS into proving the non-negativity of all eigenvalues of corresponding Jacobi matrix. This can be explained from the physical aspect that more IoT users choosing to access the terrestrial network will aggravate network congestion which will reduce the payoff accessing cellular mode.On the contrary,fewer users choose to access the terrestrial network will lessen network congestion which will increase the payoff of accessing cellular mode. Complexity Analysis:On the complexity analysis of mode selection algorithm,we need to analyze the complexity of the three phases in Algorithm 1.The complexity of the three phases isO(|U|),O(|S||U|)andO(|U|)respectively.Thus the total complexity of Algorithm 1 isO(L|S||U|),whereLis the number of iterations. Algorithm 1 Cybertwin-assisted distributed mode selection algorithm Input:Initial population state(xc(0),xs(0)).Learning rate η.Discrete time 0,∆t,···,∆nt,···,T.Output:Equilibrium population state(xc,xs).1:Phase I:randomly initialization 2:With the initial population state(xc(0),xs(0)),each cybertwin associated with IoT user selects an access mode i∈S with probability xi(0).3:for discrete time t=0,∆t,···,∆nt,···,T do 4: Phase II:average payoff calculation 5: for cybertwin k∈U do 6: for strategy i∈S do 7: Cybertwin k calculates its payoff ui(t)according to its access mode i based on(27)and sends its payoff information to the control center.8: When the control center received all payoff information from cybertwins,the average payoff u(t)is calculated based on(27)broadcast to all users.9: end for 10: end for 11: Phase III:Strategy adaptation 12: for k=1:N do 13: if the payoff of cybertwin k is less than average then 14: According to replicator dynamics in(30),the IoT user k would select a different access mode at random with a fixed probability.15: end if 16: end for 17:end for In this section,the dynamic pricing problem for the operator is set as an optimization model for leader control.The objective function and constraints of the dynamic pricing problem are formulated according to the average rate and the evolutionary equilibrium conditions deduced in previous sections.Objective optimization alone with Lagrangian method is used to solve the dynamic pricing problem. A.Optimization Model for Dynamic Pricing The payoff function of the operator is set as whereNis the number of users in the access,xcandxsare the cellular mode population and the satellite mode population of IoT users respectively,pcandpsare the prices of the operator charging users for accessing cellular mode and satellite mode respectively.In this part,an optimization model is established for the operator’s dynamic pricing with the aim of reaching the highest average rate of ULISTN.In this case,the average network throughput is used as the operator’s payoff function,and the optimization model is shown as follows: The objective function illustrates the operator’s objective,which is to maximize the payoff.The condition that the population arrives at the ESS of the evolutionary game is depicted in(37a).The pricing range in cellular mode and satellite mode is constrained in(37b)and(37c)respectively.(37d)and(37e)are the backhaul constraints on the transmission rate of TSCs and LSCs,whileCis the maximum backhaul capacity of each TSBS.(37f)is the characteristics of the total proportion for IoT users in the evolutionary game. B.Linear Programming for Optimization Model The constraint(37a)in(37)is an implicit function of the optimization variables.This implies that(37)is a non-convex optimization problem that is not straightforward to solve.However,it is possible to convert it into an equivalent convex formulation through traversal search algorithms.Meanwhile(37d)and(37e)set constraints on the proportion of users accessing cellular mode and satellite mode.The optimization problem in(37)is equivalent to the following optimization problem where(38e)can be derived from constraints(37d)and(37e).We fix the values ofxcandxsand calculate the optimal pricing scheme(pc,ps)for various(xc,xs)pairs using traversal search algorithms.The non-convex function optimization in(37)can then be transformed into linear optimization.As a result,by fixing the value ofxc,we can divide the optimal model(38)into a number of subproblems as shown below. The sub-problem described in(39)is linear optimization whose optimal solution has closed-form by adopting Lagrangian multiplier method and Karush-Kuhn-Tucker(KKT)conditions.And the Lagrangian function can be derived as whereλis the Lagrangian multiplier associated with the global equality constraint,andµ1,µ2,µ3,µ4are associated with the global inequality constraints. The KKT conditions are sufficient to guarantee optimality when the objective function is convex and the constraints are affine.If the constraints are strictly satisfied at the optimal,following formulas will be satisfied according to KKT conditions which are necessary for optimal We define the feasible region of the linear optimization asSwhich satisfies the constraints in(39).In the feasible regionS,the optimal solution of the optimization model can be calculated in the below. whereF1=(xc+xsβc/βs)Pc,max+(αsrs−αcrc)xs/βsandF2=(xs+xcβs/βc)Ps,max+(αsrs−αcrc)xc/βc. For fixedxc,the optimal pricing strategy(pc,ps)is deduced in(42)based on linear programming and KKT conditions.The optimal pricing strategy(p∗c,p∗s)can be found by traversing the value ofxcwith a certain step length∆xin the feasible domain[x0,x1]ofxc. The detailed process of cybertwin-assisted JMSDP scheme is shown in Algorithm 2.The proposed cybertwin-assisted JMSDP algorithm is implemented by the operator deployed at the ground station.The operator that acts as the leader decides the access price for both cellular mode and satellite mode at the beginning of each time according to Algorithm 2. C.Convergence and Complexity Analysis Theorem 2For traversal search scheme denoted in Phase II of Algorithm 2,if the search step∆xis small enough,the approximate solution(pc,ps)obtained in the finite step calculation can be enough close to the optimal solution. ProofLet(P∗c,P∗s)denote the optimal price strategy andP∗c,P∗sare prices for cellular mode and satellite mode respectively.Then the difference between the approximate solutionpcand the optimal solutionP∗csatisfies the following inequality as Algorithm 2 Cybertwin-assisted joint mode selection and dynamic pricing Input:Sets of potential IoT users,accessible LEO satellites.Output:Optimal pricing scheme(pc,ps).1:Phase I:Information Collection 2:O-CT collects information of average transmission rates from LEO-CTs,TSBS-CTs and TST-CTs under the IoT user mode selection scheme obtained by Algorithm 1.3:Phase II:Optimization of Dynamic Pricing 4:for xc=x0:∆x:x1 do 5: xs=1−xc.6: F1=(xc+xsβc/βs)Pc,max+(αsrs−αcrc)xs/βs.7: F2=(xs+xcβs/βc)Ps,max+(αsrs−αcrc)xc/βc.8: if F1>F2 then 9: p∗c=Pc,max,p∗s=(αsrs−αcrc+βcPc,max)/βs.10: Calculate the payoff Utotal according to(36).11: else 12: p∗c=(αcrc−αsrs+βsPs,max)/βc,p∗s=Ps,max.13: Calculate the payoff Utotal according to(36).14: end if 15: if Utotal>Umax then 16: The optimal pricing scheme is(pc,ps)=(p∗c,p∗s)and the corresponding payoff for the operator is Umax=Utotal.17: end if 18:end for 19:Phase III:Information Broadcast 20:O-CT broadcats optimal pricing(p∗c,p∗s)for cellular mode and satellite mode to all IoT-CTs. where∆xis the traversal step.L(x∗)is a finite constant relevant tox∗and can be deduced as According to Lipschitz condition[37],pcis a uniform continuous function of independent variablexcand has no sudden change.Thus,the optimal value can be approached with arbitrary errorεand the traversal step should satisfy the following inequalitywhereεis the maximum error between optimal value and approximate optimal value andL(x∗)is a finite constant.And this algorithm will approximate the optimal value with the algorithm complexity of In this section,we evaluate the performance of our proposed JMSDP scheme and the main simulation parameters are set up as shown in Tab.2.To show the performance superiority of the proposed JMSDP scheme,we compare it with the following distributed and centralized algorithms: Tab.2 Simulation parameters •Random access(RA):massive IoT users independently select access mode at random. •Max rate access(MRA):massive IoT users select the access mode with the highest rate through greedy strategy in distributed way. •Swap matching algorithm(SMA):massive IoT users are matched with TSCs or LSCs to maximize the utility through Gale-Shapely swap matching algorithm[9]. A.Performance of User Access Selection Fig.4 shows the outage probability of BSs.The outage probability increases with the increase of signal-to-noise ratio.The greater the density of IoT devices,the smaller the outage probability.As the density of IoT users increases,more IoT users access to BSs and the co-frequency interference between IoT users increases,resulting in the decrease of transmit outage probability. Fig.5 shows the relationship between SINR outage probability and SINR outage threshold under different satellite numbers.The outage probability increases with the increase of SINR.The outage probability decreases with the increase of the number of satellites,and the number of LEO satellites ensures the stability of communication as an increase in the number of LEO satellites could provide more communication options for TSTs.In terms of outage probability shown in Fig.4 and Fig.5,LEO satellites perform worse than terrestrial networks.This can be explained that the distance between the LEO satellites and TSTs is much larger than that between BSs and IoT users leading to orders-of-magnitude higher path loss. Fig.4 Relationship between transmission outage probability and SINR outage threshold under different IoT users density Fig.5 Relationship between transmissions outage probability and SINR outage threshold under different satellite number In the case of different parameter settings of the evolutionary game algorithm,the average payoff function of users’access network changes with the number of iterations.IoT users in the simulation have a 0.3 or 0.5 initial access proportion,accordingly.Fig.6 illustrates the evolution of the percentage of IoT users opting to access LSCs with the number of iterative steps and the same learning rates under various initial access proportions.As can be seen,the evolutionary game algorithm in this case reaches a stable state after 30 iterations.When the initial state deviates from the steady state,the algorithm needs more iterations to converge to the steady state. Fig.6 Proportion of users selecting cellular mode Fig.7 shows the evolution trend of the replicator dynamic equation in the evolutionary game.Each point in this direction field represents an evolutionary population state with the angle of each arrow indicating the mutation rate of population in the evolutionary game.Specifically,this figure demonstrates how the proportion of users that choose to access terrestrial cellular networks is changing with the increase of iteration steps.The closer to the evolutionary equilibrium,the smoother the changing trend.As the proportion of cellular mode IoT users becomes closer to the ESS,the difference between the payoff of each IoT user and the average payoff becomes smaller,resulting in a slower change of the population according to the replicator dynamics.Furthermore,the robustness of the evolutionary equilibrium and the convergence of the replicator dynamics in the evolutionary game are demonstrated. Fig.7 Direction field of the replicator dynamics B.Dynamic Pricing Fig.8 demonstrates how the number of IoT users choosing to access the ground network affects the operator’s payoff at various user densities.The utility function of the operator indicating the average throughput of the ULISTN grows with the IoT user density.By changing the pricing strategy,the network is shown to have fewer IoT users accessing the terrestrial network and more IoT users accessing the satellite network when the density of IoT users is higher.Furthermore,it can be seen that when the density of IoT users is higher,the average rate of IoT users decreases due to the increase of co-frequency interference caused by the increase of IoT users. Fig.8 Total utility with the access mode selection algorithm Fig.9 shows the average transmission rate of users with different user densities(low data traffic,medium data traffic and high data traffic)with RA,MRA,JMSDP,and SMA.The average transmission rate of users decreases with the increase of user density which leads to the increase of co-frequency interference between IoT users and the decrease of the average allocated bandwidth of each IoT users.Compared with RA and MRA,the average transmission rate of massive IoT users in terms of JMSDP is increased by more than 47.33% and 167.74%.Furthermore,for dense IoT scenarios,the improvement of network performance by JMSDP algorithm becomes greater and closer to the result of centralized algorithm SMA with the increase of user density. Fig.9 Average transmission rates of users in different modes Fig.10 shows the computation time of IoT users under different algorithms.The computation time of all algorithms increases with the increase of IoT density.The computational time of our cybertwin-assisted JMSDP scheme has a linear relationship with the number of IoT users,which is consistent with the analysis of algorithm complexity.Meanwhile,the computation time of SMA is a quadratic function of IoT users density,which is not suitable for massive access in ULISTN considering the delay-sensitiveness of ULISTN due to the high mobility of LEOs.Our proposed scheme works much better than SMA and has the same order of magnitude as RA and MRA.Fig.9 and Fig.10 show that the proposed JMSDP can achieve nearly optimal average rate with much less computation delay compared with SMA. Fig.10 Computation delay of users in different modes Fig.11 shows the average delay of IoT users with RA,MRA,JMSDP,and SMA.Queue delay increases with the increase of data amount generated by each IoT user.Our proposed scheme performs the best among these algorithms especially better than MRA especially when the traffic load of the ULISTN is heavy or the density of IoT users is high.Our proposed scheme can improve the performance of queue delay by 117.61%,26.70%,and 25.02% on average than RA,MRA and SWA respectively.This shows that our proposed JMSDP can achieve better network load balancing indicating better resource orchestration than others. Fig.11 Computation time of different algorithms In this paper,we have designed a cybertwin-assisted joint mode selection and dynamic pricing scheme for massive access to IoT in ULISTN.Specifically,cybertwin serves as the intelligent agent for information exchange and guarantees self-interests of IoT users and the operator.We have proposed an evolutionary game for mode selection where IoT users select access mode based on replicator dynamics to achieve a higher payoff.We have set up a dynamic pricing model for the operator to obtain maximum average throughput and solved by optimization theory.The simulation results have shown the proposed JMSDP scheme performs better in average throughput and delay than random access and max rate access.In future work,we will introduce machine learningbased techniques into the mode selection of ULISTN and extend our work considering the mobility pattern of LEOs.Furthermore,we will investigate the power allocation and cybertwin-assisted LEO access selection scheme of TSTs in satellite communication mode.
