Beam-Aware Cross-Layer DRX Design for 5G Millimeter Wave Communication System

In the fifth-generation (5G) wireless communication system, the Discontinuous Reception (DRX) is indispensable to user equipment (UE) to support evolved technologies. However, the directionality of wireless links, especially in millimeter wave bands, leads the existing DRX mechanism to unnecessary power consumption. In this paper, we first identify the problems of beam pattern mismatching. Then, we proposed a dynamic beam-aware DRX mechanism, which could be applied along with dynamic mmWave beam configuration. The proposed framework jointly optimizes beam-aware DRX operation and dynamic mmWave beam configuration for better energy efficiency. A semi-Markov model is proposed to evaluate a UE sleeping ratio and packet delay. The simulation validates the proposed model. Compared to the baseline 5G NR DRX operation, the proposed scheme achieves 41.6% improvement in the sleep ratio without sacrificing the packet delivery latency performance.


I. INTRODUCTION
In the near future, the number of connected user equipment (UE) will increase massively. It brings exponentially increasing global data traffic and exposes considerable power consumption to society. The fifth-generation (5G) New Radio (NR) cellular networks are expected to meet the demands of explosive data, such as 1000x system capacity, 100x energy efficiency, and 10x lower delay [1]. Recently, the network operators are running out of wireless spectrum, so they would like to take advantage of the additional unused resources of millimeter wave (mmWave) frequency bands for data transmission [2]. Therefore, the development in the mmWave technology is essential for the evolution of communication [3], [4]. However, the tremendous propagation loss and sensitivity to blockage in the mmWave band shorten the transmission distance [5]. The deteriorative received power causes low signal-to-interference-plus-noise ratio (SINR) and insufficient throughput.
To increase the received power, a prospective solution is beamforming technology [6]. With beamforming, NR Node The associate editor coordinating the review of this manuscript and approving it for publication was Zhenhui Yuan . B (gNB) could concentrate the transmission power in a particular direction to serve the covered UEs at one time. Connections between gNBs and UEs are established with the beam management procedures [7]. Not only does the beamforming technology enhance the received power of the covered UEs, but it also reduces the impact of interference between the different gNBs [8]. However, the massive antenna array used in mmWave beamforming technology consumes higher power than the antenna of the traditional devices and therefore reduces the battery life on UE side [9].
Discontinuous Reception (DRX) is a crucial technique for power saving. With DRX, a UE can turn off its radiofrequency (RF) module properly, and decrease power consumption without yielding unacceptable delay. It is expected that UEs in 5G networks will continue using DRX mechanism to save power [10]. Nevertheless, employing DRX in mmWave bands without considering the directionality of mmWave link will lead to sub-optimal performance. Therefore, the beam-aware Media Access Control (MAC) Layer protocol design is essential to reduce the energy consumption of UEs in the mmWave 5G wireless networks [11]- [13]. The gNB could only serve part of the UEs at the same time. Thus, the gNB needs to serve the UEs on different beams sequentially, and the order of the beams could be regarded as the serving beam pattern. If the UEs could operate their RF modules with the consideration of serving beam patterns, the amount of additional reduced power consumption is considerable, especially in the 5G NR networks with a massive number of mobile devices.
In this paper, we propose a cross-layer design that enables the DRX mechanism adaptive to the beam patterns in mmWave communications. Since the order of serving beams influences the performance of the DRX mechanism, the performance could be further improved with the assistance of the suitable serving beam patterns. The main contributions of this work are summarized as follows: • With the awareness of beam patterns, we design a DRX mechanism that is adaptive to the mmWave beamforming technology. Most research on 5G NR DRX focuses on integrating the beam searching process into the DRX mechanism, or improving the DRX mechanism under different scenarios in 5G networks [14]- [20]. To our best knowledge, our DRX mechanism is the first joint design of the DRX mechanism and the discrete serving beam pattern. ( §V) • We propose the models of the static and dynamic serving beam patterns in the future 5G networks. Considering the various requirements and scenarios, we describe how the gNB configures DRX mechanism with these beam patterns, and we show the tradeoffs between these two options. ( §IV) • We analyze the performance of the proposed crosslayer DRX mechanism via a semi-Markov model. The expected values of the main performance metrics, i.e., sleep ratio and packet delay, match the simulation results. ( §VI and §VII) • We explore the impact of the serving beam pattern on the proposed cross-layer DRX. The gNB can select beams to schedule data flexibly in a specific period. The flexible scheduling period can enhance both the sleep ratio and the packet delay of the proposed DRX mechanism.

II. BACKGROUND AND RELATED WORK A. DRX IN 5G NR NETWORKS
The DRX mechanism in 5G NR follows that in Long Term Evolution (LTE) and can support different numerologies with the timers specified in milliseconds [10], [21]. As the example in Fig. 1, a DRX cycle contains one ON Duration and one Opportunity for DRX, in which a UE turns its RF module on and off, respectively. In ON Duration, the UE is required to monitor Physical Downlink Control Channel (PDCCH) continuously. Any indicators for packets make the UE reset their drx-Inactivity Timer and extend the wake-up period. If drx-Inactivity Timer expires, the UE will turn off its RF module and enters Opportunity for DRX, in which it is allowed to turn off its radio to reduce the power consumption but at the expense of ceasing detecting the indication of data at

B. BEAMFORMING TECHNOLOGY AND THE CHALLENGES
For the lack of bandwidth resources, mmWave communication is considered as a potential technology. On the one hand, the mmWave communication relieves the spectrum shortage as well as satisfies the requirement for a higher data rate. On the other hand, the tremendous propagation loss of mmWave weakens the received power. The gNB concentrates the transmission power in specific directions to enhance the received SINR in the 5G NR communication networks. Via beamforming technology, the received power level increases. This technology could benefit not only the communication in lower frequency bands but also the one in mmWave bands.
Owing to the spatial locations of the UEs, they are served with different beams (as shown in Fig. 2). The gNB configures the DRX parameters of every UE and knows when the UEs are active. When there are buffered packets and the destination UE is active, the gNB can schedule data on the correct beam. The performance of the NR DRX under 5G NR networks is almost the same as that in the LTE networks, despite there is room for improvement in energy efficiency. Fig. 3 illustrates the unnecessary UE power consumption in 5G networks. The gNB could only select limited beams to schedule data at one time. In this case, we assume that the gNB uses one beam for transmission at one time. In every time-slot, the gNB determines the beam to be used for downlink transmission, but the UEs do not have the information of the selected beam. It is power-consuming for the UE on Beam-3 to turn on the radio and extend its ON Duration while the gNB is serving other UEs on another beam. The . An illustrative example of the unnecessary power consumption problem in 5G NR DRX operation. In the first ON Duration of the UE on Beam-3, the UE turns on its RF module while the gNB is using another beam to perform downlink transmission. The second ON Duration is prolonged after the packet reception, but the gNB is not serving the UEs on Beam-3 after time-slot t 5 . In both cases, although there are buffered packets for the UE, the gNB is not able to perform the downlink transmission for the UE. energy inefficiency should not be omitted, particularly for the scenarios with a massive number of UEs. Therefore, we aim to propose a new DRX design to decrease the power consumption without a larger packet delay for 5G NR networks.

C. RELATED WORK
Researchers have investigated power saving mechanisms in 5G mmWave communications. Lauridsen et al. combined microsleep, DRX, Discontinuous Transmission (DTX), and an aperiodic sleep mode with a wake-up signal receiver. In the short and pipelined 5G frame structure, they showed that the battery life of the 5G mobile devices could be improved from 20% to 90% compared to LTE [22]. There is also research focusing on the impact of the beamforming technology on the DRX mechanism. In [11], Beam-Aware DRX was proposed with the consideration of periodic serving beam cycles, and the performance was evaluated via semi-Markov Chain analysis. Considering the device-to-device (D2D) unmanned aerial vehicles (UAV) mmWave communication framework, Zhang et al. proposed a fast beam tracking DRX mechanism to overcome the easily-occurred beam misalignment due to the high mobility of UAVs [20]. The stateof-the-art approach is Directional DRX (DDRX) proposed by Maheshwari et al. [16]. To maintain the best beam pairs, a four-state DRX mechanism was proposed to be integrated with the process of beam searching. In [15], [16], the UE with Standalone DDRX (S-DDRX) operates beam searching after every DRX cycle.
For the DDRX mechanism, there are extensions under different scenarios to improve Quality of Service (QoS) or power efficiency. With the consideration of dual connectivity (DC) of the UE, the authors proposed Integrated DDRX (I-DDRX) in [14], [16]. The UEs can detect the indication of data via LTE link, so they only need to search for the best beam pair when packets arrive. To avoid the overhead for maintaining DC and unnecessary beam searching procedure, Liu et al. proposed a DRX mechanism for multiple beams, with which the UEs only operate beam training when they monitor poor channel quality instead of the indicator of downlink data [19]. In the Standalone D2D communication networks, two UEs with Cooperative DDRX take turns searching for the best beam pair and inform each other [16]. In the 5G networks, various service requirements are attainable with the flexible transmission time interval (TTI). With the consideration of delay constraints, Maheshwari et al. proposed an algorithm to find a suitable TTI length in S-DDRX [17]. To reduce more power consumption in the light traffic with the short bursty data, Philip et al. improved the design of I-DDRX by adding an auxiliary active state after each DRX cycle when packet arrival number is small [18].
Compared to the previous research, we focus on crosslayer design and optimization between MAC layer DRX and Physical layer mmWave beam pattern configuration. In MAC layer, DRX is aware of mmWave beam patterns so that the system could operate in an energy-efficient way. In Physical layer, a dynamic beam pattern configuration framework is proposed, and cross-layer optimization is conducted to handle the heterogeneous device and traffic distribution. To the authors' best knowledge, this research work is the first crosslayer framework that jointly optimizes beam-aware DRX operation and dynamic mmWave beam configuration.

III. PROBLEM DESCRIPTION
As shown in Fig. 3, the 5G NR DRX leads to inefficiency in power consumption due to the lack of information about beam directionality. The UE on Beam-3 turns on its RF module in ON Duration. The packet arrival makes the UE reset its drx-Inactivity Timer, and the wake-up time of the UE is extended. The gNB uses Beam-3 to serve the UE only in some specific periods. That is to say, there is no need for the UE to monitor PDCCH continuously. In this work, we proposed a 5G NR DRX framework to solve such a problem of beam behavior asynchrony. The two main proposed designs on gNB side and UE side are shown as follows: • We will define the two main strategies of the beam serving pattern on gNB side. In the proposed serving beam pattern, the difference between two strategies and what the gNB and the UE behave are introduced in §IV.
• We will demonstrate the proposed 5G NR DRX design, which is adaptive to the serving beam pattern in §V, and show how to use a semi-Markov Chain to model the DRX operation in §VI.

B. THE PROPOSED SERVING BEAM PATTERN
We would like to design a DRX mechanism in the 5G NR networks to improve the power efficiency for the UEs with prior knowledge of the best beam pairs. As mentioned above, the Static-Configured Beam Frame and Dynamic-Configured Beam Frame are the two main strategies to determine the serving beam pattern for gNB. It is essential to consider the massive effect of serving beam pattern on the DRX operation when we evaluate the performance. In this paper, we propose the 5G NR beam pattern demonstrated in Fig. 4. There are one Static-Configured Beam Frame (T s F ) and one , and the Static-Configured Beam Frame consists of N s sbc serving beam cycles. Besides, only when the gNB may serve the UEs do they turn on radios to monitor PDCCH. Thus, the UEs could decrease unnecessary power consumption in the Static-Configured Beam Frame.
In the Dynamic-Configured Beam Frame, the gNB tends to serve the UE in urgent need of data transmission. We assume that the UEs with more history traffic have higher probabilities of requesting for the wireless resources. Therefore, the gNB decides whether to serve the UE or not based on the number of arriving packets during the interval (t s begin , t s ls ), where t s begin is the beginning of Static-Configured Beam Frame and t s ls is the time when the gNB last serves the UE in Static-Configured Beam Frame. If at least one UE is receiving no less than N pkt,th packets on the beam (i.e., the beam achieving the ''serving requirement''), the gNB will add the beam to the serving candidate list. According to the total number of packets arriving on different beams, the gNB selects which beams to schedule data from the serving candidate list in the following Dynamic-Configured Beam Frame randomly. When multiple beams achieve the ''serving requirement'' simultaneously, the gNB might not serve all the candidate beams because of the limited length of Dynamic-Configured Beam Frame. From the UE's perspective, if the received packet number of the UE is more than or equal to the threshold (N pkt,th ), it will turn on its RF module to detect the indicator of data in Dynamic-Configured Beam Frame. In this paper, we refer to the event as ''Matching'' that the gNB serves the UE in the Dynamic-Configured Beam Frame when the UE turns on its RF module in the Dynamic-Configured Beam Frame.

V. THE PROPOSED DRX MECHANISM IN mmWave COMMUNICATION
The design goal in the 5G NR DRX operation is to deliver data traffic in an energy-efficient manner. In consideration of beamforming technology, we propose a cross-layer DRX design that is adaptive to the Static-Configured and Dynamic-Configured Beam Frame in the mmWave communication networks. Fig. 5 illustrates the proposed cross-layer DRX operation. It consists of four states, including Active, Inactive, DRX check , and DRX sleep , with the consideration of the beam pattern. Active and Inactive correspond to the ON Duration and the drx-Inactivity Timer in the NR DRX respectively. When there are packet arrivals, the UE stays in Active or Inactive to detect the indicator of downlink data and respond immediately. The UE with the proposed DRX mechanism turns on its RF module according to the serving beam patterns. The DRX cycle includes one DRX check and one DRX sleep . To enhance the energy efficiency, the UE wakes up for one beam period (T b ) to detect the buffered traffic in DRX check , and then go to sleep in DRX sleep . In DRX check , any packet arrival makes the UE enter Active. In the following paragraph, we will describe how to handle the energy inefficiency problem mentioned in §III.
The beam-aware DRX configuration: In the proposed DRX mechanism, the gNB could configure the DRX operation of the UEs in terms of serving beam patterns. With the detailed information of the beam pattern, the UEs turn on their radio only when the gNB is serving them. To ensure that the UEs could control their RF modules with the awareness of serving beam patterns, the proposed DRX mechanism includes the following two settings. First, for each UE, the gNB configures the duration of Active, Inactive, and DRX cycle as multiple of serving period (T F ). Second, the gNB informs the UEs of the beam pattern configuration. With VOLUME 8, 2020 FIGURE 5. The proposed cross-layer DRX operation in the Static-Configured and Dynamic-Configured Beam frame structure. On the basis of NR DRX design, the proposed DRX operation could still be characterized by ON Duration (Active), drx-Inactivity Timer (Inactive) and Opportunity for DRX (DRX sleep ). In active time, the UE with NR DRX turns on its RF module all the time. However, in Active and Inactive, the UE wake-up pattern follows gNB serving beam pattern. When the UE enters Opportunity for DRX, it only performs a short DRX check . If there are any incoming packets, the UE returns to ON Duration. the configuration of beam patterns, the UEs have the information of the duration in Static-Configured Beam Frame (T s F ) and Dynamic-Configured Beam Frame (T d F ). The UEs know when the gNB uses each beam to serve in the Static-Configured Beam Frame, and how to control their RF modules in the Dynamic-Configured Beam Frame. Therefore, the UEs could know when the gNB may serve them and turn on their RF modules accordingly.
The reduction of the unnecessary power consumption: In the Static-Configured Beam Frame, a UE with downlink traffic is served periodically according to the serving beam pattern. When the gNB configures the DRX operation with the awareness of serving beam patterns, the UEs could follow the serving beam patterns to control their RF modules. Thus, the UEs in Active and Inactive can go to sleep in terms of saving power when the gNB serves other beams in the Static-Configured Beam Frame. In the following sections, we will introduce the semi-Markov Chain for modeling the proposed DRX mechanism ( §VI) and analyze the mathematical model ( §VII).

VI. THE SEMI-MARKOV CHAIN DRX MODEL
We model the proposed DRX mechanism as the semi-Markov model in Fig. 6  reality, the UE enters DRX check firstly and then monitors the indicator. In this paper, we use the transition from DRX sleep to Active to model the event of the UE receiving the packets buffered in the DRX cycles. If there are no packets sent to the UE, then it will enter DRX check .

VII. MATHEMATICAL FORMULATION
We consider a gNB with N beams and several UEs randomly distributed in the coverage of the gNB. For i ∈ {1, 2, · · · , N }, there are M i UEs on Beam-i. For j ∈ {1, 2, · · · , M i }, each packet arrival of UE j on Beam-i follows the Poisson distribution with parameter λ ij , and the random variable of packet inter-arrival time is t ij a . Considering the extremely high data rate in the mmWave band, we assume that the buffered packets could be transmitted within one beam period. In §IV, the gNB chooses some beams to serve the UEs during the Dynamic-Configured Beam Frame based on the number of packets arriving in the last Static-Configured Beam Frame. The total number of packets sent to all UEs on Beam-i during the interval (t s begin , t s ls ) in Fig. 7 can be expressed as n i pkt = M i j=1 n ij pkt , where n ij pkt is the random variable of the number of packets sent to UE j on Beam-i in the interval (t s begin , t s ls ). Also, when there is at least one UE on Beam-i with the packet number no less than threshold (N pkt,th ), we express the probability as We define the packet number matrix without the packet number of Beam-i as = diag( P 1 P 2 · · · P i−1 P i+1 · · · P N ).
In addition to Beam-i, if there are another m beams achieving the serving requirements (i.e., existing at least one UE receiving no less than N pkt,th packets) where m ∈ {0, 1, · · · , N − 1}, the gNB will select beams to serve in the Dynamic-Configured Beam Frame randomly. The selecting probability is directly proportional to the packet number of each beam. For example, if the threshold is two, the total packet number on Beam-1 is three, and the one on Beam-2 is two. Both of them achieve the serving requirement, and the selecting probabilities are 0.6 and 0.4, respectively. When there are additional m beams satisfying the serving requirements, we express the selecting probability matrix of Beam-i as T , where J x,y is a x-by-y matrix with all its entries equal to one. 3) Let A(m, n) = a 1 a 2 · · · a C m n . For x, y ∈ {1, 2, · · · , C m n }, a x = a y if and only if x = y. Besides, we also define two functions to present the derivation in the following. Let P ∈ R n×n , A ∈ R n×m , a ∈ R m×1 , and m, n ∈ R, one function is f (P, A) = J n,m − A + P(2A − J n,m ), and the other is a im which is used to multiple every entry of each column in a n-by-m matrix, and generate a 1-by-m matrix. We define the vector power operation as a n = The probability of Matching occurring in the Dynamic-Configured Beam Frame (i.e., the gNB serves the UE, and the UE turns on its RF module at the same time) is related to the other beams with at least one UE receiving no less than N pkt,th packets. Given the condition that Beam-i achieves the serving requirement, when there are another m beams also achieve the serving requirements, the number of permutations is C N −1 m .

A. TRANSITION PROBABILITY IN THE SEMI-MARKOV CHAIN MODEL
The mean sojourn time in each state is a random variable instead of a fixed value, so we employ the semi-Markov model in Fig. 6 to analyze the proposed cross-layer DRX mechanism. To solve the semi-Markov Chain and analyze the performance of the proposed DRX, we have to calculate the transition probabilities firstly. For UE j on Beam-i, the transition probability from state Sm to state Sn is denoted as P ij m,n , where m, n ∈ S and S = {1, 3, 4} ∪ {2 1 , 2 2 , · · · , 2 N ic }. We will calculate the transition probabilities from the different states as follows: 1) Transition probability from Active: At the end of Active, the UE stays in the same state when there are buffered packets; otherwise, the UE enters Inactive. The transition probability from Active to Inactive can be determined with The Matching in the Dynamic-Configured Beam Frame impacts on the length of interval (t ls , t end ) and P ij 1,2 1 . The expected transition probability from Active to Inactive is where P ij (nm) is the no Matching probability, P ij (m) is the Matching probability, P ij 1,2 1 (nm) and P ij 1,2 1 (m) are the transition probabilities with and without Matching. We will calculate these probabilities in §VII-B. 2) Transition probability from Inactive: If there are no packets sent to the UE in Inactive, it will remain in Inactive or enter DRX check at the next state. Any packet arrival makes the UE enter Active at the end of the state. The length of Inactive is T F . For m ∈ {1, 2, · · · , N ic − 1},

B. THE IMPACT OF MATCHING ON TRANSITION PROBABILITY
In §VI, the state transition at the end of Active is based on the existence of buffered packets. In Active, every packet before the time t ls in Fig. 5 can be transmitted to the UEs successfully. That is, the packets arriving during the interval (t ls , t end ) will be buffered, so the transition probability from Active to Inactive can be expressed as (1). The time t end is fixed, while whether the Matching occurring in the Dynamic-Configured Beam Frame (i.e., the gNB serves the UE, and the UE turns on its RF module simultaneously) will vary the time t ls . In the following, we will derive P ij 1,2 1 (2a) by considering Matching and no Matching in Active individually.

1) No Matching in Active:
If there is no Matching, then UE j on Beam-i would not receive any packet during the Dynamic-Configured Beam Frame. All packets are buffered, and the time t ls is equal to t s ls . The length of the interval (t ls , t end ) = (t s ls , t end ) = T d F + T b (N − 1) in this case. Since the packets follow the Poisson distribution, the transition probability from Active to Inactive is There are two sufficient conditions that all packet buffered in the Dynamic-Configured Beam Frame. One is UE j on Beam-i turns off its RF module when the packet number is less than N pkt,th . The probability of this event is Pr{n ij pkt < N pkt,th }. The other is that the gNB does not use Beam-i to serve the UEs in the Dynamic-Configured Beam Frame. Although the number of the packet arrivals for UE j on Beam-i in the interval (t s begin , t s ls ) is more than or equal to N pkt,th , it is still possible that the gNB skips Beam-i and transmits on other beams which also meet the serving requirement in Dynamic-Configured Beam Frame. When the number of the packets transmitted to UE j on Beam-i is no less than N pkt,th , the probability of the gNB only serving the UEs on other beams is Pr{n 2) Matching in Active: If there are Matchings in the Dynamic-Configured Beam Frame, the time t ls is the end of the last Matching. We assume that the gNB uses Beam-i to serve UE j at the (N d bp − n) th beam period in the Dynamic-Configured Beam Frame, and the gNB does not use Beam-i to transmit anymore. The length of interval (t ls , t end ) = (n + i − 1)T b which determines the transition probability from Active to Inactive according to (1). In this case, when the interval (t ls , t d end ) is nT b . If the Matching occurs during the Dynamic-Configured Beam Frame, two conditions are satisfied. One is UE j on Beami turning on its RF module, and the other is the gNB using Beam-i to transmit data in the Dynamic-Configured Beam Frame. Only when the number of received packet in the interval (t s begin , t s ls ) exceeds or equals N pkt,th (i.e., achieve the serving requirement) does UE j on Beam-i turn on its RF module, and the probability is Pr{n ij pkt ≥ N pkt,th }. If the first condition is satisfied, and another m beams also achieve the serving requirement, then the probability that the gNB uses Beam-i to serve the UEs at the (N d bp − n) th beam period is p

C. STEADY STATE PROBABILITY
We could derive the solutions of the semi-Markov Chain model, i.e., the steady state probabilities ij m ∀m ∈ S for UE j on Beam-i, with the transition probabilities. ij m is the stationary distribution over state Sm for UE j on Beam-i in the long run, and we will calculate it in the following Lemma 1.

Lemma 1: The steady state probabilities of UE j on Beam-i are the solutions of the following system of equations
Thus, the steady state probability of Active can be expressed as Fig. 6 shows that the steady state probabilities of other states are ij With (13d), the steady state probability of DRX check (13c) can be expressed as The last equation is the constraint that the summation of all steady state probabilities is equal to one.

D. SLEEP RATIO
We can use sleep ratio (R Proof: No matter which state the UEs stay currently, they will enter Active or other states at the end of their current states. Fig. 5 shows that the holding times in Active and Inactive are T F , and the length of DRX cycle is T F · N dc . The UEs will turn on their RF module for one beam period at the beginning of each DRX cycle to check whether there is any buffered packet or not, so E{T h, 3

Lemma 3: The expected sleeping time of UE j on Beam-i in Active and Inactive is
Proof: Active and Inactive consist of one Static-Configured and one Dynamic-Configured Beam Frame. In the Static-Configured Beam Frame, the UEs turn on their VOLUME 8, 2020 RF module only when the gNB serves them. The gNB uses Beam-i to serve UE j for one beam period per one time and N s sbc times in one Static-Configured Beam Frame, so the sleeping time in Static-Configured Beam Frame is T s F −N s sbc · T b . The UEs will turn off their RF modules in Dynamic-Configured Beam Frame if the number of received packets is less than N pkt,th in the interval (t s begin , t s ls ). The average sleeping time of UE j on Beam-i in Dynamic-Configured Beam Frame is Pr{n ij pkt < N pkt,th } · T d F .

Theorem 1: The expected sleep ratio of UE j on Beam-i is
Proof: The sleep ratio is the proportion of the time that the UE turns off its RF module. From (16), we can get the sleeping time of UE j on Beam-i in Active and Inactive is T ij off . The UEs turn on their RF modules in DRX check and turn off their RF module in DRX sleep . Considering the sleeping time in every state and the steady state probability from (10), we can express the expected sleep ratio as (17).

E. PACKET DELAY
If a downlink packet arrives during the sleeping time of the UE, it will be buffered, and the delay (D ij pkt ) increases. D ij pkt is the period from the time when the gNB receives a packet to the time when it uses Beam-i to transmit the packet to UE j. In the following, we will calculate the expected delay in different states, and then derive E{D ij pkt } with the expected delay and steady state probability.
Lemma 4: The average delay for a packet arriving at the UEs in DRX sleep is Proof: According to [23], if the UEs receive all buffered packets at the time t served , the average delay for a packet in the interval (t begin , t end ) in Fig. 8(a) will be (19) where t a is the packet arriving time. In DRX sleep , t begin = T b and t end = t served = N dc · T F . The average delay in DRX cycles is (18).
To get E{D ij pkt }, we need to derive the delay in Active, Inactive, and DRX sleep . The delay in Active equals the one in Inactive, and would be influenced by Matchings (i.e., the gNB serves the UE, and it turns on its radio at the same time).  With the consideration of Matching and the packet arriving period, the expected delay in Active or Inactive is expressed as In the following, we will derive the delay based on the cases in Fig. 8. 1) d (s) : Fig. 8(b) shows the packets for UE j on Beam-i arriving in the serving beam cycles are buffered when the gNB transmit on other beams. There are N s sbc − 1 serving beam cycles in the interval (t begin , t s ls ), and the delay in each serving beam cycle is The delay in the interval (t begin , t s ls ) is 2) d ij (nm) : For the packet arrivals in the interval (t s ls , t d end ), we have to consider the Matchings in the Dynamic-Configured Beam Frame when we calculate the delay. If there is no Matching in the Dynamic-Configured Beam Frame, t s ls = t ls and all packets arriving in the interval (t s ls , t d end ) will be buffered until the time t end . There is no Matching occurring only when the received packet number in the interval (t s begin , t s ls ) is less than N pkt,th , or the gNB transmits on other beams in the Dynamic-Configured Beam Frame. With the consideration of the probability of no Matching in the Dynamic-Configured Beam Frame, the delay in Fig. 8 3) d ij (m) : In the Static-Configured Beam Frame, if the packets arrive after the time t s ls , they will be buffered. Fig. 8(d) shows that there are Matchings occurring, so UE j on Beam-i receives the buffered packets in the following Dynamic-Configured Beam Frame. In this case, the number of received packets in the interval (t s begin , t s ls ) of UE j on Beam-i is no less than N pkt,th , and the gNB uses Beam-i to serve UE j in the n th beam period of the Dynamic-Configured Beam Frame. When both Beam-i and another m beams achieve the serving requirements, the probability that the gNB uses Beam-i to serve UE j at the n th beam period is Pr{n 1) . Thus, the delay in Fig. 8 where there are both UE j on Beam-i and another m beams satisfy the serving requirement. The delay in this case is Fig. 8(f), all Matchings happen before the n th beam period, so the packets for UE j on Beam-i in the n th beam period in the Dynamic-Configured Beam Frame are buffered until the time t end . Since the gNB only uses other beams to transmit, so there must be another m = 0 beams achieve the serving requirement. The delay in Fig. 8(f) is expressed as The packets arriving in the interval (t d end , t end ) are buffered until the end of the state, so the delay in Fig. 8(g) is derived as Proof: When the UE turns off its RF module, the arriving packets are buffered, and the delay increases. The packet delay consists of the delay in each state of the proposed DRX mechanism. The overall average packet delay is the weighted summation of the average delay in each state, and the weights are the probabilities of a packet arriving in the different states. Fig. 6 shows that the packet delay only occurs in Active, Inactive, and DRX sleep . The probability of a packet arriving in state Sm is We can get the packet delay in Active, Inactive, and DRX sleep via Lemma 4 and (20). Thus, the expected delay for one packet is (28).

VIII. NUMERICAL ANALYSIS AND SIMULATION RESULTS
In this section, we verify the analytical model with the simulation experiments conducted on MATLAB. In the downlink  scenario we consider, there is one gNB with multiple beams, and the range of beam number is from 4 beams to 22 beams. Besides, there are two UEs on each beam. For the serving beam pattern, we set a beam period (T b ) as 1 slot. We consider the fourth numerology in the 3GPP NR frame structure, so the slot length is 0.125 ms [24]. For the data transmission rate in mmWave band, we follow the assumption in § VII, and the transmission time is short enough to be ignored. While fixing the length of the serving period (T F ) as 20 ms, we change the number of the serving beam cycle in Static-Configured Beam Frame (N s sbc ) and the number of beam period in Dynamic-Configured Beam Frame (N d bp ) to evaluate the performance of the proposed DRX mechanism. The packet arrival process is Poisson distribution with λ ij = 0.01 ms −1 or λ ij = 0.03 ms −1 for each UE, and we utilize the threshold of the packet number (N pkt,th ) = 3. As for the DRX configuration, the maximal number of Inactive (N ic ) is from 1 to 5, and the number of serving period in DRX cycle (N dc ) varies from 4 to 64. Therefore, the maximal length of Inactive is from 20 ms to 120 ms, and the length of DRX cycle is from 80 ms to 1280 ms. In the following sections, the unmentioned values of parameters are set to the default parameter setting listed in Table 1. In addition, we refer the length of Static-Configured Beam Frame as ''SF'' and that of Dynamic-Configured Beam Frame as ''DF''.

A. ANALYTICAL MODEL VERIFICATION AND EFFECT OF DRX CYCLE
We change the length of DRX cycle and the DF (T d F ) to examine the effects of these parameters. The serving period (T F = T s F + T d F ) is set to 20 ms, so the SF (T s F ) will change when we use the different DF (T d F ). Then, we validate the analytical model with simulation results. In Fig. 9(a), the sleep ratio increases, as the length of DRX cycle increases. When the length of DRX check is fixed, longer DRX cycles implies a longer length of DRX sleep . Fig. 9(b) shows that the packets arriving in the longer DRX cycles are buffered, and yield more serious delay than those under the configuration with shorter DRX cycles. That is to say, the longer the DRX cycle length is, the more efficient the power usage is, and the more serious the packet delay is. The tradeoff between power efficiency and packet delivery delay exists with different DRX configurations.

B. COMPARISON WITH OTHER DRX MECHANISMS
We compare the proposed DRX design with the 5G NR DRX mechanism under the different parameter settings, including the length of DRX cycle, the maximal duration in Inactive, the traffic density, and the beam number of the gNB. In §II-A, the UEs with basic DRX operation control their RF modules based on the packet arrivals. When there are packets buffered in the gNB, and the destination UE turns on its RF module, the gNB will use the correct beam to schedule data. Without considering the beam patterns in the 5G networks, the NR DRX may cause the inefficiency of power consumption on UE side. The length of drx-Inactivity Timer in the NR DRX corresponds to the duration of one Active (T F ) and the maximal period in Inactive (N ic · T F ) in the proposed DRX mechanism. In this paper, both of them are referred to ''the length of Inactivity Timer''. In the following, we set the length of drx-Inactivity Timer in the NR DRX as N ic · T F + T F ms.

1) EFFECT OF DRX CYCLE
For the proposed DRX mechanism, we configure the number of the serving beam cycle in Static-Configured Beam Frame (N s sbc ) as 8 and the number of the beam period in Dynamic-Configured Beam Frame (N d bp ) as 96, so the serving period (T F ) is 20 ms. Since the maximal number of substates in Inactive (N ic ) is 2, the maximal length in Inactive is 40 ms. The length of drx-Inactivity Timer in the NR DRX mechanism is set to 60 ms. In Fig. 10, the active UE with NR DRX turns on its RF module all the time, so its sleep ratio is smaller than the proposed DRX mechanisms. With the consideration of serving beam pattern, the sleep ratio in our proposed cross-layer DRX mechanism has 41.6% improvement when the DRX cycle is 80 ms. The overall performance of our proposed DRX can achieve up to 99.6%. Fig. 10(b) demonstrates that the packet delay in NR DRX is almost the same as the delay in our proposed scheme. As a result, our proposed DRX mechanism can reduce power consumption without producing a larger delay. In the 5G mmWave networks, taking the directional characteristic of the beam pattern into account is essential for the DRX design; otherwise, the unnecessary power consumption mentioned in §III will reduce the efficiency.

2) EFFECT OF INACTIVITY TIMER
In addition to the length of the DRX cycle, the length of Inactivity Timer plays a critical role in the performance of the DRX mechanism. In Fig. 11, we configure the number of the serving beam cycle in Static-Configured Beam Frame (N s sbc ) as 8 and the number of the beam period in Dynamic-Configured Beam Frame (N d bp ) as 96, so the serving period (T F = T s F + T d F ) is 20 ms. The length of DRX cycle is set to 960 ms and the length of Inactivity Timer varies from 40 ms to 120 ms. Fig. 11 depicts that both the sleep ratio and the packet delay of the proposed DRX mechanism decrease when N ic , i.e., the maximal number of substates in Inactive, increases. The UEs in Active and Inactive consume more power than that in DRX check and DRX sleep , while the packet delay in Active and Inactive is shorter than that in DRX check and DRX sleep . If N ic rises, the probability of entering DRX check and DRX sleep will decline. Therefore, the UEs with higher N ic have higher probability staying in Active and Inactive, and they cause the lower sleep ratio and lower delay. Compared with the NR DRX, the proposed DRX has more then 11.1% improvement in the energy efficiency without the larger packet delay when the packet arrival rate is 0.01 ms −1 .

3) EFFECT OF TRAFFIC LOAD
The different curves in Fig. 11 present the performance of the proposed DRX mechanism and the NR DRX under different packet arrival rates, including 0.01 ms −1 and 0.03 ms −1 . More packets arrive in a fixed period under the heavier traffic load. A packet arriving in Active and Inactive makes the UEs extend their wake-up period, and one arriving in DRX cycles triggers the UEs to enter Active. The frequent packet arrivals make the steady state probabilities of Active and Inactive high. Thus, both the sleep ratio and the packet delay are lower than those under lighter traffic load in Fig. 11. With the proposed DRX, the UEs could turn off their RF module to save more power when the gNB is serving other UEs. Under the different traffic loads, the UEs with the proposed DRX mechanism reduce more power consumption than that in the NR DRX.

4) EFFECT OF BEAM NUMBER
In Fig. 12, we evaluate the performance of the proposed DRX and the NR DRX mechanism under the number of beams of the gNB from 4 to 22. The number of the serving beam cycles in Static-Configured Beam Frame (N s sbc ) and the beam period in Dynamic-Configured Beam Frame (N d bp ) are configured as 4 and 32. Since the SF is related to the beam number, the serving period (T F ) varies from 6 ms to 15 ms with different beam numbers. For the DRX configuration, we set the length of DRX cycle and the length of Inactivity Timer to 960 ms and 120 ms. Fig. 12(a) depicts the sleep ratio in the proposed DRX increases with the increasing of the beam number. Since N s sbc is fixed and the UEs sleep when the gNB uses other beams to serve in Static-Configured Beam Frame, the UEs could save more power with the larger beam number. For the NR DRX, only the DRX configuration and the packet arrival rate affect the performance, so the sleep ratio maintains the same value with different beam numbers. Compared with the NR DRX, the proposed DRX could enhance up to 29.7% in the sleep ratio. Also, the sleep ratio in the proposed DRX is higher with the larger beam number. It is essential to design a DRX mechanism with the awareness of beam patterns in mmWave communications. In Fig. 12(b), the packet delay is insensitive to the beam number for both the proposed DRX and the NR DRX mechanism.

C. OPTIMAL DYNAMIC-CONFIGURED BEAM FRAME SETTING
The different curves shown in Fig. 9 serve as the different DF. These curves represent that the performance in the longest DF has the highest sleep ratio with the same packet delay. The DF likely affects the performance of our proposed DRX. The flexible resource in the Dynamic-Configured Beam Frame can improve the QoS for UEs in urgent of data transmission. We set the length of DRX cycle to 400 ms and change the ratio of the DF to the SF (T d F /T s F ) to evaluate the performance. Fig. 13 demonstrates the performance under different ratios of the DF to the SF. While providing the opportunity of transmission for the UEs receiving more packets in the last Static-Configured Beam Frame, the longer DF will cause more serious delay for those whom the gNB does not serve in the Dynamic-Configured Beam Frame. With the increasing DF, the sleep ratio becomes higher, and an optimal packet delay exists. Thus, Fig. 13(a) shows that the sleep ratio increases as the ratio of DF to SF increases. In Fig. 13(b), setting the ratio of DF to SF to 1.5, which means DF is 12 ms and SF is 8 ms, can minimize the packet delay. When the DF is less than 12 ms, the longer Dynamic-Configured Beam Frame can reduce the overall packet delay effectively. When DF is larger than 12 ms, the increasing delay of those whom the gNB does not serve is larger than the reduction of packet delay of those whom the gNB serves in Dynamic-Configured Beam Frame. Therefore, the delay increases with the longer Dynamic-Configured Beam Frame. The length of Dynamic-Configured Beam Frame has a significant impact on the performance of our proposed cross-layer DRX.

IX. CONCLUSIONS
In order to satisfy the high requirement of the network performance in 5G wireless communications, the beamforming technology is widely applied and brings the directionality issue to the wireless communication links. Moreover, massive antenna arrays are used in mmWave bands to form a more narrow beam to overcome the severe path loss. However, such link directionality makes the existing DRX inefficient and wastes precious energy. Because of the directionality, a UE is likely to be ''out-of-beam-coverage'' in its ON Duration when the gNB is serving the UEs on other beams.
In this work, we proposed a dynamic beam-aware DRX framework considering the strategies of gNB serving beam patterns. We use a semi-Markov chain to model the behavior of UEs and validate the model by simulations. The results show that the asynchrony between serving beam patterns and UE wake-up timing significantly impairs the performance of NR DRX. The proposed cross-layer DRX provides up to 41.6% improvement in the sleep ratio without a more considerable packet delay.