This paper aims to reduce the prebuffering requirements, while maintaining continuity, for video streaming. Current approaches do this by making use of adaptive media playout (AMP) to reduce the playout rate. However, this introduces playout distortion to the viewers and increases the viewing latency. We approach this by proposing a frame rate optimization framework that adjusts both the encoder frame generation rate and the decoder playout frame rate. Firstly, we model this problem as the joint adjustment of the encoder frame generation interval and the decoder playout frame interval. This model is used with a discontinuity penalty virtual buffer to track the accumulated difference between the receiving frame interval and the playout frame interval. We then apply Lyapunov optimization to the model to systematically derive a pair of decoupled optimization policies. We show that the occupancy of the discontinuity penalty virtual buffer is correlated to the video discontinuity and that this framework produces a very low playout distortion in addition to a significant reduction in the prebuffering requirements compared to existing approaches. Secondly, we introduced a delay constraint into the framework by using a delay accumulator virtual buffer. Simulation results show that the the delay constrained framework provides a superior tradeoff between the video quality and the delay introduced compared to the existing approach. Finally, we analyzed the impact of delayed feedback between the receiver and the sender on the optimization policies. We show that the delayed feedbacks have a minimal impact on the optimization policies.
@cite_18: We study playout buffer management at the receiver for supporting multimedia streaming services over unreliable wireless links. On one hand, memory is a precious resource on portable wireless devices and must be used judiciously. On the other hand, allocating a big playout buffer to a media streaming application reduces the probability of a playout interruption freeze due to buffer underflow, thus improving the user's perceived experience. Thus, inherent in the wireless media streaming scenario is a tradeoff between memory utilization and quality-of-service (QoS) delivered to the application layer. We study this buffer vs. QoS tradeoff in a dynamic programming (DP) framework. Within the same setting, we also address the issue of determining the optimal rebuffering level when the buffer underflows. Leveraging closed form solutions obtained for a special case of our formulation, we propose BuM, a very low complexity heuristic dynamic buffer management algorithm. The near optimality of BuM, as demonstrated by experimental results, in conjunction with its ease of implementation, makes it an attractive option from a practical perspective.
@cite_28: In this paper, we propose a server-client coordinated joint optimal frame selection with adaptive playout for delivering stored video under both limited channel bandwidth and finite client buffer condition. In particular, the client will reduce the frame playout rate whenever its buffer fullness level falls below a given threshold in an attempt to prevent buffer underflow. And when the buffer fullness exceeds the threshold, the client will switch back to the normal frame playout rate. The server is aware of the adaptive playout decision made by the client and incorporates the decision into its optimal frame selection framework for joint optimization. The objective is to minimize the number of frames that must be discarded in order to prevent client buffer underflow or overflow and make full use of the limited bandwidth resource.
@cite_3: Media streaming over wireless links is a challenging problem due to both the unreliable, time-varying nature of the wireless channel and the stringent delivery requirements of media traffic. In this paper, we use joint control of packet scheduling at the transmitter and content-aware playout at the receiver, so as to maximize the quality of media streaming over a wireless link. Our contributions are twofold. First, we formulate and study the problem of joint scheduling and playout control in the framework of Markov decision processes. Second, we propose a novel content-aware adaptive playout control, that takes into account the content of a video sequence, and in particular the motion characteristics of different scenes. We find that the joint scheduling and playout control can significantly improve the quality of the received video, at the expense of only a small amount of playout slowdown. Furthermore, the content-aware adaptive playout places the slowdown preferentially in the low-motion scenes, where its perceived effect is lower.
@cite_13: We propose an application layer algorithm for content-aware adaptive video frame scheduling on bandwidth varying channels. The proposed approach includes an iterative algorithm which sequentially uses the rate constraint and delay constraint for video frames to arrive at a decision to drop send video frames. In addition to our base algorithm, we also propose an explicit deadline adjustment (EDA) algorithm. The main idea behind EDA relates to the sender adjusting postponing the deadline of an important frame which is estimated to miss its deadline by actively dropping one (or more) less important next frame(s) and sending the deadline adjusted postponed frame to be displayed in place of the next dropped frame(s). The rate-distortion (R-D) performance of our proposed approach and its comparison with prior art method is shown.
@cite_20: We propose a new scheme for streaming media systems that combines adaptive media playout (AMP) with rate-distortion (R-D)optimized packet transmission. AMP, the client-controlled, adaptive modification of the media playout rate, allows us to flexibly adjust the playout deadlines of individual packets, and can therefore reduce reconstruction distortion. This added flexibility incurs a subjective cost, however. We introduce functions that assess the subjective cost of a schedule of playout rate modifications, and we show how to optimize the schedule with respect to these costs and distortion. Because the optimal playout rate schedule and the R-D optimal transmission schedule are interdependent, we solve the two problems jointly. In simulations that model a receiver-driven scenario, results for a short media clip show a more than 2 dB improvement in mean PSNR for R-D optimized transmission scheduling combined with a moderate amount of AMP, over R-D optimized transmission scheduling alone.
AMP has also been integrated into the design of packet schedulers @cite_20 @cite_28 @cite_18 @cite_13 @cite_3 . These techniques tend to slowdown the playout rate for important video packets. This ensures that the more important video packets have a higher probability of meeting its deadline and thus avoid being dropped. We do not focus on packet scheduling in this paper and our proposed framework could complement any packet scheduling scheme.