This paper aims to reduce the prebuffering requirements, while maintaining continuity, for video streaming. Current approaches do this by making use of adaptive media playout (AMP) to reduce the playout rate. However, this introduces playout distortion to the viewers and increases the viewing latency. We approach this by proposing a frame rate optimization framework that adjusts both the encoder frame generation rate and the decoder playout frame rate. Firstly, we model this problem as the joint adjustment of the encoder frame generation interval and the decoder playout frame interval. This model is used with a discontinuity penalty virtual buffer to track the accumulated difference between the receiving frame interval and the playout frame interval. We then apply Lyapunov optimization to the model to systematically derive a pair of decoupled optimization policies. We show that the occupancy of the discontinuity penalty virtual buffer is correlated to the video discontinuity and that this framework produces a very low playout distortion in addition to a significant reduction in the prebuffering requirements compared to existing approaches. Secondly, we introduced a delay constraint into the framework by using a delay accumulator virtual buffer. Simulation results show that the the delay constrained framework provides a superior tradeoff between the video quality and the delay introduced compared to the existing approach. Finally, we analyzed the impact of delayed feedback between the receiver and the sender on the optimization policies. We show that the delayed feedbacks have a minimal impact on the optimization policies.
@cite_35: We describe the design and the experimental evaluation of a playout delay control mechanism we have developed in order to support unicast, voice-based audio communications over the Internet. The proposed mechanism was designed to dynamically adjust the talkspurt playout delays to the traffic conditions of the underlying network without assuming either the existence of an external mechanism for maintaining an accurate clock synchronization between the sender and the receiver during the audio communication, or a specific distribution of the audio packet transmission delays. Performance figures derived from several experiments are reported that illustrate the adequacy of the proposed mechanism in dynamically adjusting the audio packet playout delay to the network traffic conditions while maintaining a small percentage of packet loss.
@cite_31: As the Internet evolves into a ubiquitous communication infrastructure and provides various services including telephony, it will be expected to meet the quality standards achieved in the public switched telephone network. Our objective in this paper is to assess to what extent today's Internet meets this expectation. Our assessment is based on delay and loss measurements taken over wide-area backbone networks and uses subjective voice quality measures capturing the various impairments incurred. First, we compile the results of various studies into a single model for assessing the voice-over-IP (VoIP) quality. Then, we identify different types of typical Internet paths and study their VoIP performance. For each type of path, we identify those characteristics that affect the VoIP perceived quality. Such characteristics include the network loss and the delay variability that should be appropriately handled by the playout scheduling at the receiver. Our findings indicate that although voice services can be adequately provided by some ISPs, a significant number of Internet backbone paths lead to poor performance.
@cite_33: A concept of waveform similarity for tackling the problem of time-scale modification of speech is proposed. It is worked out in the context of short-time Fourier transform representations. The resulting WSOLA (waveform-similarity-based synchronized overlap-add) algorithm produces high-quality speech output, is algorithmically and computationally efficient and robust, and allows for online processing with arbitrary time-scaling factors that may be specified in a time-varying fashion and can be chosen over a wide continuous range of values. >
@cite_7: The quality of service limitation of today's Internet is a major challenge for real-time voice communications. Excessive delay, packet loss, and high delay jitter all impair the communication quality. A new receiver-based playout scheduling scheme is proposed to improve the tradeoff between buffering delay and late loss for real-time voice communication over IP networks. In this scheme the network delay is estimated from past statistics and the playout time of the voice packets is adaptively adjusted. In contrast to previous work, the adjustment is not only performed between talkspurts, but also within talkspurts in a highly dynamic way. Proper reconstruction of continuous playout speech is achieved by scaling individual voice packets using a time-scale modification technique based on the Waveform Similarity Overlap-Add (WSOLA) algorithm. Results of subjective listening tests show that this operation does not impair audio quality, since the adaptation process requires infrequent scaling of the voice packets and low playout jitter is perceptually tolerable. The same time-scale modification technique is also used to conceal packet loss at very low delay, i.e., one packet time. Simulation results based on Internet measurements show that the tradeoff between buffering delay and late loss can be improved significantly. The overall audio quality is investigated based on subjective listening tests, showing typical gains of 1 on a 5-point scale of the Mean Opinion Score.
@cite_41: 
@cite_23: Delay and packet loss dramatically affect the quality of voice-over-IP (VoIP) calls and depend on the playout buffer scheme implemented at the receiver. The choice of playout algorithm can't be based on statistical metrics without considering the perceived end-to-end conversational speech quality. The authors present a method for evaluating various playout algorithms that extends the E-model concept by estimating user satisfaction from time-varying transmission impairments. This article evaluates several playout algorithms and shows a correspondence between the authors' results and those obtained via statistical loss and delay metrics.
@cite_15: In packet audio applications, packets are buffered at a receiving site and their playout delayed in order to com- pensate for variable network delays. In this paper, we con- sider the problem of adaptively adjusting the playout delay in order to keep this delay as small as possible, while at the same time avoiding excessive "loss" due to the arrival of packets at the receiver after their playout time has al- ready passed. The contributions of this paper are twofold. First, given a trace of packet audio receptions at a receiver, we present efficient algorithms for computing a bound on the achievable performance of any playout delay adjustment algorithm. More precisely, we compute upper and lower bounds (which are shown to be tight for the range of loss and delay values of interest) on the optimum (minimum) average playout delay for a given number of packet losses (due to late arrivals) at the receiver for that trace. Second, we present a new adaptive delay adjustment algorithm that tracks the network delay of recently received packets and efficiently maintains delay percentile information. This in- formation, together with a "delay spike" detection algorithm based on (but extending) our earlier work, is used to dy- namically adjust talkspurt playout delay. We show that this algorithm outperforms existing delay adjustment algorithms over a number of measured audio delay traces and performs close to the theoretical optimum over a range of parameter values of interest.
@cite_11: End-to-end packet delay is an important performance parameter on the Internet, because it heavily affects the quality of real-time applications. Currently, however, because the packet transmission quality (e.g., transmission delay, jitter, packet loss) may vary dynamically, it is not easy to handle real-time traffic. For UDP-based real-time applications, a smoothing buffer (playout buffer) is typically used at the client to compensate for variable delays. The issue of playout control has been studied previously, and several algorithms for controlling the playout buffer have been proposed. These studies considered the network parameters (e.g., packet loss ratio and playout delay), but not the quality perceived by end users. We first clarify the relations between mean opinion score (MOS) of played audio and the network parameters (e.g., packet loss, packet transmission delay, and transmission rate). Then, utilizing the MOS function, we propose a new playout buffer algorithm that considers the user's perceived quality for real-time applications. Our simulation and implementation tests show that the algorithm can enhance the perceived quality more effectively than existing algorithms.
AMP has been studied extensively for audio applications, @cite_15 @cite_35 @cite_31 @cite_7 @cite_11 @cite_23 . These audio AMP techniques depend on audio scaling techniques such as WSOLA @cite_33 , which allows audio to be scaled without changing its pitch. A recent work by has shown that audio and video scaling can be done in real-time @cite_41 . In this paper, we focus on AMP for video.