1. A multi-layer video encoding system, comprising: a base layer encoder to provide a base layer encoded bitstream of a video input; one or more enhancement layer encoders to provide corresponding one or more enhancement layer encoded bitstreams of the video input; an inter-layer analysis module to gather and analyze coding statistics from one or more of the base layer encoder and one or more enhancement layer encoders, provide one or more rate control models, and update the one or more provided rate control models based on the coding statistics, each rate control model being associated with the base layer encoder and/or the one or more enhancement layer encoders; and at least one rate control module to perform rate control on one or more of the base layer encoder and one or more enhancement layer encoders based on the inter-layer analysis module, the at least one rate control module to provide one or more rate control models, each rate control model being associated with base layer encoding by the base layer encoder and/or enhancement layer encoding by the one or more enhancement layer encoders; wherein: rate control models among the one or more rate control models associated with the enhancement layer encoding are updated during the base layer encoding using inter-layer relationships; and each of the one or more rate control models is updated during the enhancement layer encoding using inter-layer relationships.