Title: The Multi-layer Information Bottleneck Problem,
Abstract: The muti-layer information bottleneck (IB) problem, where information is
propagated (or successively refined) from layer to layer, is considered. Based
on information forwarded by the preceding layer, each stage of the network is
required to preserve a certain level of relevance with regards to a specific
hidden variable, quantified by the mutual information. The hidden variables and
the source can be arbitrarily correlated. The optimal trade-off between rates
of relevance and compression (or complexity) is obtained through a
single-letter characterization, referred to as the rate-relevance region.
Conditions of successive refinabilty are given. Binary source with BSC hidden
variables and binary source with BSC/BEC mixed hidden variables are both proved
to be successively refinable. We further extend our result to Guassian models.
A counterexample of successive refinability is also provided.