Title: Mutual Information, Relative Entropy and Estimation Error in Semi-martingale Channels,
Abstract: Fundamental relations between information and estimation have been
established in the literature for the continuous-time Gaussian and Poisson
channels, in a long line of work starting from the classical representation
theorems by Duncan and Kabanov respectively. In this work, we demonstrate that
such relations hold for a much larger family of continuous-time channels. We
introduce the family of semi-martingale channels where the channel output is a
semi-martingale stochastic process, and the channel input modulates the
characteristics of the semi-martingale. For these channels, which includes as a
special case the continuous time Gaussian and Poisson models, we establish new
representations relating the mutual information between the channel input and
output to an optimal causal filtering loss, thereby unifying and considerably
extending results from the Gaussian and Poisson settings. Extensions to the
setting of mismatched estimation are also presented where the relative entropy
between the laws governing the output of the channel under two different input
distributions is equal to the cumulative difference between the estimation loss
incurred by using the mismatched and optimal causal filters respectively. The
main tool underlying these results is the Doob--Meyer decomposition of a class
of likelihood ratio sub-martingales. The results in this work can be viewed as
the continuous-time analogues of recent generalizations for relations between
information and estimation for discrete-time Lévy channels.